[Perf] Linux/x64: 454 Regressions on 9/26/2023 5:24:05 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
CachedJsonSerializerContext - Duration of single invocation 📈 - ADX Test Multi Config Graph	4.34 μs	5.10 μs	1.18	0.20	False
NewCustomConverter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	241.36 μs	258.09 μs	1.07	0.04	False
NewDefaultOptions - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	8.37 μs	9.40 μs	1.12	0.14	False
NewCustomizedOptions - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	8.57 μs	9.84 μs	1.15	0.11	False
NewCachedCustomConverter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	9.62 μs	10.87 μs	1.13	0.11	False
CachedDefaultOptions - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.49 μs	6.69 μs	1.22	0.13	False
NewJsonSerializerContext - Duration of single invocation 📈 - ADX Test Multi Config Graph	7.24 μs	8.05 μs	1.11	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedJsonSerializerContext

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.100523353942944 > 4.4702239647536155.
IsChangePoint: Marked as a change because one of 8/21/2023 10:18:47 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.601865147504437 (T) = (0 -5004.356264736374) / Math.Sqrt((13090.965074921933 / (51)) + (11369.307920302243 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17828377012597793 = (4247.157086956503 - 5004.356264736374) / 4247.157086956503 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomConverter

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 258.0935465286832 > 250.6117123201885.
IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 9/25/2023 10:28:34 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.125609786452841 (T) = (0 -256062.12686409106) / Math.Sqrt((15271883.973650124 / (46)) + (28406660.998105086 / (8))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (8) - 2, .025) and -0.07552883728335538 = (238080.20574406016 - 256062.12686409106) / 238080.20574406016 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewDefaultOptions

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.39675822790876 > 8.753806682573117.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.345817462796044 (T) = (0 -9644.008927213637) / Math.Sqrt((55536.523787042395 / (50)) + (27969.413307408977 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.15460613477651597 = (8352.639603010874 - 9644.008927213637) / 8352.639603010874 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomizedOptions

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.844645757831696 > 9.011393570361816.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.574256900191573 (T) = (0 -9947.613130381598) / Math.Sqrt((38030.29283586491 / (50)) + (43208.476843960874 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.14300546605650022 = (8703.03198522926 - 9947.613130381598) / 8703.03198522926 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCachedCustomConverter

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.874584178946192 > 9.858509310326363.
IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -17.31964104446051 (T) = (0 -11006.718833541054) / Math.Sqrt((71108.33539221596 / (50)) + (18641.102206751137 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.15942348460916644 = (9493.26883546036 - 11006.718833541054) / 9493.26883546036 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedDefaultOptions

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.690192466813368 > 5.791846481442025.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -16.3854792821721 (T) = (0 -6860.241281141385) / Math.Sqrt((22187.885913897528 / (51)) + (27715.5753048464 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.25780288484532726 = (5454.146562865447 - 6860.241281141385) / 5454.146562865447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewJsonSerializerContext

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.048988382841117 > 7.368679430049158.
IsChangePoint: Marked as a change because one of 8/22/2023 10:50:09 AM, 9/25/2023 2:54:53 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.786029631897627 (T) = (0 -7735.024634582908) / Math.Sqrt((27701.675285527694 / (42)) + (37432.59730869336 / (12))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (12) - 2, .025) and -0.08433201875563279 = (7133.446675732711 - 7735.024634582908) / 7133.446675732711 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Take_All - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	26.27 μs	43.76 μs	1.67	0.25	True
Sum - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.97 μs	3.46 μs	1.17	0.23	False
Count - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.06 μs	1.23 μs	1.16	0.27	False
TakeLastHalf - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	20.73 μs	30.83 μs	1.49	0.14	True
Concat_TenTimes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	72.75 μs	80.80 μs	1.11	0.28	False
Where - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	23.66 μs	37.56 μs	1.59	0.24	True
Reverse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	21.92 μs	30.90 μs	1.41	0.11	False
WhereAny_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	11.92 μs	17.64 μs	1.48	0.29	False
SelectToList - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	24.25 μs	36.58 μs	1.51	0.15	True
Select - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	15.59 μs	23.48 μs	1.51	0.34	True
Min - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.43 μs	2.67 μs	1.10	0.29	False
SingleWithPredicate_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.24 μs	3.55 μs	1.10	0.48	False
WhereSingle_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	14.11 μs	18.48 μs	1.31	0.22	False
SelectToArray - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	15.38 μs	23.03 μs	1.50	0.26	False
SelectToList - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	23.05 μs	35.70 μs	1.55	0.19	True
OrderByThenBy - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	85.64 μs	143.83 μs	1.68	0.15	True
WhereSelect - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	22.90 μs	37.27 μs	1.63	0.24	True
SingleWithPredicate_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.27 μs	3.59 μs	1.10	0.46	False
CastToSameType - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.93 μs	4.28 μs	1.09	0.37	False
WhereLast_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	12.73 μs	17.35 μs	1.36	0.40	False
WhereFirst_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	7.90 μs	10.46 μs	1.32	0.34	False
SingleWithPredicate_FirstElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.16 μs	3.89 μs	1.23	0.39	False
Except - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	22.42 μs	28.51 μs	1.27	0.27	False
SelectToList - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	13.91 μs	21.58 μs	1.55	0.26	True
Where - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	14.97 μs	22.91 μs	1.53	0.30	True
SelectToArray - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	23.84 μs	36.93 μs	1.55	0.20	True
FirstWithPredicate_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.18 μs	3.59 μs	1.13	0.33	False
Intersect - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	40.93 μs	63.44 μs	1.55	0.12	True
Concat_Once - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	13.05 μs	14.07 μs	1.08	0.36	False
AnyWithPredicate_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	79.69 μs	144.06 μs	1.81	0.09	True
SelectToArray - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	23.49 μs	37.01 μs	1.58	0.24	False
SkipHalfTakeHalf - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	20.74 μs	29.14 μs	1.40	0.14	True
LastWithPredicate_FirstElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.37 μs	4.03 μs	1.20	0.49	False
LastWithPredicate_FirstElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	22.63 μs	31.62 μs	1.40	0.23	True
GroupBy - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	35.31 μs	48.57 μs	1.38	0.14	False
AnyWithPredicate_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.22 μs	3.56 μs	1.11	0.38	False
WhereAny_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	8.18 μs	11.59 μs	1.42	0.21	False
Distinct - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	26.90 μs	40.84 μs	1.52	0.18	True
Zip - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	41.61 μs	65.83 μs	1.58	0.10	True
Append - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	39.52 μs	57.60 μs	1.46	0.15	True
AppendPrepend - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	37.10 μs	55.91 μs	1.51	0.13	False
Skip_One - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	26.60 μs	44.60 μs	1.68	0.34	True
SelectToArray - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	24.89 μs	37.01 μs	1.49	0.22	False
WhereSelect - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	19.35 μs	28.74 μs	1.49	0.18	True
SelectToList - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	19.35 μs	28.32 μs	1.46	0.22	True
Select - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	18.67 μs	28.98 μs	1.55	0.20	True
ElementAt - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	585.97 ns	641.70 ns	1.10	0.17	False
Where - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	20.05 μs	29.88 μs	1.49	0.31	True
Contains_ElementNotFound - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.28 μs	3.74 μs	1.14	0.37	False
ToArray - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.45 μs	5.17 μs	1.16	0.44	False
SelectToArray - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	19.55 μs	29.11 μs	1.49	0.28	True
WhereSingleOrDefault_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	8.69 μs	11.04 μs	1.27	0.27	True
FirstWithPredicate_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	82.89 μs	142.72 μs	1.72	0.14	True
WhereSingle_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	8.27 μs	11.47 μs	1.39	0.41	False
WhereSingleOrDefault_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	11.97 μs	17.50 μs	1.46	0.32	False
OrderByDescending - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	121.26 μs	215.41 μs	1.78	0.09	True
ToDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	9.75 μs	10.58 μs	1.09	0.26	False
WhereLast_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	8.07 μs	10.89 μs	1.35	0.31	False
TakeLastHalf - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	26.52 μs	42.08 μs	1.59	0.19	True
WhereSelect - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	28.24 μs	44.28 μs	1.57	0.34	False
SelectToList - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	22.72 μs	35.09 μs	1.54	0.23	False
WhereFirst_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	13.01 μs	17.58 μs	1.35	0.21	False
OrderBy - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	104.08 μs	180.53 μs	1.73	0.15	True
Prepend - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	33.94 μs	47.85 μs	1.41	0.10	True
Select - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	23.39 μs	39.34 μs	1.68	0.20	True
SingleWithPredicate_FirstElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.19 μs	3.73 μs	1.17	0.40	False
EmptyTakeSelectToArray - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	811.43 ns	953.48 ns	1.18	0.14	False
Average - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.18 μs	2.34 μs	1.07	0.28	False
FirstWithPredicate_LastElementMatches - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.89 μs	4.38 μs	1.13	0.36	False
Select - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	23.33 μs	37.67 μs	1.61	0.22	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

System.Linq.Tests.Perf_Enumerable.Take_All(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.76206043412476 > 27.08793556935937.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -74.2542243605858 (T) = (0 -43438.97635877944) / Math.Sqrt((910394.1843902959 / (50)) + (144610.28857807736 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.6626493978853887 = (26126.359780977597 - 43438.97635877944) / 26126.359780977597 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.461064707442777 > 3.0780579850091887.
IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.202679942726265 (T) = (0 -3356.5226912469116) / Math.Sqrt((6231.4163955119875 / (39)) + (24341.345921929733 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.13566728668106126 = (2955.551093715312 - 3356.5226912469116) / 2955.551093715312 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2303460733303815 > 1.1144408040877207.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.160172623755372 (T) = (0 -1177.8693979253803) / Math.Sqrt((766.4118229108827 / (39)) + (2420.4468697071748 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.12096501756782634 = (1050.7637432620513 - 1177.8693979253803) / 1050.7637432620513 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: List)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.831156996976937 > 22.058564798910602.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -25.24202569051479 (T) = (0 -31159.692765255146) / Math.Sqrt((351677.563443753 / (49)) + (573747.0952101754 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.45854713686871235 = (21363.514402524183 - 31159.692765255146) / 21363.514402524183 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Concat_TenTimes(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 80.79634321853743 > 76.51089044567078.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.752219359379486 (T) = (0 -80930.59171546262) / Math.Sqrt((7943207.796992523 / (41)) + (5171008.334697896 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.12229378268501238 = (72111.77052219039 - 80930.59171546262) / 72111.77052219039 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Where(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.56334775452044 > 25.361782980570073.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -24.408880131853337 (T) = (0 -36654.06647312567) / Math.Sqrt((1500140.2692960585 / (51)) + (720585.7203407387 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4385306975224717 = (25480.211535460185 - 36654.06647312567) / 25480.211535460185 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Reverse(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.901252472167652 > 22.986836955996388.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -25.33851581218601 (T) = (0 -31565.1155848793) / Math.Sqrt((1084384.982221772 / (48)) + (316867.6969051189 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.34432612310566857 = (23480.25158654019 - 31565.1155848793) / 23480.25158654019 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.643105712256492 > 12.515566964143327.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -10.641973357740639 (T) = (0 -17473.113020120258) / Math.Sqrt((885056.2448162014 / (49)) + (405566.5427062642 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.26662455797646695 = (13795.021508216247 - 17473.113020120258) / 13795.021508216247 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.57618685302516 > 24.889329721143895.
IsChangePoint: Marked as a change because one of 9/19/2023 12:42:36 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -31.098682301025246 (T) = (0 -36469.286332587595) / Math.Sqrt((1626872.6465525217 / (49)) + (403661.94056934846 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4540990245910753 = (25080.332024047373 - 36469.286332587595) / 25080.332024047373 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Select(input: Array)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.479846926627797 > 15.611513204295287.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.787243642833777 (T) = (0 -21884.175549899195) / Math.Sqrt((436660.17365650315 / (49)) + (1407247.2297986026 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4781882128926673 = (14804.72876121373 - 21884.175549899195) / 14804.72876121373 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.669029827235772 > 2.654861871828226.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -5.205413586087894 (T) = (0 -2827.176152422456) / Math.Sqrt((37212.49674999803 / (41)) + (28453.011108724724 / (13))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (13) - 2, .025) and -0.11414785990306257 = (2537.523298450205 - 2827.176152422456) / 2537.523298450205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5529979918890633 > 3.3901385546303953.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.344887605863168 (T) = (0 -3717.8102459368297) / Math.Sqrt((27216.972212802757 / (38)) + (21879.711345446973 / (14))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (14) - 2, .025) and -0.13635817723935978 = (3271.6887337131548 - 3717.8102459368297) / 3271.6887337131548 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.476456690081246 > 12.846855992969262.
IsChangePoint: Marked as a change because one of 9/19/2023 5:34:53 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -24.018582309514162 (T) = (0 -18400.353233933736) / Math.Sqrt((1156820.249256477 / (49)) + (39413.41690814158 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.31367949351367946 = (14006.729438029499 - 18400.353233933736) / 14006.729438029499 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.026231053841997 > 16.7925682710665.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.359864221862686 (T) = (0 -23053.707234668494) / Math.Sqrt((657840.117307801 / (48)) + (364481.8182317971 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.45777530325249177 = (15814.307721658193 - 23053.707234668494) / 15814.307721658193 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.70240273646071 > 24.575215389808772.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -32.53207622949979 (T) = (0 -36188.37612303959) / Math.Sqrt((1802479.7646629268 / (48)) + (305569.093576691 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.43564877554324144 = (25206.98428440209 - 36188.37612303959) / 25206.98428440209 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.OrderByThenBy(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 143.83383472222224 > 90.01589252285626.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -38.430749466552875 (T) = (0 -143871.00896208175) / Math.Sqrt((6938528.097746182 / (49)) + (7834521.387566977 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.6316258056974703 = (88176.47309799766 - 143871.00896208175) / 88176.47309799766 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSelect(input: List)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.27142717391304 > 25.679231347071482.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -49.28487276318413 (T) = (0 -37915.778041228354) / Math.Sqrt((641158.7630268905 / (48)) + (259370.44512512864 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.5710771710038521 = (24133.619112421937 - 37915.778041228354) / 24133.619112421937 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: Array)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5936665507286008 > 3.405322034573069.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.346782555975985 (T) = (0 -3644.3106022727834) / Math.Sqrt((20396.138594750864 / (40)) + (18871.54445954364 / (13))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (13) - 2, .025) and -0.1281486256321637 = (3230.3461791044383 - 3644.3106022727834) / 3230.3461791044383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.276155832867698 > 4.0980790694017655.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -4.765731698138736 (T) = (0 -4162.542562256385) / Math.Sqrt((21992.268898310405 / (39)) + (54529.776994683154 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.08277979489133076 = (3844.31126429926 - 4162.542562256385) / 3844.31126429926 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.346282398148144 > 13.367018013506051.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -10.289483231557856 (T) = (0 -14990.124468688577) / Math.Sqrt((255677.75200406162 / (12)) + (1364015.3061347327 / (42))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (42) - 2, .025) and -0.1893268443194745 = (12603.872972585457 - 14990.124468688577) / 12603.872972585457 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: List)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.460952363095236 > 8.429306393703737.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.428271443973369 (T) = (0 -10830.911957334296) / Math.Sqrt((140151.3665712502 / (50)) + (158566.24962209963 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.34305107540407676 = (8064.408089674219 - 10830.911957334296) / 8064.408089674219 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.8880023763987204 > 3.418196828669158.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.601877753375796 (T) = (0 -3727.342526279697) / Math.Sqrt((10716.348588249934 / (40)) + (34890.27135670842 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.15652012497166465 = (3222.8946524999023 - 3727.342526279697) / 3222.8946524999023 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Except(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.511025076034063 > 22.370361168180892.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.28630288424859 (T) = (0 -28124.284095600065) / Math.Sqrt((1642342.9574806513 / (49)) + (919713.6123171696 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.20405835646586862 = (23357.907816154344 - 28124.284095600065) / 23357.907816154344 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: Array)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.576718644446647 > 15.244814397868563.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.069877936101438 (T) = (0 -21399.747758961523) / Math.Sqrt((403175.5839128292 / (51)) + (197863.3031697682 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4346945136003466 = (14915.891540742805 - 21399.747758961523) / 14915.891540742805 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Where(input: Array)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.90719487958989 > 15.866240581280893.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -15.41845579379482 (T) = (0 -21789.313390872343) / Math.Sqrt((404593.0496691293 / (49)) + (690501.0410598568 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4305142233482421 = (15231.80478406749 - 21789.313390872343) / 15231.80478406749 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.92983891454966 > 25.410630941389098.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -23.981951819679242 (T) = (0 -36692.200350321495) / Math.Sqrt((2153402.43908407 / (48)) + (606407.2542421686 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.4078613469363985 = (26062.367881727987 - 36692.200350321495) / 26062.367881727987 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: Array)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.588650563362477 > 3.244491137293287.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.256128578514847 (T) = (0 -3687.403361054065) / Math.Sqrt((49666.461224947096 / (39)) + (62107.0850524346 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.14705377543802936 = (3214.673487863238 - 3687.403361054065) / 3214.673487863238 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Intersect(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.44445614601262 > 42.533671851537996.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -23.433203429770177 (T) = (0 -62702.31125756461) / Math.Sqrt((2494020.0098005105 / (51)) + (2778774.829866138 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4755086364005315 = (42495.38749601996 - 62702.31125756461) / 42495.38749601996 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Concat_Once(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.071042196046525 > 13.658043888648368.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.355585748342514 (T) = (0 -14251.697559256872) / Math.Sqrt((291491.6388248227 / (41)) + (498052.8077004989 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.10148554083988212 = (12938.615198153178 - 14251.697559256872) / 12938.615198153178 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IOrderedEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 144.06003074845677 > 84.55828063710584.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -36.172836961311454 (T) = (0 -140726.12708347838) / Math.Sqrt((3854355.578777275 / (50)) + (10403579.233494595 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.726045665791319 = (81530.94085083861 - 140726.12708347838) / 81530.94085083861 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Range)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.00678122131449 > 24.4907743731737.
IsChangePoint: Marked as a change because one of 9/19/2023 3:42:14 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.939514654214644 (T) = (0 -36733.10610632607) / Math.Sqrt((1803154.944619166 / (51)) + (1075336.610973507 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4273166602817443 = (25735.778981992098 - 36733.10610632607) / 25735.778981992098 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SkipHalfTakeHalf(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.142478805152013 > 21.190985479807896.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.708876762106673 (T) = (0 -29129.20275076581) / Math.Sqrt((451303.9480062207 / (51)) + (600752.3804812928 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.45116168199281786 = (20073.023641834265 - 29129.20275076581) / 20073.023641834265 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.0337373294679395 > 3.533903643299506.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.138472177804148 (T) = (0 -3930.0844677136115) / Math.Sqrt((69193.78319642779 / (40)) + (92785.58885574824 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.16657343291734433 = (3368.913054950456 - 3930.0844677136115) / 3368.913054950456 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IOrderedEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.61644287736646 > 22.399106047149008.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -42.6608042676181 (T) = (0 -31961.812317655633) / Math.Sqrt((588771.3121040417 / (48)) + (230026.64963264205 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.5445591287947459 = (20693.16203038219 - 31961.812317655633) / 20693.16203038219 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.GroupBy(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.572961777605784 > 37.28593647679438.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.21243312056073 (T) = (0 -49542.85212505012) / Math.Sqrt((1840502.2386760355 / (48)) + (1084703.2696849143 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.33233150422967245 = (37185.078914496444 - 49542.85212505012) / 37185.078914496444 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5577632269061827 > 3.36195928127845.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 2:54:53 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.222879369168745 (T) = (0 -3728.3847415477694) / Math.Sqrt((75999.8658569623 / (39)) + (24105.0211345211 / (13))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (13) - 2, .025) and -0.1356764213116515 = (3282.9639425300957 - 3728.3847415477694) / 3282.9639425300957 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: List)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.587355902011282 > 8.397343125318681.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.877778586745357 (T) = (0 -11115.791848976802) / Math.Sqrt((59086.135047270356 / (51)) + (200084.13184289835 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.39361386174749474 = (7976.2351352033575 - 11115.791848976802) / 7976.2351352033575 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Distinct(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.83852018900344 > 28.711216911932862.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -49.615875519998745 (T) = (0 -40565.84596661838) / Math.Sqrt((1563401.6778637765 / (51)) + (98265.86390815648 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.40334709285271664 = (28906.49517373235 - 40565.84596661838) / 28906.49517373235 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Zip(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 65.8320785801964 > 44.64310059144376.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.897515723653584 (T) = (0 -66557.8139966394) / Math.Sqrt((3611182.3922664574 / (48)) + (2095227.4670445535 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.4801878503277676 = (44965.788620613974 - 66557.8139966394) / 44965.788620613974 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Append(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.59587962962962 > 41.442765578391345.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -32.08556753091047 (T) = (0 -56816.63820664982) / Math.Sqrt((1939851.7342994034 / (49)) + (902638.8002952867 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.41012771436892587 = (40291.83855313202 - 56816.63820664982) / 40291.83855313202 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.AppendPrepend(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.90526390865955 > 39.47167184382005.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.09257903898907 (T) = (0 -56571.271235419605) / Math.Sqrt((1332098.0685740465 / (50)) + (3056727.4312971737 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.48593301276946815 = (38071.2123287325 - 56571.271235419605) / 38071.2123287325 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Skip_One(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.60085978835979 > 27.38856383898073.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -17.6694369497474 (T) = (0 -45039.684624283975) / Math.Sqrt((863283.4853734003 / (49)) + (4628641.965611767 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.7397714552589492 = (25888.276582615978 - 45039.684624283975) / 25888.276582615978 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.01214413711289 > 25.558194615066228.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.16273381294369 (T) = (0 -37005.92563166397) / Math.Sqrt((2433776.4514426277 / (51)) + (1084246.790660447 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4131690157014257 = (26186.482452204134 - 37005.92563166397) / 26186.482452204134 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSelect(input: Array)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.7429102233367 > 20.556257185034514.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.859598857067304 (T) = (0 -29424.84741596278) / Math.Sqrt((375019.30648682645 / (51)) + (463125.3215764435 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4975420747840171 = (19648.76173526315 - 29424.84741596278) / 19648.76173526315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: List)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.321913031193485 > 20.491717965418307.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.671744158585852 (T) = (0 -29299.034839399905) / Math.Sqrt((487800.09991073655 / (51)) + (484625.3447031826 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.5185399025551324 = (19294.214653234092 - 29299.034839399905) / 19294.214653234092 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Select(input: List)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.982183507351024 > 20.04558936273995.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -41.433861671848 (T) = (0 -29314.578539816124) / Math.Sqrt((541060.8524071439 / (51)) + (185019.60869243104 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.5084021316668105 = (19434.19325947452 - 29314.578539816124) / 19434.19325947452 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ElementAt(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 641.7013134152994 > 617.9770864612199.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.34227240744542 (T) = (0 -662.4200660801408) / Math.Sqrt((575.3849787309966 / (40)) + (768.1091467968092 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.10160921445326186 = (601.3203751285846 - 662.4200660801408) / 601.3203751285846 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Where(input: List)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.877273670708956 > 20.49590227320937.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -30.01515099544161 (T) = (0 -29739.371409760548) / Math.Sqrt((670129.5655333118 / (51)) + (392767.88206326665 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.5077459486261888 = (19724.391524220733 - 29739.371409760548) / 19724.391524220733 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7371928257207836 > 3.4480265826239695.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.413588730844547 (T) = (0 -3695.967785277001) / Math.Sqrt((18397.0642580543 / (40)) + (14225.593360548532 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.10846736213183071 = (3334.304564609668 - 3695.967785277001) / 3334.304564609668 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ToArray(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.171158813996923 > 4.687636447140868.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -5.729399778741045 (T) = (0 -5040.292514365142) / Math.Sqrt((111646.64525699346 / (39)) + (36478.01551266873 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.09177236601587037 = (4616.61484688272 - 5040.292514365142) / 4616.61484688272 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: List)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.10818589786382 > 20.67806879807137.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -30.283290965395366 (T) = (0 -29664.37090830939) / Math.Sqrt((739492.836441703 / (49)) + (328374.7584493203 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4668133215134621 = (20223.685231943222 - 29664.37090830939) / 20223.685231943222 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: List)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.039823999885373 > 8.51648768450228.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -23.654393625421694 (T) = (0 -11162.51564773883) / Math.Sqrt((95383.24435445196 / (51)) + (54763.847185854036 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3593333800320427 = (8211.757183124351 - 11162.51564773883) / 8211.757183124351 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IOrderedEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 142.7243611566186 > 88.07389590967098.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -73.39700586309381 (T) = (0 -140916.32499053856) / Math.Sqrt((5308906.855866136 / (51)) + (2125804.048679288 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.7100802681480753 = (82403.33954799872 - 140916.32499053856) / 82403.33954799872 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: List)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.472360096504516 > 8.608520573312038.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -10.764596903736063 (T) = (0 -11058.258889303806) / Math.Sqrt((240575.7366709494 / (52)) + (238693.7127718254 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.327739704689673 = (8328.634633915995 - 11058.258889303806) / 8328.634633915995 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.501658142389527 > 12.920579773364762.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 10:37:55 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -15.560828821742945 (T) = (0 -17301.027162161645) / Math.Sqrt((1359488.6062361342 / (50)) + (78493.27489592788 / (5))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (5) - 2, .025) and -0.22890106768890411 = (14078.454008261464 - 17301.027162161645) / 14078.454008261464 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.OrderByDescending(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 215.40741946347032 > 126.48668228944291.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -24.056579223748468 (T) = (0 -205266.2814547425) / Math.Sqrt((10794997.38947296 / (51)) + (46729497.215968676 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.6783753634071928 = (122300.58062699446 - 205266.2814547425) / 122300.58062699446 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ToDictionary(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.577860118038739 > 10.383694227555335.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 2:54:53 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -8.05126267860194 (T) = (0 -10654.365847060273) / Math.Sqrt((81302.7337924194 / (42)) + (89311.99702885175 / (12))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (12) - 2, .025) and -0.07896000624162879 = (9874.662439225083 - 10654.365847060273) / 9874.662439225083 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: List)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.88501350508779 > 8.488959773748554.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -28.771994626531598 (T) = (0 -10834.046542638762) / Math.Sqrt((72023.50637140354 / (51)) + (28176.467626221627 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3231258577312215 = (8188.22070427678 - 10834.046542638762) / 8188.22070427678 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.07745359078591 > 28.89398553665037.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -38.90414697938277 (T) = (0 -42556.88397765576) / Math.Sqrt((1480881.112586243 / (50)) + (365108.33637605 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.46596503645006454 = (29029.944725496447 - 42556.88397765576) / 29029.944725496447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSelect(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.28297622619047 > 29.810769813567937.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -36.70236601444275 (T) = (0 -44735.31526592581) / Math.Sqrt((2426558.085195251 / (50)) + (327427.8780295966 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.48319356594088797 = (30161.48147699605 - 44735.31526592581) / 30161.48147699605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: Range)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.08753622707529 > 24.202216475690204.
IsChangePoint: Marked as a change because one of 9/18/2023 8:11:22 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.491092821461123 (T) = (0 -35617.30824449749) / Math.Sqrt((1632027.4359475705 / (50)) + (1556628.609072675 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.4329681288915013 = (24855.617878989335 - 35617.30824449749) / 24855.617878989335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.581573674636175 > 13.376868558097277.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.948971065848039 (T) = (0 -14594.324025080312) / Math.Sqrt((732282.6878244624 / (13)) + (888359.3476733258 / (41))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (41) - 2, .025) and -0.15337121145431476 = (12653.622597947422 - 14594.324025080312) / 12653.622597947422 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.OrderBy(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 180.53089080459768 > 96.39466979656173.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.979989448478658 (T) = (0 -163321.6984176039) / Math.Sqrt((9951788.178472683 / (51)) + (132078932.51668848 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.7323376032242039 = (94278.2158129176 - 163321.6984176039) / 94278.2158129176 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Prepend(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 47.85424316206543 > 35.8755750234319.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -60.24469324827494 (T) = (0 -47676.78791229351) / Math.Sqrt((613579.3576488794 / (51)) + (154867.31912179515 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.39792891336448405 = (34105.302105488874 - 47676.78791229351) / 34105.302105488874 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Select(input: IList)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.33882092052923 > 24.78844615338115.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -12.818125040197165 (T) = (0 -36924.22618181848) / Math.Sqrt((1511828.0389695375 / (52)) + (3264772.398365989 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.46877629128844145 = (25139.448669496 - 36924.22618181848) / 25139.448669496 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: Array)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7261376878840724 > 3.2997881585693793.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -5.063866758183407 (T) = (0 -3627.319177997099) / Math.Sqrt((56596.686900091765 / (41)) + (49920.297021658655 / (13))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (13) - 2, .025) and -0.11218266459789437 = (3261.441931670948 - 3627.319177997099) / 3261.441931670948 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 953.4792467340919 > 831.0403509077438.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.852885087533757 (T) = (0 -954.7326370168043) / Math.Sqrt((695.295403697499 / (51)) + (485.3775237182028 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.22061723651583015 = (782.1720097464988 - 954.7326370168043) / 782.1720097464988 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Average(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.338599212831311 > 2.3050190059031515.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -4.200578578511119 (T) = (0 -2535.151235753744) / Math.Sqrt((4816.829620242058 / (38)) + (85203.41908012494 / (14))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (14) - 2, .025) and -0.15021967768331834 = (2204.0583072442696 - 2535.151235753744) / 2204.0583072442696 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: List)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.3810469520565345 > 4.013469142047674.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.032321652909785 (T) = (0 -4406.428556044247) / Math.Sqrt((31484.74019444757 / (41)) + (71173.56673156975 / (13))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (13) - 2, .025) and -0.14429073377069732 = (3850.7945804333012 - 4406.428556044247) / 3850.7945804333012 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Select(input: IEnumerable)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.66950908739365 > 24.40659448699244.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.37300701453872 (T) = (0 -36875.14873724371) / Math.Sqrt((1752195.4816866303 / (51)) + (957232.8494257272 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.46497626490953897 = (25171.157799966622 - 36875.14873724371) / 25171.157799966622 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Reflection.Attributes

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
IsDefinedClassHitInherit - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	610.97 ns	718.77 ns	1.18	0.11	False
IsDefinedMethodOverrideMiss - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	803.99 ns	934.87 ns	1.16	0.11	False
GetCustomAttributesClassMiss - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.42 μs	1.57 μs	1.11	0.12	False
IsDefinedMethodOverrideHit - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	859.24 ns	1.04 μs	1.21	0.13	False
GetCustomAttributesMethodOverrideMiss - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.98 μs	2.25 μs	1.14	0.41	False
GetCustomAttributesMethodBaseHitInherit - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.33 μs	3.68 μs	1.10	0.24	False
IsDefinedClassHit - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	623.30 ns	711.10 ns	1.14	0.08	False
IsDefinedClassMiss - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	433.05 ns	526.44 ns	1.22	0.10	False
IsDefinedMethodOverrideHitInherit - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	867.40 ns	985.59 ns	1.14	0.18	False
GetCustomAttributesMethodBaseHit - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.97 μs	2.32 μs	1.18	0.39	False
IsDefinedMethodOverrideMissInherit - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	816.59 ns	983.60 ns	1.20	0.13	False
IsDefinedClassMissInherit - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.15 μs	2.62 μs	1.22	0.05	False
GetCustomAttributesMethodOverrideHit - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.89 μs	2.17 μs	1.15	0.37	False
IsDefinedMethodBaseMissInherit - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.22 μs	1.37 μs	1.12	0.06	False
IsDefinedMethodBaseHitInherit - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	815.38 ns	982.72 ns	1.21	0.20	False
IsDefinedMethodBaseHit - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	803.28 ns	1.03 μs	1.28	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'

Payloads

Baseline
Compare

System.Reflection.Attributes.IsDefinedClassHitInherit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 718.7712781467 > 644.7267929617442.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -21.31970628153255 (T) = (0 -710.9536859360055) / Math.Sqrt((245.4513616399054 / (52)) + (45.91538693115182 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.13726023500823298 = (625.1459991748139 - 710.9536859360055) / 625.1459991748139 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.IsDefinedMethodOverrideMiss

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 934.8672874098556 > 849.8927145431103.
IsChangePoint: Marked as a change because one of 8/31/2023 1:23:15 PM, 9/21/2023 9:38:47 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -10.165914186337531 (T) = (0 -889.911836257299) / Math.Sqrt((402.9892332774793 / (22)) + (992.5597333454303 / (31))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (31) - 2, .025) and -0.08819510663977943 = (817.7870225912371 - 889.911836257299) / 817.7870225912371 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.GetCustomAttributesClassMiss

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.57008314755079 > 1.4850448831375684.
IsChangePoint: Marked as a change because one of 8/22/2023 3:00:25 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.197523823645344 (T) = (0 -1547.129218962365) / Math.Sqrt((1578.9514353264037 / (49)) + (397.87515689258163 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.07321702132183376 = (1441.5809554127595 - 1547.129218962365) / 1441.5809554127595 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.IsDefinedMethodOverrideHit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.037579984995918 > 922.3621358354543.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -5.923164134863644 (T) = (0 -1014.0184338844558) / Math.Sqrt((660.8530807792829 / (50)) + (1785.2073000546397 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.14313924983241244 = (887.0471677296654 - 1014.0184338844558) / 887.0471677296654 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.GetCustomAttributesMethodOverrideMiss

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.254950019981491 > 2.1869085798256744.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -5.333474163513746 (T) = (0 -2304.873276874243) / Math.Sqrt((6089.330904064179 / (49)) + (5022.626007508424 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.09404284878172736 = (2106.7486336945917 - 2304.873276874243) / 2106.7486336945917 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.GetCustomAttributesMethodBaseHitInherit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.677216753636939 > 3.4565837430047996.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 4:14:13 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.787761210664045 (T) = (0 -3531.482609824832) / Math.Sqrt((9608.388664752438 / (44)) + (12074.68594253818 / (10))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (10) - 2, .025) and -0.07825780320497824 = (3275.1746375755115 - 3531.482609824832) / 3275.1746375755115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.IsDefinedClassHit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 711.0957521833698 > 653.4941497185288.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -24.99175840574524 (T) = (0 -708.4536554051042) / Math.Sqrt((198.9035461885404 / (50)) + (27.839507553392366 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.13207771032807525 = (625.7994914499243 - 708.4536554051042) / 625.7994914499243 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.IsDefinedClassMiss

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 526.4352008195274 > 454.8056423141701.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -28.162386247654094 (T) = (0 -519.9757691475002) / Math.Sqrt((86.38802398733253 / (51)) + (28.527616517506726 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.19175676283883145 = (436.3103154614275 - 519.9757691475002) / 436.3103154614275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.IsDefinedMethodOverrideHitInherit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 985.589623810468 > 917.4118056052125.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -17.620782283896613 (T) = (0 -992.5398721246866) / Math.Sqrt((509.84712891288734 / (51)) + (95.8497985158516 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.12991967059325937 = (878.4163139699638 - 992.5398721246866) / 878.4163139699638 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.GetCustomAttributesMethodBaseHit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3209645593616477 > 2.149020090991288.
IsChangePoint: Marked as a change because one of 9/26/2023 6:21:18 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.644632816793429 (T) = (0 -2399.758558686289) / Math.Sqrt((7642.037537329512 / (46)) + (7667.291154715471 / (6))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (6) - 2, .025) and -0.1377247003344496 = (2109.2611929589357 - 2399.758558686289) / 2109.2611929589357 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.IsDefinedMethodOverrideMissInherit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 983.6046129536496 > 860.1409701053609.
IsChangePoint: Marked as a change because one of 8/31/2023 1:23:15 PM, 9/20/2023 11:58:56 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -10.434986382264032 (T) = (0 -969.7463793619531) / Math.Sqrt((915.0783439589525 / (50)) + (459.3267628287231 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.14175929886191282 = (849.3439732249876 - 969.7463793619531) / 849.3439732249876 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.IsDefinedClassMissInherit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.617256605630796 > 2.2798635194266836.
IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.168630208293237 (T) = (0 -2568.5933502963208) / Math.Sqrt((4555.958563601094 / (51)) + (1446.9735723471529 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.13270045279248474 = (2267.6722199270607 - 2568.5933502963208) / 2267.6722199270607 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.GetCustomAttributesMethodOverrideHit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.166712367683686 > 1.9929177702278458.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -4.969825769629921 (T) = (0 -2094.759680832416) / Math.Sqrt((5759.445570184245 / (49)) + (5002.679153883586 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.09619951024280045 = (1910.9292252542984 - 2094.759680832416) / 1910.9292252542984 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.IsDefinedMethodBaseMissInherit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3657321424603175 > 1.2725831361455928.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.8411249706479085 (T) = (0 -1353.9707882924895) / Math.Sqrt((490.6896476208479 / (48)) + (1141.1816023044312 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.11056144813499294 = (1219.1768321926381 - 1353.9707882924895) / 1219.1768321926381 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.IsDefinedMethodBaseHitInherit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 982.7222993626139 > 865.3664417803536.
IsChangePoint: Marked as a change because one of 8/31/2023 1:23:15 PM, 9/21/2023 9:38:47 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -8.048764803389094 (T) = (0 -962.2395194824805) / Math.Sqrt((852.89279427919 / (51)) + (721.0929964229307 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.1330186418108214 = (849.2706862656761 - 962.2395194824805) / 849.2706862656761 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.IsDefinedMethodBaseHit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0276859199313855 > 846.8908787768079.
IsChangePoint: Marked as a change because one of 8/31/2023 1:23:15 PM, 9/21/2023 9:38:47 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.982096924338995 (T) = (0 -1000.2480310242106) / Math.Sqrt((1260.911368708688 / (51)) + (1301.5962860282866 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.175530887162447 = (850.8904716563062 - 1000.2480310242106) / 850.8904716563062 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Exceptions.Handling

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ThrowAndCatchFinally - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	8.03 μs	9.59 μs	1.19	0.17	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Exceptions.Handling*'

Payloads

Baseline
Compare

Exceptions.Handling.ThrowAndCatchFinally(kind: Hardware)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.585733216404925 > 8.45044311593029.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -16.900419702815366 (T) = (0 -9634.261139277602) / Math.Sqrt((27228.231094830873 / (49)) + (32138.801002079228 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.19415358342724998 = (8067.857663356029 - 9634.261139277602) / 8067.857663356029 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Perf_Frozen<Int16>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Contains_True - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	6.56 μs	11.57 μs	1.77	0.38	True
TryGetValue_True - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.75 μs	2.45 μs	1.40	0.21	True
TryGetValue_True - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	58.58 μs	96.82 μs	1.65	0.47	True
TryGetValue_True - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	7.66 μs	12.01 μs	1.57	0.32	True
ToFrozenSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.15 μs	1.34 μs	1.17	0.16	False
Contains_True - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.55 μs	2.19 μs	1.41	0.17	True
Contains_True - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	55.81 μs	91.94 μs	1.65	0.36	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_Frozen<Int16>*'

Payloads

Baseline
Compare

System.Collections.Perf_Frozen<Int16>.Contains_True(Count: 64)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.572442816574618 > 7.2555965391046655.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -44.97365835271446 (T) = (0 -11516.828075270441) / Math.Sqrt((111982.89421773063 / (51)) + (33023.98673035151 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.6645214945114573 = (6919.002315828111 - 11516.828075270441) / 6919.002315828111 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<Int16>.TryGetValue_True(Count: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4458512910042547 > 1.7655462597622815.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -20.035297993927145 (T) = (0 -2370.620983140403) / Math.Sqrt((3057.454554053837 / (49)) + (4769.6053433784 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.42730425172034553 = (1660.9079530752238 - 2370.620983140403) / 1660.9079530752238 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<Int16>.TryGetValue_True(Count: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.81638734567902 > 61.77648856085526.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -30.717806331096543 (T) = (0 -96886.4846182898) / Math.Sqrt((5905299.761878484 / (48)) + (5176572.018931759 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.6062486016860138 = (60318.48651360194 - 96886.4846182898) / 60318.48651360194 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<Int16>.TryGetValue_True(Count: 64)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.007255269979767 > 8.11964167790447.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -65.64998694819526 (T) = (0 -12107.890568994171) / Math.Sqrt((110336.33759884023 / (48)) + (9702.593644850462 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.5940823733753275 = (7595.523776702198 - 12107.890568994171) / 7595.523776702198 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<Int16>.ToFrozenSet(Count: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3412089160514895 > 1.2335015363240258.
IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -12.464789680501509 (T) = (0 -1327.4049328894014) / Math.Sqrt((1362.5090954666325 / (47)) + (1022.2447069962608 / (8))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (8) - 2, .025) and -0.1332401659300789 = (1171.3359381327316 - 1327.4049328894014) / 1171.3359381327316 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<Int16>.Contains_True(Count: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1902527902028646 > 1.6542912137252583.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -26.07179641931128 (T) = (0 -2202.456938148133) / Math.Sqrt((3183.628367399257 / (52)) + (1928.216206584918 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.3810522970385845 = (1594.7672241456034 - 2202.456938148133) / 1594.7672241456034 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<Int16>.Contains_True(Count: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 91.94165493393889 > 60.55883267666586.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -92.63527899206856 (T) = (0 -92238.4704033248) / Math.Sqrt((6560903.349016654 / (49)) + (75461.23303886007 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.6461672540393262 = (56032.259284099004 - 92238.4704033248) / 56032.259284099004 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.20 μs	4.76 μs	1.13	0.16	False
SystemTextJson_Reflection_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.24 μs	2.92 μs	1.30	0.12	False
SystemTextJson_SourceGen_ - Duration of single invocation 📈 - ADX Test Multi Config Graph	1.98 μs	2.48 μs	1.25	0.19	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.759936389312976 > 4.4125286362806895.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.245156214045533 (T) = (0 -4838.974900428409) / Math.Sqrt((15431.850797896708 / (49)) + (6408.081091886213 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.13615990759127064 = (4259.061482540195 - 4838.974900428409) / 4259.061482540195 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.SystemTextJson_Reflection_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.923174176604202 > 2.351735359787945.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -24.113325805869888 (T) = (0 -2956.3214726733536) / Math.Sqrt((7373.082560418133 / (49)) + (2426.2426782906628 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2893584522792629 = (2292.8623668982955 - 2956.3214726733536) / 2292.8623668982955 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.SystemTextJson_SourceGen_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4782372230038194 > 2.09051685319042.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.860883129170753 (T) = (0 -2425.4871395201662) / Math.Sqrt((3789.6635423805396 / (50)) + (4839.6426567005665 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.21262543214796145 = (2000.194846007662 - 2425.4871395201662) / 2000.194846007662 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.84 μs	2.11 μs	1.14	0.22	False
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.91 μs	3.94 μs	1.35	0.15	False
SerializeToUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.69 μs	2.16 μs	1.28	0.19	True
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.43 μs	1.95 μs	1.36	0.14	True
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.81 μs	2.32 μs	1.28	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.106913475656443 > 1.9647598040213863.
IsChangePoint: Marked as a change because one of 8/22/2023 10:50:09 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -5.99801433554254 (T) = (0 -2193.039448934367) / Math.Sqrt((3990.193031171009 / (50)) + (6522.677342307518 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.12754156853651827 = (1944.9743673581822 - 2193.039448934367) / 1944.9743673581822 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.937913406822692 > 3.0301876039838493.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -25.95294931610033 (T) = (0 -3961.5145027102294) / Math.Sqrt((9490.262539856054 / (51)) + (4469.798504494782 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.36650029897637926 = (2899.022053400009 - 3961.5145027102294) / 2899.022053400009 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.156437410154283 > 1.7576746529170868.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -45.53875979318735 (T) = (0 -2160.127484820386) / Math.Sqrt((2780.183091693671 / (48)) + (210.31960272524873 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.2846982838403234 = (1681.4278589702474 - 2160.127484820386) / 1681.4278589702474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToWriter(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.95147577703793 > 1.496071655094317.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.05245497564219 (T) = (0 -1906.196629156383) / Math.Sqrt((1952.1934228152893 / (52)) + (2466.3053461570485 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.3434175507069774 = (1418.9159789919684 - 1906.196629156383) / 1418.9159789919684 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.322353847660471 > 1.9036254847550769.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.13483533958027 (T) = (0 -2304.5626461894494) / Math.Sqrt((3664.2932551040385 / (50)) + (1491.7506946776473 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2545344770389464 = (1836.9862991959092 - 2304.5626461894494) / 1836.9862991959092 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.IterateForEach<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ConcurrentQueue - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	110.93 μs	178.58 μs	1.61	0.28	True
ImmutableQueue - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	27.48 μs	29.98 μs	1.09	0.40	False
ImmutableHashSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	383.36 μs	556.35 μs	1.45	0.19	True
ConcurrentStack - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	58.79 μs	91.94 μs	1.56	0.24	True
ImmutableList - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	142.04 μs	192.34 μs	1.35	0.18	True
ImmutableSortedDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	141.76 μs	216.41 μs	1.53	0.30	True
SortedDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	225.56 μs	349.82 μs	1.55	0.14	True
ConcurrentDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	126.19 μs	198.62 μs	1.57	0.15	False
ImmutableDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	370.81 μs	557.44 μs	1.50	0.15	True
ImmutableSortedSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	134.44 μs	204.02 μs	1.52	0.22	True
SortedList - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	100.95 μs	132.16 μs	1.31	0.17	True
ConcurrentBag - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	57.31 μs	90.29 μs	1.58	0.25	False
SortedSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	30.37 μs	35.06 μs	1.15	0.29	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach<Int32>*'

Payloads

Baseline
Compare

System.Collections.IterateForEach<Int32>.ConcurrentQueue(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 178.5807289772727 > 109.61339792373681.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -104.11675854002256 (T) = (0 -177242.04218362397) / Math.Sqrt((11407964.564557899 / (51)) + (1150078.1383228549 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.7241442500821381 = (102800.00769957628 - 177242.04218362397) / 102800.00769957628 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.ImmutableQueue(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.983937643129774 > 29.617240126727907.
IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -4.582324872388262 (T) = (0 -30499.065844327968) / Math.Sqrt((8639737.71113302 / (41)) + (1035525.4272482971 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.08715187628339278 = (28054.098520801006 - 30499.065844327968) / 28054.098520801006 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.ImmutableHashSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 556.3469827586206 > 403.206449994992.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -71.02167879909696 (T) = (0 -555464.1806041813) / Math.Sqrt((115074248.5534856 / (50)) + (14365310.901194172 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.4500776584170098 = (383058.2295920335 - 555464.1806041813) / 383058.2295920335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.ConcurrentStack(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 91.9368809867878 > 60.12387336783275.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -33.01717454456801 (T) = (0 -89436.42248738334) / Math.Sqrt((4366399.475948696 / (48)) + (3494380.5616279966 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.5687969345119868 = (57009.559694993135 - 89436.42248738334) / 57009.559694993135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.ImmutableList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 192.33734751016263 > 140.73350163357924.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -20.905652542875 (T) = (0 -189716.10467249385) / Math.Sqrt((22010431.630812123 / (49)) + (26578493.092006654 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4154164905255676 = (134035.5336697039 - 189716.10467249385) / 134035.5336697039 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.ImmutableSortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 216.4062500611546 > 151.50319082889732.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.25213998095413 (T) = (0 -215685.9253145624) / Math.Sqrt((26289245.132078554 / (49)) + (59710359.17941991 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4987511484927564 = (143910.43204969048 - 215685.9253145624) / 143910.43204969048 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.SortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 349.824722037037 > 235.8688399692029.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -38.80810213159348 (T) = (0 -353762.22158576094) / Math.Sqrt((48509166.634304225 / (51)) + (38186345.66292874 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.5514315540563605 = (228023.09303353867 - 353762.22158576094) / 228023.09303353867 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.ConcurrentDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 198.62149780389367 > 128.89012846214553.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -16.086210196729382 (T) = (0 -202859.1827357678) / Math.Sqrt((8534775.776572902 / (50)) + (72426222.3685033 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.6420349789829553 = (123541.32849314505 - 202859.1827357678) / 123541.32849314505 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.ImmutableDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 557.4397987068967 > 389.61321829997763.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.529877192428895 (T) = (0 -538498.1014520903) / Math.Sqrt((86264978.90066133 / (50)) + (226981557.90826908 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.4704190085099033 = (366220.85156379663 - 538498.1014520903) / 366220.85156379663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.ImmutableSortedSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 204.02204024122807 > 141.476378372252.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -34.750350215512455 (T) = (0 -208469.29285938013) / Math.Sqrt((19434459.09288797 / (50)) + (14704244.9510851 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.5061904098554171 = (138408.32573047097 - 208469.29285938013) / 138408.32573047097 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.SortedList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 132.16043139948286 > 102.13563582098585.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.510495326312356 (T) = (0 -132424.515841387) / Math.Sqrt((5352647.648055328 / (51)) + (11565698.268814998 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3423433001920542 = (98651.75013160979 - 132424.515841387) / 98651.75013160979 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.ConcurrentBag(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.2863554238506 > 60.28118553068368.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -32.11177417926276 (T) = (0 -90351.60112789435) / Math.Sqrt((7345892.727490075 / (51)) + (2701165.3900959147 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.5703958063554887 = (57534.28579103169 - 90351.60112789435) / 57534.28579103169 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.SortedSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.06131159248055 > 32.17791920562094.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/23/2023 6:58:52 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -5.193879888390581 (T) = (0 -33890.21301588558) / Math.Sqrt((2690854.919620121 / (30)) + (3905918.588225466 / (23))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (23) - 2, .025) and -0.08468462928294276 = (31244.300970955523 - 33890.21301588558) / 31244.300970955523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SystemTextJson_Reflection_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	102.03 μs	126.62 μs	1.24	0.06	False
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	211.83 μs	242.71 μs	1.15	0.06	False
SystemTextJson_SourceGen_ - Duration of single invocation 📈 - ADX Test Multi Config Graph	91.60 μs	105.89 μs	1.16	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.SystemTextJson_Reflection_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 126.62226432891704 > 106.38004868827036.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -17.31224256759681 (T) = (0 -126849.79405298803) / Math.Sqrt((6623399.442737236 / (52)) + (7112720.169396701 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.23212957272773063 = (102951.66747127383 - 126849.79405298803) / 102951.66747127383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 242.71163511439732 > 217.78174943333332.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.060283591935336 (T) = (0 -236588.27907447005) / Math.Sqrt((30341610.828377966 / (49)) + (30931339.956498314 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.12444690574042208 = (210404.13546132014 - 236588.27907447005) / 210404.13546132014 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.SystemTextJson_SourceGen_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.89266546803651 > 97.57432297252495.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -17.726090785563162 (T) = (0 -107961.92332552525) / Math.Sqrt((4731914.26886648 / (51)) + (2408124.6295307465 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.15856061929445928 = (93186.25329356693 - 107961.92332552525) / 93186.25329356693 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Scoped - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	24.35 μs	27.79 μs	1.14	0.02	False
Transient - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	22.38 μs	25.22 μs	1.13	0.05	False
BuildProvider - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.61 μs	6.36 μs	1.13	0.07	False
Scoped - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	24.39 μs	28.43 μs	1.17	0.04	False
Scoped - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	24.06 μs	28.35 μs	1.18	0.05	False
Scoped - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	24.44 μs	28.14 μs	1.15	0.07	False
BuildProvider - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.70 μs	6.78 μs	1.19	0.09	False
Transient - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	22.09 μs	25.45 μs	1.15	0.07	False
Singleton - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	22.75 μs	26.42 μs	1.16	0.05	False
Singleton - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	21.78 μs	25.50 μs	1.17	0.07	False
Singleton - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	22.17 μs	26.18 μs	1.18	0.06	False
BuildProvider - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.51 μs	6.74 μs	1.22	0.11	False
Transient - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	22.14 μs	25.46 μs	1.15	0.07	False
Singleton - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	22.31 μs	26.14 μs	1.17	0.07	False
Transient - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	21.44 μs	25.00 μs	1.17	0.06	False
BuildProvider - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.52 μs	6.88 μs	1.25	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'

Payloads

Baseline
Compare

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.793241265597153 > 24.815777103470996.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.00433542662638 (T) = (0 -27843.743636851283) / Math.Sqrt((339963.35571846773 / (51)) + (144270.35034577487 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.15429452606679359 = (24121.87098532606 - 27843.743636851283) / 24121.87098532606 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.216172462406018 > 23.70169370871509.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -15.374801230335828 (T) = (0 -25707.907886372006) / Math.Sqrt((373319.0524224013 / (51)) + (155951.81697790985 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.14770717107509934 = (22399.361556912176 - 25707.907886372006) / 22399.361556912176 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Expressions")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.360815299027609 > 5.809815045378254.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.371562350308054 (T) = (0 -6473.242834404913) / Math.Sqrt((12981.78488610656 / (48)) + (37371.01152897163 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.16542829720932803 = (5554.389617881591 - 6473.242834404913) / 5554.389617881591 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.425902482355795 > 25.475308316199378.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -26.27690291676403 (T) = (0 -28106.327414313353) / Math.Sqrt((300512.9429845869 / (48)) + (54591.02313564389 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.15195872139743125 = (24398.72791641163 - 28106.327414313353) / 24398.72791641163 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Runtime")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.347680885143856 > 25.617077537774723.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -36.24050303538282 (T) = (0 -28254.19642870491) / Math.Sqrt((302028.7723830308 / (51)) + (25687.74297952998 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.16619216194425526 = (24227.736517796522 - 28254.19642870491) / 24227.736517796522 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.142742257855808 > 25.01277562770642.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -38.387297704474925 (T) = (0 -28165.00555135112) / Math.Sqrt((272518.25526140386 / (50)) + (14901.47141673362 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.1615892659037826 = (24246.957490122073 - 28165.00555135112) / 24246.957490122073 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Runtime")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.7754387785522345 > 5.851163153290112.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.786774028391198 (T) = (0 -6606.221099176077) / Math.Sqrt((13640.475593221016 / (51)) + (20230.524856546705 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.179649532154155 = (5600.155740419337 - 6606.221099176077) / 5600.155740419337 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Dynamic")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.451132436528983 > 23.163352869810026.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -16.900249739543842 (T) = (0 -25067.954614772876) / Math.Sqrt((235433.5281297305 / (52)) + (88112.19490735825 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.12342288183134306 = (22313.907808169668 - 25067.954614772876) / 22313.907808169668 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.415682186447814 > 23.90881519781676.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -8.410225431503974 (T) = (0 -26207.793974751028) / Math.Sqrt((307838.6423273525 / (51)) + (535177.3141121513 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.15984161682278164 = (22596.01103687199 - 26207.793974751028) / 22596.01103687199 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.500542997643677 > 23.443858039837.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -21.824166688241096 (T) = (0 -25851.505788909388) / Math.Sqrt((340000.24785143766 / (51)) + (63188.56292999536 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.14485902880239032 = (22580.51440267893 - 25851.505788909388) / 22580.51440267893 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.175021080508476 > 23.353723312455422.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.301906381381361 (T) = (0 -26195.166060471773) / Math.Sqrt((315633.4541600281 / (50)) + (214007.97094086066 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.154107126872073 = (22697.343643884607 - 26195.166060471773) / 22697.343643884607 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Dynamic")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.737052315211188 > 5.82659881786537.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -15.087390536988973 (T) = (0 -6615.6633434607675) / Math.Sqrt((14962.193994417888 / (48)) + (16824.441836333095 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.18103773492505498 = (5601.568136077021 - 6615.6633434607675) / 5601.568136077021 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "ILEmit")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.46043023208469 > 23.26997549101484.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.58380279249165 (T) = (0 -26026.363308950535) / Math.Sqrt((248943.19401731424 / (49)) + (278078.8945834648 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.16625351354106546 = (22316.214276540406 - 26026.363308950535) / 22316.214276540406 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Runtime")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.13536697322919 > 23.848044970120736.
IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -30.07903678764601 (T) = (0 -25855.87088015755) / Math.Sqrt((175549.49384925357 / (50)) + (35396.503208059185 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.1485473569548479 = (22511.802167835212 - 25855.87088015755) / 22511.802167835212 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Expressions")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.996163826669157 > 23.611397319498636.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.344390871591274 (T) = (0 -25271.382543371135) / Math.Sqrt((286317.477354281 / (51)) + (271295.1282435661 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.13849833675777368 = (22197.11854418621 - 25271.382543371135) / 22197.11854418621 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "ILEmit")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.8808521784834 > 5.799649055984982.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -12.48587761592401 (T) = (0 -6661.485511011651) / Math.Sqrt((13716.412465403728 / (49)) + (29397.66954628477 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.19576628198761206 = (5570.892582736886 - 6661.485511011651) / 5570.892582736886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Reflection.Activator<EmptyClass>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
CreateInstanceType - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	442.58 ns	564.75 ns	1.28	0.19	True
CreateInstanceGeneric - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	312.21 ns	452.56 ns	1.45	0.21	True
CreateInstanceNames - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	12.06 μs	13.63 μs	1.13	0.42	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Activator<EmptyClass>*'

Payloads

Baseline
Compare

System.Reflection.Activator<EmptyClass>.CreateInstanceType

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 564.7521106400574 > 457.378104295875.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -16.66315385616183 (T) = (0 -575.2777707150733) / Math.Sqrt((174.51176330907663 / (51)) + (265.1617568921862 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3189886049975192 = (436.1506752487489 - 575.2777707150733) / 436.1506752487489 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Reflection.Activator<EmptyClass>.CreateInstanceGeneric

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 452.5568543277151 > 324.54570750997357.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -68.70649165643678 (T) = (0 -448.51810795761935) / Math.Sqrt((79.12442796958112 / (50)) + (10.210965512541025 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.4524473086825635 = (308.8016379502578 - 448.51810795761935) / 308.8016379502578 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Reflection.Activator<EmptyClass>.CreateInstanceNames

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.630619256315333 > 12.635636679397205.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.176875960099711 (T) = (0 -14293.837530316992) / Math.Sqrt((649015.3640273053 / (47)) + (279727.2720696842 / (4))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (4) - 2, .025) and -0.14292415315317195 = (12506.374540149705 - 14293.837530316992) / 12506.374540149705 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.ContainsTrueComparer<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableSortedSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	668.48 μs	1.39 ms	2.09	0.30	True
SortedSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	580.70 μs	1.01 ms	1.73	0.22	True
HashSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	86.41 μs	183.00 μs	2.12	0.40	True
ImmutableHashSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	183.40 μs	284.12 μs	1.55	0.13	True
FrozenSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	82.75 μs	119.25 μs	1.44	0.28	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer<Int32>*'

Payloads

Baseline
Compare

System.Collections.ContainsTrueComparer<Int32>.ImmutableSortedSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.394021526041667 > 705.2975073556386.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.908807777088384 (T) = (0 -1279778.1472842263) / Math.Sqrt((1025494994.4255087 / (51)) + (6062564005.958934 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.8400243676726515 = (695522.3907729816 - 1279778.1472842263) / 695522.3907729816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrueComparer<Int32>.SortedSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0063278658854167 > 591.9114279477164.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -55.07579024559061 (T) = (0 -1011374.6532664764) / Math.Sqrt((551086521.5706757 / (49)) + (197347828.18909714 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.7357005645197306 = (582689.5917074984 - 1011374.6532664764) / 582689.5917074984 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrueComparer<Int32>.HashSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 182.99608681797847 > 91.83700494908211.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -26.344319228776826 (T) = (0 -175563.79534156574) / Math.Sqrt((20624982.981831513 / (50)) + (38825524.04532427 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.9132511981055097 = (91762.02033241009 - 175563.79534156574) / 91762.02033241009 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrueComparer<Int32>.ImmutableHashSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 284.1193391262755 > 199.06266545474764.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -55.81258892341038 (T) = (0 -283676.95658639167) / Math.Sqrt((29280752.446328178 / (50)) + (7967544.398780599 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.46170919221696893 = (194072.0890973805 - 283676.95658639167) / 194072.0890973805 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrueComparer<Int32>.FrozenSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 119.24939076028137 > 88.23624518107826.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.28790698904108 (T) = (0 -120904.44939821848) / Math.Sqrt((8984454.744045306 / (50)) + (6474325.890622647 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.4340213276068467 = (84311.47227077073 - 120904.44939821848) / 84311.47227077073 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
HeapSort - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	281.34 μs	364.01 μs	1.29	0.10	True
K_Max_Elements - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	292.99 μs	400.85 μs	1.37	0.14	True
HeapSort - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.41 ms	6.29 ms	1.43	0.11	True
K_Max_Elements - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	44.70 μs	57.83 μs	1.29	0.08	True
Dequeue_And_Enqueue - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	27.55 μs	35.06 μs	1.27	0.06	False
HeapSort - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	10.03 μs	13.50 μs	1.35	0.11	True
Dequeue_And_Enqueue - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	670.16 μs	874.12 μs	1.30	0.09	True
Dequeue_And_Enqueue - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	9.75 ms	14.20 ms	1.46	0.09	True
K_Max_Elements - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	8.06 μs	10.56 μs	1.31	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>*'

Payloads

Baseline
Compare

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 100)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 364.0141997987478 > 286.6527509896779.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -26.889199354678087 (T) = (0 -355809.83647814114) / Math.Sqrt((45654829.03035784 / (51)) + (33789503.880630076 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.30037310993977345 = (273621.34279646893 - 355809.83647814114) / 273621.34279646893 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.K_Max_Elements(Size: 1000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 400.84874260355025 > 306.4639300954861.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.712937329515682 (T) = (0 -399542.7582573537) / Math.Sqrt((44485393.569683425 / (50)) + (55192393.287977375 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.36207527858901756 = (293333.83002974885 - 399542.7582573537) / 293333.83002974885 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 1000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.2943640991453 > 4.587284718073918.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -25.957421540168664 (T) = (0 -6130578.450976801) / Math.Sqrt((10702746110.709892 / (52)) + (17367394648.082836 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.399642570592552 = (4380102.877537772 - 6130578.450976801) / 4380102.877537772 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.K_Max_Elements(Size: 100)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.83498753063726 > 45.469751931138354.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -42.81555986765169 (T) = (0 -57952.95503219914) / Math.Sqrt((711071.3145724268 / (51)) + (389236.4685036414 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.32700252205272023 = (43672.076027822906 - 57952.95503219914) / 43672.076027822906 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Dequeue_And_Enqueue(Size: 10)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.06470581636501 > 28.85838333775506.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -31.77514203312454 (T) = (0 -35497.22472341288) / Math.Sqrt((344300.73184804973 / (50)) + (156422.23026355487 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.27792117108708464 = (27777.31954563096 - 35497.22472341288) / 27777.31954563096 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 10)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.499484137806634 > 10.75926405080899.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -44.10172654128589 (T) = (0 -13641.823535873253) / Math.Sqrt((68008.12777686633 / (49)) + (17097.378910115207 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.32146525107844004 = (10323.255586736193 - 13641.823535873253) / 10323.255586736193 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Dequeue_And_Enqueue(Size: 100)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 874.1215233796297 > 706.3615296744356.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -32.677069299396564 (T) = (0 -876523.6209666964) / Math.Sqrt((235357763.9882868 / (51)) + (148881698.30377856 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.31774835581924177 = (665167.6832651127 - 876523.6209666964) / 665167.6832651127 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Dequeue_And_Enqueue(Size: 1000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.196946078431374 > 10.206123382562499.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -21.485696382577153 (T) = (0 -13883235.194831058) / Math.Sqrt((33237724594.01363 / (48)) + (141037955342.507 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.4153088007888271 = (9809332.90819162 - 13883235.194831058) / 9809332.90819162 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.K_Max_Elements(Size: 10)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.562970084433228 > 8.425393652727502.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -24.921410735700302 (T) = (0 -10721.941499424347) / Math.Sqrt((66680.40284531409 / (51)) + (27036.277295659347 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.3091031817050305 = (8190.295195417403 - 10721.941499424347) / 8190.295195417403 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.89 μs	3.31 μs	1.14	0.20	False
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.68 μs	3.27 μs	1.22	0.14	False
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.92 μs	3.86 μs	1.32	0.13	False
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.32 μs	2.99 μs	1.29	0.18	True
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.98 μs	3.62 μs	1.22	0.14	False
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.35 μs	4.01 μs	1.20	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.313105085569446 > 2.8545623857804614.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -8.974612100332093 (T) = (0 -3394.60416723701) / Math.Sqrt((6901.269965046071 / (50)) + (19582.872251850913 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.23088097318584502 = (2757.865497303836 - 3394.60416723701) / 2757.865497303836 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromUtf8Bytes(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.2745644355670103 > 2.7195455971374947.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -21.12400126337315 (T) = (0 -3276.662332827777) / Math.Sqrt((4393.75161681264 / (48)) + (3841.6240962540555 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.26437193329792086 = (2591.5335879697236 - 3276.662332827777) / 2591.5335879697236 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromReader(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.8594632433773053 > 3.1255916564075497.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -12.93661919319168 (T) = (0 -3748.9589365623297) / Math.Sqrt((8270.41118307464 / (51)) + (12330.983923231228 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.24465948159370468 = (3012.0358154200007 - 3748.9589365623297) / 3012.0358154200007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.9857343557996656 > 2.4229674756207253.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.03233394723909 (T) = (0 -2978.84548050223) / Math.Sqrt((4112.050493442509 / (51)) + (5009.227693746491 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2837106637630339 = (2320.496015644308 - 2978.84548050223) / 2320.496015644308 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromString(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6190479662238273 > 3.173021887178539.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -24.090959238799083 (T) = (0 -3601.2754790565346) / Math.Sqrt((7826.447303325559 / (50)) + (1242.2892858473597 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.19019099420389468 = (3025.7962768953626 - 3601.2754790565346) / 3025.7962768953626 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromReader(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.011814827896225 > 3.3831623565500175.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.49234173062742 (T) = (0 -3960.531358159432) / Math.Sqrt((10322.340495472106 / (51)) + (4527.547542047779 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2056245448864972 = (3285.0453940719112 - 3960.531358159432) / 3285.0453940719112 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.IO.Tests.Perf_RandomAccess

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ReadScatter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	192.16 μs	218.36 μs	1.14	0.06	False
ReadScatter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	191.01 μs	213.59 μs	1.12	0.07	False
ReadScatter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	9.84 ms	10.62 ms	1.08	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_RandomAccess*'

Payloads

Baseline
Compare

System.IO.Tests.Perf_RandomAccess.ReadScatter(fileSize: 1048576, buffersSize: 16384, options: Asynchronous)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 218.35607409037559 > 203.10640819818042.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -16.293555560545567 (T) = (0 -220799.57563447353) / Math.Sqrt((19043963.96190519 / (49)) + (9513242.881816676 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.13992510057296423 = (193696.5643826005 - 220799.57563447353) / 193696.5643826005 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_RandomAccess.ReadScatter(fileSize: 1048576, buffersSize: 16384, options: None)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 213.59477166754476 > 201.9623301941069.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.797113185277902 (T) = (0 -221329.46964337962) / Math.Sqrt((20450157.928195324 / (51)) + (67311803.85858257 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.14609489704730955 = (193116.18105410985 - 221329.46964337962) / 193116.18105410985 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_RandomAccess.ReadScatter(fileSize: 104857600, buffersSize: 65536, options: Asynchronous)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.62420130844156 > 10.323642967272729.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -10.421935208832329 (T) = (0 -10512958.25959596) / Math.Sqrt((15451410294.59842 / (48)) + (14616308917.51422 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.06667740806977646 = (9855799.119829355 - 10512958.25959596) / 9855799.119829355 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
CaseInsensitiveNotMatching - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.41 μs	6.51 μs	1.20	0.09	True
MissingProperties - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.30 μs	2.55 μs	1.11	0.18	False
Baseline - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.16 μs	6.53 μs	1.27	0.13	True
CaseInsensitiveMatching - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.15 μs	6.44 μs	1.25	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.CaseInsensitiveNotMatching

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.510755545922656 > 5.567852450746095.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -26.4899578971109 (T) = (0 -6525.714498550347) / Math.Sqrt((16906.416838370562 / (50)) + (7711.281448847515 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.23951315774190293 = (5264.739997144235 - 6525.714498550347) / 5264.739997144235 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.MissingProperties

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.5489734967408677 > 2.323054095278499.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.439531999848477 (T) = (0 -2569.703476146654) / Math.Sqrt((5203.023874355299 / (51)) + (7059.008604423788 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.14296931667603968 = (2248.269869238322 - 2569.703476146654) / 2248.269869238322 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.Baseline

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.528738219935259 > 5.412455804201913.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -26.5337086820953 (T) = (0 -6606.57412976045) / Math.Sqrt((21289.55928350008 / (50)) + (9215.264900970682 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2655536686827024 = (5220.303408102118 - 6606.57412976045) / 5220.303408102118 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.CaseInsensitiveMatching

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.441923351016345 > 5.500424815534093.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -15.704047363966573 (T) = (0 -6547.422052509661) / Math.Sqrt((24317.7136223795 / (51)) + (18038.661076541055 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.23949641059505863 = (5282.324334740403 - 6547.422052509661) / 5282.324334740403 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.12 ms	1.39 ms	1.24	0.04	False
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.02 ms	1.25 ms	1.22	0.11	False
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.27 ms	1.56 ms	1.23	0.06	False
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.09 ms	1.31 ms	1.20	0.08	False
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.23 ms	1.44 ms	1.17	0.06	False
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.01 ms	1.24 ms	1.23	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3858915662878788 > 1.183855398954327.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.011588497777645 (T) = (0 -1359725.184422912) / Math.Sqrt((660296809.7697545 / (50)) + (646246884.357536 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.21229497354360322 = (1121612.4904389915 - 1359725.184422912) / 1121612.4904389915 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromString(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2453368787774726 > 1.0749692384667968.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -20.74426878299501 (T) = (0 -1229600.3695217844) / Math.Sqrt((928952795.1493777 / (51)) + (232011762.5828806 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17272642320384013 = (1048497.1986582915 - 1229600.3695217844) / 1048497.1986582915 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromReader(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5623083250000005 > 1.3585479685708675.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -23.181485731613837 (T) = (0 -1563733.3317463498) / Math.Sqrt((520701880.77929527 / (50)) + (456467877.5761985 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.19822350891366555 = (1305043.1076619946 - 1563733.3317463498) / 1305043.1076619946 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3101450252403846 > 1.1591295421995194.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -10.881177413278794 (T) = (0 -1320903.955284299) / Math.Sqrt((530589413.6365857 / (51)) + (1188693419.5893936 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.19920508068926288 = (1101482.9544627077 - 1320903.955284299) / 1101482.9544627077 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromReader(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4407784025349653 > 1.2705378611177884.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -34.38504303913468 (T) = (0 -1442913.068958819) / Math.Sqrt((904178241.8911017 / (50)) + (47820328.88073273 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.1614424361978853 = (1242345.7452461957 - 1442913.068958819) / 1242345.7452461957 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2365788506410258 > 1.058057111904297.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.507599088390126 (T) = (0 -1242236.9640536946) / Math.Sqrt((596455927.5076629 / (50)) + (330757526.4188127 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.21395075424297613 = (1023300.9532815504 - 1242236.9640536946) / 1023300.9532815504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SystemTextJson_SourceGen_ - Duration of single invocation 📈 - ADX Test Multi Config Graph	1.06 ms	1.21 ms	1.13	0.06	False
SystemTextJson_Reflection_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.12 ms	1.38 ms	1.24	0.11	False
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.36 ms	2.71 ms	1.15	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.SystemTextJson_SourceGen_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.205628699704142 > 1.1157571994535256.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -15.407896318209167 (T) = (0 -1231510.5078582878) / Math.Sqrt((736722471.2457913 / (48)) + (465823848.61204654 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.16773080421196004 = (1054618.4989008398 - 1231510.5078582878) / 1054618.4989008398 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.SystemTextJson_Reflection_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3826458307291665 > 1.1695047302187502.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -43.12980935356808 (T) = (0 -1370313.7884610617) / Math.Sqrt((640310582.4494108 / (51)) + (85161145.39882869 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.22415202377622873 = (1119398.376873125 - 1370313.7884610617) / 1119398.376873125 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.7091201395833333 > 2.485112866119792.
IsChangePoint: Marked as a change because one of 8/28/2023 11:06:53 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.196496708588734 (T) = (0 -2743075.540344551) / Math.Sqrt((3576559958.618361 / (51)) + (2234817298.6145005 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.14913600146542363 = (2387076.4964690628 - 2743075.540344551) / 2387076.4964690628 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	786.43 ns	850.89 ns	1.08	0.13	False
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	765.87 ns	819.12 ns	1.07	0.18	False
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	523.53 ns	579.79 ns	1.11	0.17	False
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	498.27 ns	575.50 ns	1.15	0.16	False
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	768.93 ns	854.60 ns	1.11	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<Int32>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 850.893459779281 > 805.8658796392851.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.690647961612873 (T) = (0 -849.3929469971573) / Math.Sqrt((302.1978558313762 / (49)) + (249.5396002821591 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.10431528755654705 = (769.1580081957923 - 849.3929469971573) / 769.1580081957923 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromString(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 819.1239760654752 > 802.5629474185519.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -8.265741718077287 (T) = (0 -843.8302254901521) / Math.Sqrt((569.2038047766608 / (49)) + (307.2023397077713 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.10145189110721496 = (766.107201143308 - 843.8302254901521) / 766.107201143308 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 579.7905013798446 > 529.2555846712875.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -23.98338874233614 (T) = (0 -584.3837479539886) / Math.Sqrt((266.8222927801291 / (51)) + (19.81042321380662 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.1507117126002576 = (507.8454851506287 - 584.3837479539886) / 507.8454851506287 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 575.499801137092 > 522.4280891692491.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.056454512511142 (T) = (0 -573.7035717605372) / Math.Sqrt((229.83899816191717 / (50)) + (146.27667636496625 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.14109780804920544 = (502.7645901286303 - 573.7035717605372) / 502.7645901286303 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromReader(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 854.6022169712102 > 820.0284693515058.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 11:21:57 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -5.628299791408654 (T) = (0 -847.7791475316618) / Math.Sqrt((326.7258026538941 / (42)) + (1172.4973363080926 / (11))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (11) - 2, .025) and -0.07642473501454103 = (787.5879473544516 - 847.7791475316618) / 787.5879473544516 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.ScopeValidation

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TransientWithScopeValidation - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.56 μs	3.57 μs	1.40	0.09	True
Transient - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.47 μs	3.67 μs	1.49	0.06	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ScopeValidation*'

Payloads

Baseline
Compare

Microsoft.Extensions.DependencyInjection.ScopeValidation.TransientWithScopeValidation

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5721370103556045 > 2.6120338411876216.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.785264039048204 (T) = (0 -3621.334552260676) / Math.Sqrt((2982.00679607743 / (49)) + (6561.751511684511 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.46299851526088 = (2475.282452091159 - 3621.334552260676) / 2475.282452091159 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.ScopeValidation.Transient

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6721773304263565 > 2.538397354217031.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -32.97041877294565 (T) = (0 -3571.464970536285) / Math.Sqrt((3824.075075782129 / (50)) + (4607.074656322817 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.47827343088079644 = (2415.9704801082075 - 3571.464970536285) / 2415.9704801082075 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeToUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.48 μs	1.76 μs	1.19	0.12	False
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.59 μs	1.87 μs	1.17	0.18	False
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.32 μs	2.79 μs	1.21	0.12	False
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.21 μs	1.48 μs	1.22	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7629894654426332 > 1.5334554374938363.
IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -12.869732564389988 (T) = (0 -1708.4049071552156) / Math.Sqrt((1682.9397781614334 / (51)) + (1692.6720703828535 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.19174492480392372 = (1433.5323537763732 - 1708.4049071552156) / 1433.5323537763732 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8696111285011061 > 1.6849958322701724.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.482050844976513 (T) = (0 -1885.5776661835157) / Math.Sqrt((1461.8879202561432 / (50)) + (720.5568242253956 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.17578538005376304 = (1603.675039825123 - 1885.5776661835157) / 1603.675039825123 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeObjectProperty(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.7936287893727556 > 2.431156968575569.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -12.758170418713044 (T) = (0 -2827.0966293718484) / Math.Sqrt((3909.8027757387304 / (49)) + (5965.154599957528 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.21783974181810864 = (2321.402835114648 - 2827.0966293718484) / 2321.402835114648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToWriter(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4796633165829147 > 1.2868926319132117.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -33.60179552569488 (T) = (0 -1465.8720589968648) / Math.Sqrt((1335.7772639851544 / (51)) + (91.34923773636048 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.1911946630385935 = (1230.5898477227915 - 1465.8720589968648) / 1230.5898477227915 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<BinaryData>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.21 μs	2.36 μs	1.07	0.02	False
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.44 μs	1.60 μs	1.11	0.03	False
SerializeToUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.86 μs	1.97 μs	1.06	0.01	False
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.95 μs	3.15 μs	1.07	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<BinaryData>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3612361376114377 > 2.337106216145719.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -26.76401089549868 (T) = (0 -2362.6533632701603) / Math.Sqrt((917.9466631473354 / (51)) + (32.77376372520845 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.06154265763469063 = (2225.679153143577 - 2362.6533632701603) / 2225.679153143577 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToWriter(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5954931427588475 > 1.5093338955328477.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.296257224427567 (T) = (0 -1553.558409715717) / Math.Sqrt((484.3969367690402 / (49)) + (1036.170461410555 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.08344444366559721 = (1433.9068503222852 - 1553.558409715717) / 1433.9068503222852 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9671978408183979 > 1.9537631531696555.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -23.10151077023108 (T) = (0 -1970.0606222557108) / Math.Sqrt((676.262738789881 / (50)) + (36.7288562860563 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.05918576868427504 = (1859.9764842978661 - 1970.0606222557108) / 1859.9764842978661 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeObjectProperty(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.148498728527992 > 3.079961449913841.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -10.496460061592272 (T) = (0 -3205.413443014745) / Math.Sqrt((2346.560130883036 / (49)) + (2233.148495540168 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.08769291575675575 = (2946.9838376069574 - 3205.413443014745) / 2946.9838376069574 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<Location>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DataContractSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	20.46 μs	22.44 μs	1.10	0.07	False
XmlSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	24.36 μs	29.18 μs	1.20	0.08	False
DataContractSerializer_BinaryXml_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	9.04 μs	11.72 μs	1.30	0.17	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream<Location>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Xml_FromStream<Location>.DataContractSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.443702651515153 > 21.446833560532074.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.064644992338955 (T) = (0 -22702.375343823922) / Math.Sqrt((340946.7200823265 / (49)) + (142637.84989576 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.11187024155529572 = (20418.1877483002 - 22702.375343823922) / 20418.1877483002 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<Location>.XmlSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.181711646492865 > 25.707908662585808.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.10929687289938 (T) = (0 -29242.597617139985) / Math.Sqrt((616233.6229653957 / (50)) + (368811.842379073 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.16950435086699572 = (25004.265777601762 - 29242.597617139985) / 25004.265777601762 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<Location>.DataContractSerializer_BinaryXml_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.721943197703103 > 9.439256304308122.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -17.69823803394302 (T) = (0 -11385.292470207693) / Math.Sqrt((75718.74419533323 / (51)) + (56979.6462115993 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.24217365384952852 = (9165.620631965889 - 11385.292470207693) / 9165.620631965889 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DataContractJsonSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.85 ms	5.72 ms	1.18	0.06	False
SystemTextJson_SourceGen_ - Duration of single invocation 📈 - ADX Test Multi Config Graph	641.75 μs	763.30 μs	1.19	0.18	False
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.89 ms	2.62 ms	1.38	0.11	False
SystemTextJson_Reflection_ - Duration of single invocation 📈 - ADX Test Multi Config Graph	595.32 μs	758.17 μs	1.27	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.DataContractJsonSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.722136507936509 > 5.075294564446079.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -29.496558286140353 (T) = (0 -5774966.449513591) / Math.Sqrt((24841224615.99735 / (50)) + (1936688668.0975869 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.19044260056921936 = (4851108.694154801 - 5774966.449513591) / 4851108.694154801 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_SourceGen_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 763.3047156593407 > 637.4059162504623.
IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -60.66450067383461 (T) = (0 -760870.4220352154) / Math.Sqrt((256032248.97203276 / (49)) + (4855158.874318186 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2536298465648892 = (606933.8761518007 - 760870.4220352154) / 606933.8761518007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6202028867781157 > 1.9947930587521854.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -41.57063444131317 (T) = (0 -2594659.4977098256) / Math.Sqrt((1489070693.8322172 / (51)) + (759601897.6266135 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.368435530093083 = (1896077.2653523057 - 2594659.4977098256) / 1896077.2653523057 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_Reflection_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 758.173699490316 > 620.5252297832076.
IsChangePoint: Marked as a change because one of 8/13/2023 2:50:54 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -35.1949412904858 (T) = (0 -749602.0252239524) / Math.Sqrt((219281696.21500066 / (50)) + (53461769.88322117 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2465970912777687 = (601318.6060426359 - 749602.0252239524) / 601318.6060426359 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<StructRecord>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.02 μs	6.34 μs	1.26	0.12	True
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	6.44 μs	7.89 μs	1.23	0.12	False
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.54 μs	6.90 μs	1.25	0.12	True
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.95 μs	7.47 μs	1.26	0.07	True
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.24 μs	6.96 μs	1.33	0.12	True
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	6.36 μs	7.76 μs	1.22	0.09	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<StructRecord>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.336314949270307 > 5.245754427992908.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -28.388151854338602 (T) = (0 -6455.824100448592) / Math.Sqrt((11925.388739778005 / (51)) + (8466.974273817346 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2709593341900615 = (5079.489112499941 - 6455.824100448592) / 5079.489112499941 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromReader(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.8906534084719295 > 6.674361640026115.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.218229575445466 (T) = (0 -7937.338213010487) / Math.Sqrt((41092.181056410576 / (50)) + (7627.399601342729 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.218216045979089 = (6515.542328644335 - 7937.338213010487) / 6515.542328644335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.901024153588269 > 5.821366009135901.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.71968045484871 (T) = (0 -7005.62292960716) / Math.Sqrt((24968.0782705336 / (51)) + (18452.46489373896 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.25167794085275735 = (5596.985215569342 - 7005.62292960716) / 5596.985215569342 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromString(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.465522166587188 > 6.211875001584773.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.220045378998028 (T) = (0 -7406.862681192988) / Math.Sqrt((11542.359386435495 / (48)) + (51230.567501321784 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.25609041257624743 = (5896.759187900716 - 7406.862681192988) / 5896.759187900716 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromUtf8Bytes(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.955107329663405 > 5.558636442690243.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.585701592875267 (T) = (0 -6687.1140472809975) / Math.Sqrt((17827.242205427945 / (51)) + (33230.90045852127 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2546192575597651 = (5329.994743016648 - 6687.1140472809975) / 5329.994743016648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromReader(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.761772019262363 > 6.4598572477989356.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -21.782759075041152 (T) = (0 -7651.515005322159) / Math.Sqrt((25589.943647456956 / (51)) + (15963.516954408818 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.23581395070578778 = (6191.478095025784 - 7651.515005322159) / 6191.478095025784 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableHashSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.33 ms	1.66 ms	1.24	0.06	False
ImmutableSortedDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	817.19 μs	1.07 ms	1.31	0.06	True
SortedDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	592.75 μs	734.33 μs	1.24	0.13	True
ImmutableSortedSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	151.63 μs	194.33 μs	1.28	0.14	True
HashSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	23.78 μs	27.60 μs	1.16	0.41	False
ImmutableList - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	135.96 μs	178.17 μs	1.31	0.13	True
SortedDictionaryDeepCopy - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	93.14 μs	129.24 μs	1.39	0.17	True
LinkedList - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	25.08 μs	27.42 μs	1.09	0.34	False
ConcurrentStack - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	18.17 μs	20.89 μs	1.15	0.28	False
ConcurrentDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	72.46 μs	76.35 μs	1.05	0.25	False
ImmutableStack - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	21.55 μs	25.66 μs	1.19	0.17	False
ImmutableDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.25 ms	1.57 ms	1.25	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection<Int32>*'

Payloads

Baseline
Compare

System.Collections.CtorFromCollection<Int32>.ImmutableHashSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6575740298076922 > 1.375260905208333.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -15.844825726217106 (T) = (0 -1644680.6094524574) / Math.Sqrt((1071095877.5518084 / (51)) + (1593739068.7509177 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2458038986271667 = (1320176.1619664533 - 1644680.6094524574) / 1320176.1619664533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ImmutableSortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0732344505555556 > 861.4242651875.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.693542860379196 (T) = (0 -1083492.39245394) / Math.Sqrt((179270333.03335118 / (51)) + (722621237.1858835 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.32743107949899053 = (816232.5029054466 - 1083492.39245394) / 816232.5029054466 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.SortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 734.3337606292517 > 586.1714845312499.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.90930247443798 (T) = (0 -755372.2298613589) / Math.Sqrt((173695686.9568377 / (52)) + (257997750.12307253 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.3329783861527779 = (566680.0285048153 - 755372.2298613589) / 566680.0285048153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ImmutableSortedSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 194.32694003527337 > 156.26944164791496.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -23.416811831331195 (T) = (0 -196537.5687224454) / Math.Sqrt((11384330.117133103 / (51)) + (17082511.179319963 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3379280367908102 = (146896.96554522144 - 196537.5687224454) / 146896.96554522144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.HashSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.60158096187944 > 24.527526202038338.
IsChangePoint: Marked as a change because one of 8/22/2023 10:50:09 AM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -4.485327115502104 (T) = (0 -25548.808622496897) / Math.Sqrt((1810605.496226513 / (39)) + (1522694.8281811753 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.07429846990677641 = (23781.85331001535 - 25548.808622496897) / 23781.85331001535 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ImmutableList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 178.16548214285714 > 142.44414395952026.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -23.235530470160164 (T) = (0 -180422.45533319373) / Math.Sqrt((18855731.458246406 / (50)) + (11759566.681640811 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.3064207959336959 = (138104.3962977076 - 180422.45533319373) / 138104.3962977076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.SortedDictionaryDeepCopy(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 129.23924129098359 > 100.48305446828606.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -47.786541015713794 (T) = (0 -128258.91419145248) / Math.Sqrt((9523339.406236034 / (51)) + (1095805.1999196692 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3384781364481951 = (95824.43724617138 - 128258.91419145248) / 95824.43724617138 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.LinkedList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.423070737969486 > 25.875421710372855.
IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.348006423965183 (T) = (0 -27591.986242979146) / Math.Sqrt((1158377.5360448654 / (40)) + (1589543.7767142141 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.09510778554138972 = (25195.680833680188 - 27591.986242979146) / 25195.680833680188 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ConcurrentStack(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.891262197993452 > 19.24646073648848.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 8:25:02 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -4.260644148940328 (T) = (0 -20027.949604327176) / Math.Sqrt((2015292.2040811558 / (39)) + (1169836.3572793694 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.08487107180617919 = (18461.133423885163 - 20027.949604327176) / 18461.133423885163 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ConcurrentDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 76.34969366830066 > 75.20680372555566.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -12.719259287707349 (T) = (0 -77968.89388276958) / Math.Sqrt((5867956.491521967 / (39)) + (1262646.8399195778 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.08698763375590998 = (71729.32925958015 - 77968.89388276958) / 71729.32925958015 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ImmutableStack(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.66194754644588 > 22.638184026064945.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.073793868076504 (T) = (0 -23970.66823498079) / Math.Sqrt((250754.16322947244 / (39)) + (1014829.6956507354 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.09039458385453839 = (21983.480649953908 - 23970.66823498079) / 21983.480649953908 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ImmutableDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5732205366071428 > 1.3107010792067308.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.53101645687781 (T) = (0 -1550963.8395833333) / Math.Sqrt((1453649067.000542 / (51)) + (2434809096.1236496 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.21671771308180512 = (1274711.317923466 - 1550963.8395833333) / 1274711.317923466 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ForEach - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	480.49 ns	550.49 ns	1.15	0.12	False
GetCredential_HostPort - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	562.37 ns	683.28 ns	1.22	0.16	False
GetCredential_HostPort - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	422.83 ns	526.02 ns	1.24	0.09	True
GetCredential_HostPort - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	538.44 ns	638.08 ns	1.19	0.08	False
ForEach - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.74 μs	2.01 μs	1.16	0.28	False
GetCredential_Uri - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	740.98 ns	831.48 ns	1.12	0.17	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'

Payloads

Baseline
Compare

System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 550.4886839837037 > 493.4083150535499.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -21.27600447000733 (T) = (0 -556.4788621288907) / Math.Sqrt((111.68418498870264 / (50)) + (52.309149941539715 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.17592253920820677 = (473.22748189144244 - 556.4788621288907) / 473.22748189144244 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "name5", hostPortCount: 10)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 683.2834415823503 > 608.5332316704016.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.964194361941024 (T) = (0 -672.143376770291) / Math.Sqrt((203.06417671938982 / (49)) + (135.33295049033126 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.15901713038347343 = (579.9253170209012 - 672.143376770291) / 579.9253170209012 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "notfound", hostPortCount: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 526.0206143961277 > 455.13816073038794.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -20.143977283460504 (T) = (0 -520.265640595885) / Math.Sqrt((120.55381396112794 / (51)) + (66.34986682440483 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2027240074466949 = (432.57275765233555 - 520.265640595885) / 432.57275765233555 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "notfound", hostPortCount: 10)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 638.0766804004594 > 578.5687517002698.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -35.678422758532996 (T) = (0 -636.3185545150793) / Math.Sqrt((161.92416590199431 / (51)) + (15.591335734870443 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17524977322776203 = (541.4326120374086 - 636.3185545150793) / 541.4326120374086 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 10)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0105194393186325 > 1.9546905005494466.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -5.978777472240219 (T) = (0 -1935.9735887438717) / Math.Sqrt((3360.395119284008 / (39)) + (7142.967660303488 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.08156942984452979 = (1789.9670010293821 - 1935.9735887438717) / 1789.9670010293821 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 831.4793406613146 > 778.2333696080393.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.453184503033338 (T) = (0 -821.881164696636) / Math.Sqrt((483.9521942858789 / (50)) + (54.279921716276284 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.12140204609590531 = (732.9049983081148 - 821.881164696636) / 732.9049983081148 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromStream<Location>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SystemTextJson_SourceGen_ - Duration of single invocation 📈 - ADX Test Multi Config Graph	4.47 μs	5.06 μs	1.13	0.07	False
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	10.65 μs	11.68 μs	1.10	0.09	False
DataContractJsonSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	36.52 μs	41.74 μs	1.14	0.10	False
SystemTextJson_Reflection_ - Duration of single invocation 📈 - ADX Test Multi Config Graph	5.54 μs	6.45 μs	1.17	0.14	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream<Location>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_FromStream<Location>.SystemTextJson_SourceGen_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.062637275058654 > 4.686312308807639.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -15.053958034857796 (T) = (0 -5094.791734405136) / Math.Sqrt((13659.617055775221 / (49)) + (6088.820018937004 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.14337239228448628 = (4455.933839915109 - 5094.791734405136) / 4455.933839915109 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<Location>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.684642221391174 > 10.81188632969008.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.959059347490138 (T) = (0 -11640.307223718934) / Math.Sqrt((117001.32490437996 / (49)) + (21353.523958094425 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.11782803804366802 = (10413.32550943243 - 11640.307223718934) / 10413.32550943243 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<Location>.DataContractJsonSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.74354686857261 > 39.285597228314145.
IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.09508311434479 (T) = (0 -41937.517350160895) / Math.Sqrt((648714.3715268668 / (48)) + (101058.99918493505 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.09849614117241903 = (38177.20953065999 - 41937.517350160895) / 38177.20953065999 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<Location>.SystemTextJson_Reflection_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.453526267901527 > 5.495967038852023.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -12.66173608658774 (T) = (0 -6578.190495756338) / Math.Sqrt((17589.835663683945 / (51)) + (37417.35866950243 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.23390356917345417 = (5331.203069752705 - 6578.190495756338) / 5331.203069752705 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DataContractSerializer_BinaryXml_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.24 μs	5.16 μs	1.22	0.21	False
XmlSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	16.80 μs	19.31 μs	1.15	0.10	False
DataContractSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	10.57 μs	11.87 μs	1.12	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>.DataContractSerializer_BinaryXml_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.162177787919018 > 4.486933287508903.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -16.739915911649625 (T) = (0 -5198.812163707522) / Math.Sqrt((14093.456506978144 / (50)) + (10432.972501387883 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.20934157361271752 = (4298.878230223153 - 5198.812163707522) / 4298.878230223153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>.XmlSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.312013462732917 > 17.71669208813318.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.836734759785903 (T) = (0 -18825.184651706666) / Math.Sqrt((107842.41037653945 / (41)) + (953974.5556624011 / (13))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (13) - 2, .025) and -0.11126852325203747 = (16940.2662433165 - 18825.184651706666) / 16940.2662433165 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>.DataContractSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.872293503835273 > 11.077040259278679.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -8.456698231057686 (T) = (0 -11812.851366137336) / Math.Sqrt((62768.30297020849 / (51)) + (88806.39025058957 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.12307274641811213 = (10518.33142938676 - 11812.851366137336) / 10518.33142938676 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in LinqBenchmarks

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Order00LinqQueryX - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	7.01 secs	9.99 secs	1.42	0.04	True
Where01LinqMethodNestedX - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.65 secs	1.86 secs	1.12	0.35	False
Order00LinqMethodX - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	7.03 secs	10.22 secs	1.45	0.08	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

LinqBenchmarks.Order00LinqQueryX

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.989727866733334 > 7.462386329895.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -54.537536443534194 (T) = (0 -10108328395.305954) / Math.Sqrt((11511176557694410 / (49)) + (10840559983598168 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4140167961720248 = (7148662181.857285 - 10108328395.305954) / 7148662181.857285 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Where01LinqMethodNestedX

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8584279997142856 > 1.7371891126125.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.907874415867362 (T) = (0 -1837542738.743119) / Math.Sqrt((5276083966504676 / (39)) + (2511828298313143 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.07145447486235314 = (1714998426.7686064 - 1837542738.743119) / 1714998426.7686064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Order00LinqMethodX

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.217728600133333 > 7.5425748474475.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -39.25327955505472 (T) = (0 -10031047383.333334) / Math.Sqrt((18801097395270710 / (51)) + (19632807514420372 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3971570821146843 = (7179613167.154203 - 10031047383.333334) / 7179613167.154203 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	46.04 μs	56.92 μs	1.24	0.04	True
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	46.08 μs	49.23 μs	1.07	0.06	False
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	47.02 μs	58.63 μs	1.25	0.08	True
SerializeToUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	46.01 μs	55.40 μs	1.20	0.04	False
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	45.19 μs	48.20 μs	1.07	0.11	False
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	46.06 μs	55.11 μs	1.20	0.03	True
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	44.74 μs	48.99 μs	1.10	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeToString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 56.91970808804291 > 49.711412735376605.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.166871389097533 (T) = (0 -57366.76126877651) / Math.Sqrt((1021243.7735932113 / (51)) + (3366044.963904725 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.22052292534499557 = (47001.78921470164 - 57366.76126877651) / 47001.78921470164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeToString(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.231276210937494 > 47.50427956445982.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 2:54:53 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.444593633150706 (T) = (0 -48142.343646129215) / Math.Sqrt((1286494.1611737008 / (38)) + (1860773.2500367046 / (13))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (13) - 2, .025) and -0.06958322825175815 = (45010.37635455283 - 48142.343646129215) / 45010.37635455283 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeObjectProperty(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.63389762126865 > 49.71417815733159.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.989755457207156 (T) = (0 -58631.70702846856) / Math.Sqrt((966710.5616386901 / (49)) + (3151298.0047187433 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.22513773861212105 = (47857.23692985624 - 58631.70702846856) / 47857.23692985624 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeToUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.39855197990544 > 49.027446529907316.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -30.31368367615643 (T) = (0 -55391.88553745548) / Math.Sqrt((1022928.6172868243 / (50)) + (284279.92824142386 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.19841697226373012 = (46220.878725393784 - 55391.88553745548) / 46220.878725393784 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeToWriter(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.197617066563474 > 46.99983939297504.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 9:33:39 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -8.901809251223481 (T) = (0 -48220.5650208722) / Math.Sqrt((1247965.1122327228 / (41)) + (1652701.67318864 / (12))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (12) - 2, .025) and -0.08190302687314037 = (44570.13597626837 - 48220.5650208722) / 44570.13597626837 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeToWriter(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.107365038145545 > 48.862995766912526.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -23.65217995039113 (T) = (0 -55304.055428933265) / Math.Sqrt((748684.7588839732 / (49)) + (428531.56142863235 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.17596242615890773 = (47028.76061233952 - 55304.055428933265) / 47028.76061233952 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeObjectProperty(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.99318542779261 > 48.09923755121903.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.326066850807044 (T) = (0 -48206.0319255042) / Math.Sqrt((713114.8963565578 / (39)) + (2302760.3125525634 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.05943708434154038 = (45501.55232244408 - 48206.0319255042) / 45501.55232244408 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Perf_Frozen<NotKnownComparable>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TryGetValue_True - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	140.94 μs	187.53 μs	1.33	0.20	True
Contains_True - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	131.44 μs	167.82 μs	1.28	0.09	True
ToFrozenSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	27.43 μs	36.54 μs	1.33	0.04	True
ToFrozenDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	17.95 μs	22.47 μs	1.25	0.15	False
ToFrozenSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.12 μs	1.43 μs	1.27	0.16	True
Contains_True - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.49 μs	1.81 μs	1.21	0.02	True
Contains_True - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	16.47 μs	20.84 μs	1.27	0.12	True
ToFrozenDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	157.60 μs	191.45 μs	1.21	0.09	True
TryGetValue_True - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	17.89 μs	22.96 μs	1.28	0.16	False
TryGetValue_True - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.58 μs	1.89 μs	1.20	0.03	True
ToFrozenSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	226.84 μs	303.05 μs	1.34	0.03	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_Frozen<NotKnownComparable>*'

Payloads

Baseline
Compare

System.Collections.Perf_Frozen<NotKnownComparable>.TryGetValue_True(Count: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 187.52570180722893 > 151.12738484852161.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -24.699020046809515 (T) = (0 -186609.3252925352) / Math.Sqrt((11627572.76822654 / (51)) + (11316910.485397637 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.30110865426130984 = (143423.32185814306 - 186609.3252925352) / 143423.32185814306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<NotKnownComparable>.Contains_True(Count: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 167.8201233599291 > 137.9241995106826.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -32.88999375769099 (T) = (0 -168907.85644537094) / Math.Sqrt((9875879.474151442 / (49)) + (3874068.725150278 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2668321449326649 = (133330.88927448142 - 168907.85644537094) / 133330.88927448142 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenSet(Count: 64)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.54370563089623 > 28.835982681585563.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -50.12411765136666 (T) = (0 -36217.21230443293) / Math.Sqrt((349985.5324037129 / (50)) + (100383.73865391065 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.32968940025426613 = (27237.34753206831 - 36217.21230443293) / 27237.34753206831 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenDictionary(Count: 64)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.468287019230768 > 18.886983913356662.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -45.58447105804024 (T) = (0 -22330.10864010989) / Math.Sqrt((201987.86766657844 / (50)) + (14922.000847368636 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.24040433688222748 = (18002.281978662628 - 22330.10864010989) / 18002.281978662628 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenSet(Count: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4254705586685854 > 1.1681490404030932.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.83876620694512 (T) = (0 -1413.7349329434633) / Math.Sqrt((1254.117315701998 / (50)) + (1493.4310644077611 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.26504321547156257 = (1117.5388442492645 - 1413.7349329434633) / 1117.5388442492645 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<NotKnownComparable>.Contains_True(Count: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8074435411278462 > 1.5535341111917818.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -23.056386022521227 (T) = (0 -1776.8634892321331) / Math.Sqrt((637.1732885685561 / (51)) + (595.9710633672181 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.19745290921193534 = (1483.869198999665 - 1776.8634892321331) / 1483.869198999665 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<NotKnownComparable>.Contains_True(Count: 64)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.84470124177293 > 17.193976228079688.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -40.19736223907399 (T) = (0 -20952.253305873786) / Math.Sqrt((75733.59398801347 / (51)) + (42648.591081409504 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.268147109831158 = (16521.942244274313 - 20952.253305873786) / 16521.942244274313 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenDictionary(Count: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 191.45355711382112 > 162.9357782453589.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -32.45350546971243 (T) = (0 -192981.89410477527) / Math.Sqrt((16874882.323199034 / (51)) + (3418567.2738430034 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.22414730030041638 = (157645.9745141911 - 192981.89410477527) / 157645.9745141911 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<NotKnownComparable>.TryGetValue_True(Count: 64)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.960618267350927 > 18.675792772639348.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -16.753922952685414 (T) = (0 -22499.279350447974) / Math.Sqrt((241690.79754653404 / (51)) + (211033.09583993946 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.256347919926652 = (17908.478211801015 - 22499.279350447974) / 17908.478211801015 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<NotKnownComparable>.TryGetValue_True(Count: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.894349565276766 > 1.6660520283694755.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.643935502433052 (T) = (0 -1874.8028876635576) / Math.Sqrt((586.4826800767423 / (48)) + (669.434821022196 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.19310866707356814 = (1571.3597088034192 - 1874.8028876635576) / 1571.3597088034192 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenSet(Count: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 303.04653039148354 > 239.0482241621377.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -56.20878782195327 (T) = (0 -301819.8879048235) / Math.Sqrt((16823197.84445803 / (49)) + (5161475.865782938 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.31239778647846306 = (229975.9196597643 - 301819.8879048235) / 229975.9196597643 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.55 μs	5.67 μs	1.25	0.04	True
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.12 μs	6.32 μs	1.23	0.06	True
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	6.21 μs	7.80 μs	1.26	0.07	False
SerializeToUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.68 μs	6.04 μs	1.29	0.06	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.6728911390790095 > 4.7412343558376095.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.611833417192312 (T) = (0 -5685.166159332055) / Math.Sqrt((10767.926414661657 / (51)) + (26400.162004391776 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.269221729712672 = (4479.253723948667 - 5685.166159332055) / 4479.253723948667 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.319788668274771 > 5.247581859530345.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -30.304422524940563 (T) = (0 -6228.030642317533) / Math.Sqrt((15241.044837807749 / (50)) + (5646.972011574127 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2525011734574279 = (4972.474896071802 - 6228.030642317533) / 4972.474896071802 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeObjectProperty(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.797081569802758 > 6.556452128210475.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.473255572251066 (T) = (0 -7631.893800085111) / Math.Sqrt((23732.2997897881 / (49)) + (12225.811866310129 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2124444194284261 = (6294.633945927979 - 7631.893800085111) / 6294.633945927979 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.043091579595414 > 4.924109214755985.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -17.714316126673424 (T) = (0 -6056.896574623868) / Math.Sqrt((10671.399696021617 / (51)) + (21419.40015820182 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2790310368310573 = (4735.535260841291 - 6056.896574623868) / 4735.535260841291 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
XmlSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.24 ms	1.43 ms	1.15	0.11	False
DataContractSerializer_BinaryXml_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.57 ms	2.12 ms	1.35	0.12	True
DataContractSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.42 ms	3.02 ms	1.25	0.06	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.XmlSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4282390466720776 > 1.3086348996875001.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.27111505720311 (T) = (0 -1419573.3800818971) / Math.Sqrt((1008470604.8357935 / (51)) + (238620970.81095603 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.13763971739569011 = (1247823.329631648 - 1419573.3800818971) / 1247823.329631648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.DataContractSerializer_BinaryXml_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1217881378531076 > 1.6735595015865388.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -49.547685486031476 (T) = (0 -2121036.8990004347) / Math.Sqrt((2982548091.8623176 / (50)) + (177012326.14246836 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.3125384056786555 = (1615980.8275505207 - 2121036.8990004347) / 1615980.8275505207 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.DataContractSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.0184505031328315 > 2.529633327387921.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -31.18431097197464 (T) = (0 -2992260.5596147664) / Math.Sqrt((2356421967.1080465 / (49)) + (1055498078.1518055 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.22559947352504234 = (2441466.9100733963 - 2992260.5596147664) / 2441466.9100733963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Parse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	974.76 ns	1.07 μs	1.10	0.10	False
Parse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	977.00 ns	1.14 μs	1.17	0.09	False
Parse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.54 μs	2.70 μs	1.06	0.05	False
Parse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.53 μs	1.66 μs	1.09	0.24	False
Parse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	6.22 μs	6.82 μs	1.10	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'

Payloads

Baseline
Compare

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: HelloWorld)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0745204572322669 > 1027.1456786563729.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -12.949878680947515 (T) = (0 -1078.366261377906) / Math.Sqrt((732.2343261040229 / (49)) + (199.43500041496839 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.10701595313044344 = (974.1198926072193 - 1078.366261377906) / 974.1198926072193 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: HelloWorld)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1423843063033348 > 1025.4515247811275.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -5.321831933140008 (T) = (0 -1136.944731528778) / Math.Sqrt((615.5297468106642 / (49)) + (2757.1440917287 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.14156055023603187 = (995.956571286298 - 1136.944731528778) / 995.956571286298 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: BasicJson)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.70043805573012 > 2.6585310245714733.
IsChangePoint: Marked as a change because one of 9/17/2023 1:05:21 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.385269607290406 (T) = (0 -2759.081142477862) / Math.Sqrt((2167.2190664756017 / (51)) + (3747.4974002295307 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.07807982410492277 = (2559.2549649731113 - 2759.081142477862) / 2559.2549649731113 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: HelloWorld)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6639536354299203 > 1.625413948345632.
IsChangePoint: Marked as a change because one of 9/25/2023 11:21:57 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.672178529561361 (T) = (0 -1647.1350067157255) / Math.Sqrt((2497.8431633835 / (42)) + (2178.8957936884035 / (11))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (11) - 2, .025) and -0.06952884609805299 = (1540.056645246124 - 1647.1350067157255) / 1540.056645246124 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: BasicJson)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.815552178070928 > 6.558403144441262.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.838749783817117 (T) = (0 -6566.8796710268625) / Math.Sqrt((16487.391188068395 / (40)) + (21129.784520131936 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.0552132215671444 = (6223.272734655557 - 6566.8796710268625) / 6223.272734655557 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.57 ms	1.86 ms	1.18	0.08	False
SerializeToUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.57 ms	1.81 ms	1.15	0.05	False
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.62 ms	1.86 ms	1.15	0.04	False
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.48 ms	1.76 ms	1.19	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8560758159722222 > 1.6477200888194445.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -17.55455391799516 (T) = (0 -1862234.925173611) / Math.Sqrt((1552898006.0431278 / (50)) + (1032830432.3815972 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.1909382540545702 = (1563670.424418394 - 1862234.925173611) / 1563670.424418394 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8074640697916666 > 1.623785377447917.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -39.56256300834903 (T) = (0 -1805568.4214890255) / Math.Sqrt((868286514.186556 / (51)) + (119501338.35169764 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17655102860181376 = (1534628.2291170333 - 1805568.4214890255) / 1534628.2291170333 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8581884880952382 > 1.636573529621528.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -35.09464431057338 (T) = (0 -1839676.3203136923) / Math.Sqrt((767071777.3756245 / (49)) + (184491986.34413576 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.1763861944238485 = (1563837.0537106646 - 1839676.3203136923) / 1563837.0537106646 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToWriter(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7593194449786325 > 1.5609549995880683.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -21.061210735799825 (T) = (0 -1791303.400963367) / Math.Sqrt((1131505259.6319342 / (51)) + (604906296.286138 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.18319383114018678 = (1513956.0009683066 - 1791303.400963367) / 1513956.0009683066 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Sort<IntStruct>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Array_Comparison - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	483.28 μs	773.39 μs	1.60	0.19	True
LinqQuery - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.36 ms	1.82 ms	1.33	0.08	True
LinqOrderByExtension - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.32 ms	1.81 ms	1.37	0.06	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort<IntStruct>*'

Payloads

Baseline
Compare

System.Collections.Sort<IntStruct>.Array_Comparison(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 773.3941334166667 > 508.27026587346154.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.709920711079594 (T) = (0 -812831.7475008334) / Math.Sqrt((138378782.349969 / (49)) + (1200194729.7141006 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.6683104499620695 = (487218.5195023587 - 812831.7475008334) / 487218.5195023587 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<IntStruct>.LinqQuery(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.81570602672 > 1.385719845972.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -31.145875115290973 (T) = (0 -1790623.0033433335) / Math.Sqrt((1242403595.7024255 / (51)) + (797833965.2331177 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3517581696959129 = (1324662.2387687482 - 1790623.0033433335) / 1324662.2387687482 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<IntStruct>.LinqOrderByExtension(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.81135576 > 1.3795573050490002.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -60.84864364166618 (T) = (0 -1791719.1833433332) / Math.Sqrt((304905560.67038685 / (50)) + (224344038.066995 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.3657587205791113 = (1311885.5888275825 - 1791719.1833433332) / 1311885.5888275825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Sort<BigStruct>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
LinqOrderByExtension - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.40 ms	1.86 ms	1.32	0.03	False
Array_Comparison - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	532.57 μs	863.29 μs	1.62	0.10	False
LinqQuery - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.42 ms	1.88 ms	1.32	0.04	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort<BigStruct>*'

Payloads

Baseline
Compare

System.Collections.Sort<BigStruct>.LinqOrderByExtension(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8589082132666668 > 1.4894973310699997.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -38.519011972393926 (T) = (0 -1881126.3332933334) / Math.Sqrt((624249600.743951 / (51)) + (378044811.1787607 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.317119526284035 = (1428212.3191966643 - 1881126.3332933334) / 1428212.3191966643 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.Array_Comparison(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 863.2941000166667 > 573.0621115160001.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -26.41004588406041 (T) = (0 -888691.9563539681) / Math.Sqrt((253099389.86813015 / (50)) + (500102648.9657317 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.6379416993969469 = (542566.2932210374 - 888691.9563539681) / 542566.2932210374 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.LinqQuery(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.877070213293333 > 1.4913045799499998.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.086730681392647 (T) = (0 -1921059.3607215383) / Math.Sqrt((407717700.4901247 / (48)) + (4880449509.601135 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.3459408035464758 = (1427298.5525512404 - 1921059.3607215383) / 1427298.5525512404 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TwoArguments_DefineMessage - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.73 μs	1.99 μs	1.15	0.11	False
NoArguments - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	367.02 ns	407.44 ns	1.11	0.10	False
FourArguments_DefineMessage - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.32 μs	1.41 μs	1.07	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'

Payloads

Baseline
Compare

Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments_DefineMessage

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9921020095618853 > 1.8172953536667964.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.271790373362695 (T) = (0 -2013.2291452367042) / Math.Sqrt((1687.8064615742883 / (48)) + (1927.5894977744101 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.17633465418866887 = (1711.4425202624425 - 2013.2291452367042) / 1711.4425202624425 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Logging.FormattingOverhead.NoArguments

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 407.44404276286735 > 388.31142162979603.
IsChangePoint: Marked as a change because one of 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -5.059948397173813 (T) = (0 -391.2503502551013) / Math.Sqrt((81.86360856630607 / (40)) + (226.60063682090615 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.05844978698848201 = (369.64469648417895 - 391.2503502551013) / 369.64469648417895 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4088286717070808 > 1.3884849273466155.
IsChangePoint: Marked as a change because one of 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -5.688533101546755 (T) = (0 -1406.9258539350237) / Math.Sqrt((940.7208631565971 / (39)) + (2549.4942772645004 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.061642607995941276 = (1325.234917418087 - 1406.9258539350237) / 1325.234917418087 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
GetService_4Injected - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.09 μs	2.91 μs	1.39	0.11	True
GetService_0Injected - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	910.65 ns	1.27 μs	1.40	0.11	True
GetService_3Injected - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.81 μs	2.61 μs	1.44	0.11	True
GetService_2Injected - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.55 μs	2.09 μs	1.35	0.09	True
GetService_5Injected - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.94 μs	6.09 μs	1.23	0.11	False
GetService_1Injected - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.22 μs	1.69 μs	1.39	0.14	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'

Payloads

Baseline
Compare

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_4Injected

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.9056073789571695 > 2.2064085438826297.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -28.643409775383013 (T) = (0 -2889.446269542152) / Math.Sqrt((4267.651277425106 / (51)) + (2524.5702223456396 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.36061557949741435 = (2123.631621658675 - 2889.446269542152) / 2123.631621658675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_0Injected

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2722343773975757 > 926.986109788165.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -39.506701022785805 (T) = (0 -1246.9935597690342) / Math.Sqrt((683.0839628809694 / (51)) + (305.67000555737667 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4290663428460831 = (872.5931906601071 - 1246.9935597690342) / 872.5931906601071 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_3Injected

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6115747709551655 > 1.9060114423584065.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.730110559081242 (T) = (0 -2456.321978536238) / Math.Sqrt((2147.251396102249 / (52)) + (11322.258226528478 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.3439351833600871 = (1827.7086640406108 - 2456.321978536238) / 1827.7086640406108 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_2Injected

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0899965197527544 > 1.6301415767350702.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -45.240940468279454 (T) = (0 -2093.5723195169526) / Math.Sqrt((1137.9228906342473 / (51)) + (500.0947558617145 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3555642154747936 = (1544.4287298360614 - 2093.5723195169526) / 1544.4287298360614 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_5Injected

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.088678178712313 > 5.257858499738231.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -16.004412608845374 (T) = (0 -6037.170545433062) / Math.Sqrt((14467.912573049 / (49)) + (16471.899987358138 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.21375484744803847 = (4973.962046888152 - 6037.170545433062) / 4973.962046888152 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_1Injected

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6864262602992197 > 1.2958261793163262.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -17.623043990613425 (T) = (0 -1704.5621416864815) / Math.Sqrt((1510.9996908906605 / (49)) + (2603.3958514346673 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.36974218936874886 = (1244.4401252414045 - 1704.5621416864815) / 1244.4401252414045 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Threading.Channels.Tests.BoundedChannelPerfTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TryWriteThenTryRead - Duration of single invocation 📈 - ADX Test Multi Config Graph	578.15 ns	813.95 ns	1.41	0.26	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.BoundedChannelPerfTests*'

Payloads

Baseline
Compare

System.Threading.Channels.Tests.BoundedChannelPerfTests.TryWriteThenTryRead

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 813.9472104638559 > 597.7637272704812.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -35.95077300618256 (T) = (0 -833.7352607856044) / Math.Sqrt((374.36836314064885 / (51)) + (176.8974034583329 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4485197988562041 = (575.577400767285 - 833.7352607856044) / 575.577400767285 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<Location>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	6.36 μs	7.80 μs	1.22	0.13	False
SystemTextJson_Reflection_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.51 μs	4.38 μs	1.25	0.03	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<Location>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_ToString<Location>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.795293211175767 > 6.753395267665461.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.873076444092685 (T) = (0 -7529.907557888454) / Math.Sqrt((42664.09114489326 / (49)) + (35039.102675109265 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.18306533854477672 = (6364.743613527362 - 7529.907557888454) / 6364.743613527362 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<Location>.SystemTextJson_Reflection_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.382192952307168 > 3.724286215826771.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.17556198283109 (T) = (0 -4286.0677871922835) / Math.Sqrt((6450.542925774637 / (49)) + (5495.168222080848 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.21005047617161338 = (3542.0570229042405 - 4286.0677871922835) / 3542.0570229042405 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<Location>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.74 μs	4.25 μs	1.14	0.06	False
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.33 μs	5.32 μs	1.23	0.06	False
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.97 μs	3.73 μs	1.25	0.05	True
SerializeToUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.26 μs	4.09 μs	1.26	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<Location>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.248775824644318 > 3.727445103626883.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.31642564829539 (T) = (0 -4268.289769002717) / Math.Sqrt((9395.675022935498 / (51)) + (4575.614620441105 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.19750109357626222 = (3564.3305813239267 - 4268.289769002717) / 3564.3305813239267 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeObjectProperty(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.324377477362762 > 4.5258938376182085.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.943489523721219 (T) = (0 -5314.581141577098) / Math.Sqrt((10964.876359415683 / (49)) + (15590.851279312848 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2202764789848623 = (4355.227059689172 - 5314.581141577098) / 4355.227059689172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToWriter(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.728687965141146 > 3.1171112482342473.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -12.282440419059807 (T) = (0 -3738.086599317912) / Math.Sqrt((4745.814373789386 / (49)) + (13938.219217119813 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.24476518886649382 = (3003.0455806061564 - 3738.086599317912) / 3003.0455806061564 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.094908498521996 > 3.4512919692438184.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -20.13754476615844 (T) = (0 -4052.425622330989) / Math.Sqrt((8581.585713826615 / (50)) + (4631.416382947271 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.22128398357832466 = (3318.1681548443025 - 4052.425622330989) / 3318.1681548443025 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeToUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	62.10 μs	78.28 μs	1.26	0.03	True
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	68.39 μs	84.89 μs	1.24	0.08	True
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	66.56 μs	82.74 μs	1.24	0.05	True
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	50.17 μs	67.11 μs	1.34	0.13	True
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	54.71 μs	69.41 μs	1.27	0.08	True
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	56.17 μs	70.98 μs	1.26	0.11	True
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	64.66 μs	78.38 μs	1.21	0.09	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.27548130234507 > 65.76856722473141.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -49.675051714096824 (T) = (0 -78523.00741344581) / Math.Sqrt((1954650.1120229121 / (50)) + (229725.8968117725 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.24462346613851788 = (63089.76935576012 - 78523.00741344581) / 63089.76935576012 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeObjectProperty(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.88510588951083 > 71.38636576615758.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.97853244851889 (T) = (0 -83528.48508671646) / Math.Sqrt((1802279.392041349 / (49)) + (1028205.0987115636 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.22184918319591734 = (68362.35292823623 - 83528.48508671646) / 68362.35292823623 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 82.73748579977205 > 71.31902199732464.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -40.80835576363664 (T) = (0 -82289.92447429968) / Math.Sqrt((2137429.3249177486 / (51)) + (395262.4941146444 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2285507881789993 = (66981.29639090678 - 82289.92447429968) / 66981.29639090678 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToWriter(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.11237686653327 > 52.83981798333977.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.70955163682225 (T) = (0 -68673.89402323433) / Math.Sqrt((2430805.4014970516 / (49)) + (8702812.438107016 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.3411246802916597 = (51206.196584421625 - 68673.89402323433) / 51206.196584421625 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToString(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.40755883112094 > 59.23357230724887.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -28.250660132043564 (T) = (0 -70752.08747541308) / Math.Sqrt((1615169.2621884958 / (47)) + (836233.0857821776 / (4))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (4) - 2, .025) and -0.2453331320387827 = (56813.78392268591 - 70752.08747541308) / 56813.78392268591 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.97659782697326 > 60.705017556179776.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -26.673441898860325 (T) = (0 -71559.94397819055) / Math.Sqrt((1795856.812720555 / (49)) + (961545.63176216 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.24407560530641445 = (57520.57485329874 - 71559.94397819055) / 57520.57485329874 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToWriter(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.3795190920398 > 67.39235381287568.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -32.216988451134 (T) = (0 -77948.19264686778) / Math.Sqrt((2460380.7489516786 / (49)) + (712108.661901841 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.24604007195993752 = (62556.7302376243 - 77948.19264686778) / 62556.7302376243 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Reflection.Invoke

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Method0_NoParms - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	307.99 ns	394.69 ns	1.28	0.20	False
Ctor0_NoParams - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	164.49 ns	234.43 ns	1.43	0.22	True
Property_Get_class - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	169.70 ns	247.11 ns	1.46	0.32	True
Property_Get_int - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	199.97 ns	270.23 ns	1.35	0.22	True
StaticMethod4_ByRefParams_int_string_struct_class - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	811.80 ns	927.98 ns	1.14	0.08	False
Ctor4_ActivatorCreateInstance - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.07 μs	4.91 μs	1.21	0.27	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'

Payloads

Baseline
Compare

System.Reflection.Invoke.Method0_NoParms

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 394.68895854166664 > 323.71749863238125.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.243476247148937 (T) = (0 -391.9376125835239) / Math.Sqrt((71.21205531316441 / (51)) + (45.60596392926279 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2547474143883566 = (312.363753922999 - 391.9376125835239) / 312.363753922999 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.Ctor0_NoParams

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 234.42503904584223 > 167.48086040880432.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -34.129831601630904 (T) = (0 -235.98665665603335) / Math.Sqrt((18.434074591244297 / (50)) + (18.685053561859853 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.4807861283163441 = (159.36579371144603 - 235.98665665603335) / 159.36579371144603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.Property_Get_class

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 247.10651067708335 > 178.42019455319817.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -26.885693316541452 (T) = (0 -242.19571457200755) / Math.Sqrt((26.738379242722434 / (49)) + (25.380597892273666 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4112347915585935 = (171.61971630852594 - 242.19571457200755) / 171.61971630852594 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.Property_Get_int

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 270.2292962592365 > 209.32416971205876.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.414473145962813 (T) = (0 -276.5579006643851) / Math.Sqrt((39.66107824305392 / (51)) + (42.458091761792055 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.37656842342675606 = (200.9038533485583 - 276.5579006643851) / 200.9038533485583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.StaticMethod4_ByRefParams_int_string_struct_class

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 927.9831969537817 > 850.0272501328126.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -8.946964031485004 (T) = (0 -925.7561827667873) / Math.Sqrt((213.28207086355633 / (51)) + (606.3859092103288 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.13716966044421092 = (814.0880072417343 - 925.7561827667873) / 814.0880072417343 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.Ctor4_ActivatorCreateInstance

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.909723602083333 > 4.36762046984375.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.171227403115338 (T) = (0 -4888.4478344866075) / Math.Sqrt((27284.70352179208 / (49)) + (5317.710988095822 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.14403816946807488 = (4272.976168932817 - 4888.4478344866075) / 4272.976168932817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.40 μs	3.09 μs	1.29	0.09	True
SerializeToUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.75 μs	3.29 μs	1.20	0.06	True
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.68 μs	4.47 μs	1.22	0.13	False
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.00 μs	3.94 μs	1.32	0.04	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToWriter(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.0921917176904334 > 2.5144395759214393.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -28.08972432584993 (T) = (0 -3061.1546916830534) / Math.Sqrt((3693.6054543107766 / (52)) + (1984.9388272780498 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.27967619973249563 = (2392.1322380794136 - 3061.1546916830534) / 2392.1322380794136 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.2887926167219783 > 2.7967871370184216.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -16.82887728440361 (T) = (0 -3361.598340702436) / Math.Sqrt((4194.018251397944 / (48)) + (5970.832375167887 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.24843678105202596 = (2692.6460287958694 - 3361.598340702436) / 2692.6460287958694 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeObjectProperty(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.474734952655414 > 3.8021180278362614.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.302551254683225 (T) = (0 -4494.436028954796) / Math.Sqrt((10424.439877489742 / (51)) + (10390.006856843189 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.2183975808073977 = (3688.8090552317576 - 4494.436028954796) / 3688.8090552317576 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.943661716224264 > 3.0483833522213915.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -9.580587012964848 (T) = (0 -3701.906023287948) / Math.Sqrt((5299.796848232939 / (50)) + (28148.139884363787 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.27996693724159233 = (2892.188786739901 - 3701.906023287948) / 2892.188786739901 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.09 μs	2.59 μs	1.23	0.16	False
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.51 μs	3.15 μs	1.26	0.10	False
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.64 μs	2.16 μs	1.32	0.19	False
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.93 μs	2.46 μs	1.28	0.12	True
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.80 μs	3.39 μs	1.21	0.14	False
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.29 μs	2.97 μs	1.30	0.10	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.5861451145412406 > 2.181477995728843.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -30.02265931401307 (T) = (0 -2613.322315418126) / Math.Sqrt((5174.571255480067 / (48)) + (896.0772031048158 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.26465600181740506 = (2066.4293781570536 - 2613.322315418126) / 2066.4293781570536 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.152799716484407 > 2.672274532125848.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -15.6604862570426 (T) = (0 -3068.9039675156887) / Math.Sqrt((7875.802973165859 / (50)) + (3590.8128368820053 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.19870325707032754 = (2560.1865594460774 - 3068.9039675156887) / 2560.1865594460774 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.16260035773826 > 1.7507870811299875.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -32.01764609750642 (T) = (0 -2167.02188040557) / Math.Sqrt((4443.388751310821 / (51)) + (440.5553466110435 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2618540261823942 = (1717.3316686730122 - 2167.02188040557) / 1717.3316686730122 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.464019559474733 > 2.0219334510104225.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.72064608762096 (T) = (0 -2486.2672669576564) / Math.Sqrt((4542.453554770665 / (51)) + (2932.5445940129975 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2943945445738267 = (1920.7955390265092 - 2486.2672669576564) / 1920.7955390265092 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3882350662696084 > 2.824010632056791.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.036231744765175 (T) = (0 -3409.4660353340178) / Math.Sqrt((5718.967196043675 / (51)) + (4857.375788245274 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2386479929943872 = (2752.5705887527865 - 3409.4660353340178) / 2752.5705887527865 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.971730186463653 > 2.3697792146011865.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -25.99749005372781 (T) = (0 -2905.032944674799) / Math.Sqrt((4542.996568152694 / (51)) + (2054.429404384109 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2815548446413013 = (2266.8034511530395 - 2905.032944674799) / 2266.8034511530395 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CreateAddAndClear<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.49 ms	2.99 ms	1.20	0.04	False
SortedDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	537.68 μs	748.87 μs	1.39	0.12	True
ImmutableSortedSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	679.34 μs	738.21 μs	1.09	0.28	False
ImmutableHashSet - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.40 ms	2.84 ms	1.18	0.04	False
ImmutableList - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	497.89 μs	542.46 μs	1.09	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<Int32>*'

Payloads

Baseline
Compare

System.Collections.CreateAddAndClear<Int32>.ImmutableDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.9858941798941805 > 2.621881162832643.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -16.446913649153696 (T) = (0 -2984784.4938051743) / Math.Sqrt((2067873083.9813106 / (49)) + (3235468399.4946785 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.19154150146802476 = (2504977.3676601322 - 2984784.4938051743) / 2504977.3676601322 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.SortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 748.8672352272727 > 567.461295093598.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.332624268407766 (T) = (0 -735658.8536756213) / Math.Sqrt((204461049.19016764 / (50)) + (412459290.46724373 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.3477456365624889 = (545843.9884487151 - 735658.8536756213) / 545843.9884487151 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.ImmutableSortedSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 738.2073871212119 > 725.3015761820651.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.210970785986003 (T) = (0 -754927.91241416) / Math.Sqrt((977373101.2897781 / (41)) + (532357665.06628203 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.08123173721422085 = (698211.0184438552 - 754927.91241416) / 698211.0184438552 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.ImmutableHashSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.8350835233716474 > 2.5207850010679613.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.727808816795875 (T) = (0 -2827614.8278566636) / Math.Sqrt((1990782505.3960922 / (51)) + (1764707623.1097908 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.1697803744856729 = (2417218.556175474 - 2827614.8278566636) / 2417218.556175474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.ImmutableList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 542.4551556697613 > 517.7379225640792.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -6.14406746274229 (T) = (0 -539943.1514444283) / Math.Sqrt((793818965.5356381 / (41)) + (133952405.78559688 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.06519387837451289 = (506896.5963908675 - 539943.1514444283) / 506896.5963908675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	127.16 μs	167.89 μs	1.32	0.14	True
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	103.70 μs	147.11 μs	1.42	0.09	True
SerializeToUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	103.66 μs	136.19 μs	1.31	0.13	True
SerializeToUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	120.60 μs	163.72 μs	1.36	0.09	True
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	125.94 μs	171.18 μs	1.36	0.09	True
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	101.24 μs	134.06 μs	1.32	0.10	True
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	105.68 μs	145.50 μs	1.38	0.13	False
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	120.86 μs	169.92 μs	1.41	0.08	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeObjectProperty(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 167.88872549019612 > 133.2248610051186.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -39.06524888261003 (T) = (0 -168605.2277874075) / Math.Sqrt((15023339.842710463 / (49)) + (3444916.6611710875 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.33401882883954753 = (126388.94155195385 - 168605.2277874075) / 126388.94155195385 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 147.1094838888889 > 114.076444406596.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.317425232610375 (T) = (0 -145050.73857658915) / Math.Sqrt((7384042.832407976 / (50)) + (10293963.144434439 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.34013430589813565 = (108235.971527778 - 145050.73857658915) / 108235.971527778 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 136.18680665024633 > 107.74602593708882.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.0233541380518 (T) = (0 -138163.63674553973) / Math.Sqrt((7235496.11851829 / (48)) + (6879908.672579545 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.36521723821215984 = (101202.67520682198 - 138163.63674553973) / 101202.67520682198 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 163.71509384498484 > 126.95054933444653.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -36.39701567626639 (T) = (0 -164533.40317094116) / Math.Sqrt((10493364.403887518 / (51)) + (4535512.525620164 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3441544210821161 = (122406.62277365646 - 164533.40317094116) / 122406.62277365646 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 171.17541652777783 > 131.78596582721818.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -47.35965263027709 (T) = (0 -169794.68633572236) / Math.Sqrt((11307635.511597993 / (50)) + (2373398.0128889 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.33779066126738466 = (126921.7159692711 - 169794.68633572236) / 126921.7159692711 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 134.05901623563219 > 104.19077583677903.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -42.9112931292594 (T) = (0 -135785.97242726292) / Math.Sqrt((10907547.688068427 / (51)) + (1859286.652608509 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.35198695517675105 = (100434.38060355475 - 135785.97242726292) / 100434.38060355475 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 145.49582781208278 > 110.40797641192921.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -47.51667943413278 (T) = (0 -144145.42728507397) / Math.Sqrt((8465682.630414145 / (50)) + (1378907.4259793875 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.3539652265496512 = (106461.6907868483 - 144145.42728507397) / 106461.6907868483 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 169.91611171296296 > 127.8736565048077.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -28.73464802056134 (T) = (0 -167577.94976808914) / Math.Sqrt((9398800.047543913 / (51)) + (9515135.100011064 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3783924528605796 = (121574.91824647937 - 167577.94976808914) / 121574.91824647937 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SortedDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	477.69 μs	672.34 μs	1.41	0.16	True
ImmutableSortedDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	128.68 μs	142.74 μs	1.11	0.20	False
SortedList - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	161.13 μs	240.61 μs	1.49	0.14	True
ImmutableDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	104.24 μs	109.65 μs	1.05	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse<Int32, Int32>*'

Payloads

Baseline
Compare

System.Collections.TryGetValueFalse<Int32, Int32>.SortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 672.3427789208075 > 489.88467719301474.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -29.71309396568075 (T) = (0 -661217.1251825256) / Math.Sqrt((117704235.83952145 / (50)) + (161905667.4485155 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.41661132698735154 = (466759.7333057534 - 661217.1251825256) / 466759.7333057534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 142.7377533994102 > 133.97938325094265.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.657029237517161 (T) = (0 -140575.55968502726) / Math.Sqrt((16150604.23155342 / (41)) + (19186023.279671375 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.07799355198805741 = (130404.82424572483 - 140575.55968502726) / 130404.82424572483 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueFalse<Int32, Int32>.SortedList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 240.6146975961539 > 170.22095519750792.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -71.21835778150226 (T) = (0 -239283.86656598613) / Math.Sqrt((14285548.535001978 / (49)) + (3067334.672049688 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.44132765224294424 = (166016.2879645172 - 239283.86656598613) / 166016.2879645172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 109.6532745091447 > 109.13809776667833.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -10.176330785889414 (T) = (0 -113155.34320930377) / Math.Sqrt((7536030.9707502555 / (40)) + (6597765.562727621 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.07879885530227632 = (104890.12168778949 - 113155.34320930377) / 104890.12168778949 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	90.51 μs	106.49 μs	1.18	0.04	False
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	97.52 μs	124.74 μs	1.28	0.04	False
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	109.43 μs	128.78 μs	1.18	0.07	False
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	108.04 μs	127.53 μs	1.18	0.06	False
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	90.83 μs	104.93 μs	1.16	0.02	False
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	124.75 μs	145.68 μs	1.17	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 106.4899471153846 > 97.40781245710788.
IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -20.91557181856686 (T) = (0 -107761.29663729129) / Math.Sqrt((4401596.352107561 / (49)) + (1743589.4285413392 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.16377851846589728 = (92596.0523651383 - 107761.29663729129) / 92596.0523651383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 124.74173221784777 > 103.97118094634433.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -23.438020012334178 (T) = (0 -122304.62914001662) / Math.Sqrt((5870009.959744048 / (50)) + (3405885.872191407 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2324835872598023 = (99234.28628525446 - 122304.62914001662) / 99234.28628525446 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 128.77930447658403 > 116.12188473779324.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.930600250340774 (T) = (0 -130053.34781786126) / Math.Sqrt((9163413.58467756 / (49)) + (2287479.5580802877 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.15406958536927653 = (112691.07986781151 - 130053.34781786126) / 112691.07986781151 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.52534108606558 > 107.3762216732166.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -19.731538127994444 (T) = (0 -126472.49831172849) / Math.Sqrt((5014837.049740893 / (49)) + (5238046.0218339255 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.22756431806318989 = (103027.18680457622 - 126472.49831172849) / 103027.18680457622 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 104.92762359344424 > 93.87629148103935.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -30.910472790657433 (T) = (0 -103820.8503510304) / Math.Sqrt((3567546.172518368 / (51)) + (566175.572149329 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.15864327352916666 = (89605.5349588295 - 103820.8503510304) / 89605.5349588295 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 145.67823216510902 > 129.71911742861172.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -28.886076893777968 (T) = (0 -146461.96220861364) / Math.Sqrt((7173543.363626883 / (48)) + (1941379.0137529105 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.18643320636846064 = (123447.28841239813 - 146461.96220861364) / 123447.28841239813 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SystemTextJson_SourceGen_ - Duration of single invocation 📈 - ADX Test Multi Config Graph	2.03 μs	2.42 μs	1.19	0.09	True
SystemTextJson_Reflection_ - Duration of single invocation 📈 - ADX Test Multi Config Graph	2.26 μs	2.96 μs	1.31	0.13	True
DataContractJsonSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	15.90 μs	16.76 μs	1.05	0.03	False
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.81 μs	5.80 μs	1.21	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.SystemTextJson_SourceGen_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4186503322742694 > 2.1138731043666485.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -8.392358568370218 (T) = (0 -2503.702281074288) / Math.Sqrt((1753.4981490799553 / (51)) + (14046.44404730437 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.24938231873597272 = (2003.9520677764497 - 2503.702281074288) / 2003.9520677764497 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.SystemTextJson_Reflection_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.9623363091730504 > 2.402294092927051.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -39.85784013207219 (T) = (0 -2986.2747248284904) / Math.Sqrt((4095.2846237446265 / (50)) + (894.7698408364362 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.3043319029659087 = (2289.5052386881184 - 2986.2747248284904) / 2289.5052386881184 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.DataContractJsonSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.76129792787944 > 16.562799414157215.
IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/25/2023 7:14:09 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.7378045545839615 (T) = (0 -16877.629897355506) / Math.Sqrt((93408.53374891964 / (44)) + (249210.22203319482 / (9))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (9) - 2, .025) and -0.08596562414511184 = (15541.587617602376 - 16877.629897355506) / 15541.587617602376 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.799655861111112 > 5.193780729546862.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 10:28:34 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.789273168571568 (T) = (0 -5637.140646116426) / Math.Sqrt((24705.889269713618 / (45)) + (48355.72932551139 / (8))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (8) - 2, .025) and -0.1263807163554262 = (5004.649462000948 - 5637.140646116426) / 5004.649462000948 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.Logging.EventSourceLogger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
NestedScopes_TwoMessages - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	871.17 ns	1.18 μs	1.35	0.13	True
NestedScopes_TwoMessages - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	922.64 ns	1.20 μs	1.30	0.08	True
NestedScopes_TwoMessages - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	6.39 μs	7.17 μs	1.12	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.EventSourceLogger*'

Payloads

Baseline
Compare

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: False, Json: False)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1778345048269878 > 934.7803172323318.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -21.268986510359245 (T) = (0 -1185.4880109126943) / Math.Sqrt((540.7913171318789 / (49)) + (733.0570884392902 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.3334837584582256 = (889.0157104600629 - 1185.4880109126943) / 889.0157104600629 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: False, Json: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1976037476358226 > 955.3878390227565.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.508210506118473 (T) = (0 -1177.480866155432) / Math.Sqrt((592.6263083719042 / (51)) + (377.83111546825535 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3168534001567813 = (894.1624527189163 - 1177.480866155432) / 894.1624527189163 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: True, Json: False)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.173946441779374 > 6.611144546691158.
IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/26/2023 6:21:18 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.355130199478978 (T) = (0 -6954.953068467188) / Math.Sqrt((13916.869976165463 / (49)) + (15622.723101545043 / (6))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (6) - 2, .025) and -0.11506346151198914 = (6237.271068891901 - 6954.953068467188) / 6237.271068891901 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	92.83 μs	112.16 μs	1.21	0.03	False
SerializeToWriter - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	84.34 μs	101.66 μs	1.21	0.05	False
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	96.41 μs	110.39 μs	1.14	0.04	False
SerializeToUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	90.89 μs	107.40 μs	1.18	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 112.16001139088729 > 97.71059737893884.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -31.13759349591331 (T) = (0 -110921.3407012225) / Math.Sqrt((2580608.4283008464 / (50)) + (922332.1040757544 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.17525805034606698 = (94380.41344925105 - 110921.3407012225) / 94380.41344925105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToWriter(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 101.66271709057071 > 88.49440290944675.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.825556969155823 (T) = (0 -100576.6244693736) / Math.Sqrt((5188846.566888608 / (52)) + (936603.7460726317 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.19029810511174663 = (84497.00460535585 - 100576.6244693736) / 84497.00460535585 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeObjectProperty(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.38803926566192 > 100.0133249258321.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -38.49272219860525 (T) = (0 -110852.38812229387) / Math.Sqrt((2430504.3554841243 / (51)) + (347760.67747655464 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.16338771346874703 = (95284.13171201311 - 110852.38812229387) / 95284.13171201311 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 107.40403263546798 > 94.4524114104118.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -39.686034612319 (T) = (0 -106439.24300170982) / Math.Sqrt((2603652.4924170077 / (48)) + (513968.35288296436 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.18960479926307613 = (89474.45661588259 - 106439.24300170982) / 89474.45661588259 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
XmlSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	7.86 μs	9.52 μs	1.21	0.15	False
DataContractSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.33 μs	3.70 μs	1.11	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.XmlSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.524131639752184 > 8.372461632325914.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -10.26436490104848 (T) = (0 -9437.644741186403) / Math.Sqrt((47449.31472693986 / (51)) + (71816.07620271172 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17572130289433102 = (8027.110436761916 - 9437.644741186403) / 8027.110436761916 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.DataContractSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.701912081337188 > 3.501207233619583.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 2:54:53 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -5.923982813933151 (T) = (0 -3568.0526778245517) / Math.Sqrt((5261.411560720138 / (40)) + (16708.701127673692 / (13))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (13) - 2, .025) and -0.06666009337286663 = (3345.0699993304115 - 3568.0526778245517) / 3345.0699993304115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Diagnostics.Perf_Activity

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
EnumerateActivityLinkTagsSmall - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	652.91 ns	910.61 ns	1.39	0.19	True
EnumerateActivityLinkTagsLarge - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	142.36 μs	193.27 μs	1.36	0.18	True
EnumerateActivityEventsLarge - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	137.89 μs	205.99 μs	1.49	0.10	False
EnumerateActivityLinksSmall - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	787.08 ns	1.10 μs	1.39	0.12	False
EnumerateActivityTagObjectsSmall - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	639.91 ns	910.10 ns	1.42	0.16	True
EnumerateActivityTagObjectsLarge - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	139.64 μs	200.32 μs	1.43	0.12	True
EnumerateActivityLinksLarge - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	147.36 μs	213.67 μs	1.45	0.18	True
EnumerateActivityEventsSmall - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	777.63 ns	1.13 μs	1.45	0.16	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'

Payloads

Baseline
Compare

System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsSmall

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 910.6079902350924 > 679.8648226200595.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -32.4585995410601 (T) = (0 -907.7272552445993) / Math.Sqrt((321.6235186417771 / (51)) + (226.88123850987787 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3964162118761967 = (650.0406164899756 - 907.7272552445993) / 650.0406164899756 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsLarge

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 193.26900763888892 > 144.6944850566197.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.124014353526437 (T) = (0 -198878.5957297591) / Math.Sqrt((15080335.48481601 / (50)) + (41570733.6702004 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.42453772527729317 = (139609.21652043046 - 198878.5957297591) / 139609.21652043046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Diagnostics.Perf_Activity.EnumerateActivityEventsLarge

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 205.9937734375 > 144.78733010597827.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -63.67711374510129 (T) = (0 -205091.85867384102) / Math.Sqrt((15602633.291357584 / (51)) + (2211781.9837025376 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.46437672090465365 = (140054.02827432318 - 205091.85867384102) / 140054.02827432318 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Diagnostics.Perf_Activity.EnumerateActivityLinksSmall

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0973480264249318 > 841.1736183916825.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.888284537082123 (T) = (0 -1137.822990220219) / Math.Sqrt((640.8125825429194 / (50)) + (1805.9280999984203 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.43399645642572693 = (793.462902311678 - 1137.822990220219) / 793.462902311678 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsSmall

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 910.1027216318801 > 678.886861070485.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -28.55878680743987 (T) = (0 -908.0111764257622) / Math.Sqrt((327.5899387261387 / (49)) + (302.40973117847324 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.3992076190315625 = (648.9467067469419 - 908.0111764257622) / 648.9467067469419 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsLarge

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 200.32174155982906 > 147.47807492617545.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -87.99599321470578 (T) = (0 -201556.12952450576) / Math.Sqrt((10242305.157031475 / (51)) + (1202154.7773531806 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.44745647410214573 = (139248.4908049001 - 201556.12952450576) / 139248.4908049001 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Diagnostics.Perf_Activity.EnumerateActivityLinksLarge

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 213.66560677592955 > 147.26657388294245.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -24.18945631294752 (T) = (0 -207766.2716340576) / Math.Sqrt((11545420.861988332 / (51)) + (29801848.180868775 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4761970796144479 = (140744.2640980714 - 207766.2716340576) / 140744.2640980714 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Diagnostics.Perf_Activity.EnumerateActivityEventsSmall

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1263267439363838 > 815.4613498645917.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -28.294530984306387 (T) = (0 -1100.7199742629873) / Math.Sqrt((373.38931894258866 / (48)) + (462.4257115016792 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.3996462912379219 = (786.427243192601 - 1100.7199742629873) / 786.427243192601 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Scoped - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	559.59 ns	741.37 ns	1.32	0.13	False
Transient - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	24.28 μs	35.01 μs	1.44	0.09	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable*'

Payloads

Baseline
Compare

Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Scoped

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 741.3679180043454 > 604.4178824203908.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.436054196360484 (T) = (0 -712.098224766534) / Math.Sqrt((249.48154276768398 / (50)) + (418.9911381074598 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.246347637890632 = (571.347995629628 - 712.098224766534) / 571.347995629628 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Transient

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.01441692849189 > 26.279352921803387.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -52.078305930435846 (T) = (0 -35427.111365804754) / Math.Sqrt((304825.9544440407 / (49)) + (134240.08453555146 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4148203153179752 = (25040.007541765226 - 35427.111365804754) / 25040.007541765226 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DataContractSerializer_BinaryXml_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.21 ms	2.85 ms	1.29	0.11	False
XmlSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.25 ms	1.48 ms	1.18	0.16	False
DataContractSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.00 ms	3.62 ms	1.21	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.DataContractSerializer_BinaryXml_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.8492890340531556 > 2.3106089214161036.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -25.458059984768315 (T) = (0 -2874445.217989352) / Math.Sqrt((4113264735.3243713 / (51)) + (2155020723.910799 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2827490684390628 = (2240847.636309083 - 2874445.217989352) / 2240847.636309083 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.XmlSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4792560872564935 > 1.3336955246394229.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.14105103313694 (T) = (0 -1494342.6379563147) / Math.Sqrt((1255868961.6801226 / (50)) + (131254488.43350573 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.1604151497699361 = (1287765.5365431786 - 1494342.6379563147) / 1287765.5365431786 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.DataContractSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6155331242236026 > 3.145954917092743.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -17.856769530102778 (T) = (0 -3689769.897285629) / Math.Sqrt((6950600530.656714 / (49)) + (4864920139.1497345 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2170589789096713 = (3031710.016708631 - 3689769.897285629) / 3031710.016708631 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	7.73 μs	10.14 μs	1.31	0.09	True
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	7.66 μs	9.24 μs	1.21	0.11	False
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	8.13 μs	10.06 μs	1.24	0.10	False
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	9.16 μs	11.08 μs	1.21	0.04	True
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	7.71 μs	9.96 μs	1.29	0.12	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.136859519381817 > 8.118086745689213.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -26.611389471807687 (T) = (0 -10030.265586043675) / Math.Sqrt((42261.529143106716 / (49)) + (20286.565497146876 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2568776663045377 = (7980.303775732277 - 10030.265586043675) / 7980.303775732277 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromUtf8Bytes(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.242707661118677 > 7.952731215010693.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.034186076089659 (T) = (0 -9475.649860317564) / Math.Sqrt((27714.612281676396 / (47)) + (64471.36289238339 / (4))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (4) - 2, .025) and -0.2367675740091285 = (7661.625401126198 - 9475.649860317564) / 7661.625401126198 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.063711311154842 > 8.54493717475192.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.721283139419691 (T) = (0 -9883.176908011908) / Math.Sqrt((38132.33673629173 / (50)) + (49434.615462057096 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.20572427680803726 = (8196.87974946979 - 9883.176908011908) / 8196.87974946979 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromReader(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.078164237841044 > 9.536948692717884.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.13572283793201 (T) = (0 -11175.15457248608) / Math.Sqrt((34678.70683160066 / (50)) + (36978.007393337 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.24605241865065627 = (8968.446596000831 - 11175.15457248608) / 8968.446596000831 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.962163229740765 > 7.746835081562373.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.602687247225779 (T) = (0 -9458.59513503526) / Math.Sqrt((35155.2300012208 / (51)) + (118281.35320142866 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.27126998229795835 = (7440.272535923348 - 9458.59513503526) / 7440.272535923348 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<BinaryData>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.30 μs	3.61 μs	1.10	0.01	False
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.61 μs	4.93 μs	1.07	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<BinaryData>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.612802603232715 > 3.4794996215805103.
IsChangePoint: Marked as a change because one of 9/15/2023 9:01:53 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -15.133288739633707 (T) = (0 -3589.721588688468) / Math.Sqrt((2909.631350833954 / (51)) + (958.8291359834612 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.07831015969161348 = (3329.0251013818643 - 3589.721588688468) / 3329.0251013818643 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromReader(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.92863937024514 > 4.855924514575633.
IsChangePoint: Marked as a change because one of 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.472580101044249 (T) = (0 -4829.194027149335) / Math.Sqrt((2334.9496989334953 / (39)) + (13185.839006063461 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.05149629805536213 = (4592.687616761418 - 4829.194027149335) / 4592.687616761418 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.10 μs	3.73 μs	1.20	0.16	False
SystemTextJson_Reflection_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.73 μs	1.93 μs	1.12	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7323516390342166 > 3.338203501307552.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -23.939743798307223 (T) = (0 -3710.0290310293003) / Math.Sqrt((12173.343555695912 / (51)) + (782.9885736938168 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.15539525340001847 = (3211.0474922860203 - 3710.0290310293003) / 3211.0474922860203 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>.SystemTextJson_Reflection_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9340360617337105 > 1.7100080295649096.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -38.17008352813366 (T) = (0 -1951.5449288019527) / Math.Sqrt((1782.4019270592116 / (50)) + (182.36451482474072 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.21401926377130706 = (1607.5073823289663 - 1951.5449288019527) / 1607.5073823289663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	282.29 μs	322.08 μs	1.14	0.05	False
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	278.86 μs	329.52 μs	1.18	0.04	False
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	272.63 μs	335.45 μs	1.23	0.03	False
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	289.62 μs	340.78 μs	1.18	0.04	False
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	285.02 μs	340.32 μs	1.19	0.01	False
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	275.04 μs	314.75 μs	1.14	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 322.08375892857146 > 294.23512549107147.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -15.823207994857652 (T) = (0 -326928.2168357569) / Math.Sqrt((60319473.12261521 / (50)) + (31798007.43465819 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.17157894997384682 = (279049.2410631438 - 326928.2168357569) / 279049.2410631438 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 329.52027759308504 > 290.8511590887525.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.146731423702969 (T) = (0 -329592.16596485145) / Math.Sqrt((44371471.16557682 / (51)) + (45459962.72541628 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17665728397975763 = (280108.8902030045 - 329592.16596485145) / 280108.8902030045 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 335.4461881205674 > 289.0280026510511.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.012423837892294 (T) = (0 -327130.78522237187) / Math.Sqrt((59182952.52620558 / (49)) + (42459078.35853457 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.17271989416308298 = (278950.48668533954 - 327130.78522237187) / 278950.48668533954 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 340.77998835403736 > 303.40029436921293.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -26.299704651241502 (T) = (0 -341346.5862686391) / Math.Sqrt((49335696.799134314 / (52)) + (10152865.742032247 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.16805132986996849 = (292235.9467769614 - 341346.5862686391) / 292235.9467769614 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: SourceGen)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 340.3178688888889 > 304.4043580034723.
IsChangePoint: Marked as a change because one of 9/26/2023 10:37:55 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -38.70081981533558 (T) = (0 -340490.02209342655) / Math.Sqrt((39040083.30638142 / (50)) + (3828426.629560999 / (5))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (5) - 2, .025) and -0.16461583171463637 = (292362.52231959713 - 340490.02209342655) / 292362.52231959713 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 314.75118467565596 > 290.28716847771705.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.81129294507482 (T) = (0 -323957.6482849186) / Math.Sqrt((46951640.23398351 / (48)) + (37702782.25499902 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.1594375451302722 = (279409.31328777986 - 323957.6482849186) / 279409.31328777986 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Reflection.Activator<EmptyStruct>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
CreateInstanceGeneric - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	277.53 ns	350.58 ns	1.26	0.12	True
CreateInstanceType - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	351.24 ns	434.39 ns	1.24	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Activator<EmptyStruct>*'

Payloads

Baseline
Compare

System.Reflection.Activator<EmptyStruct>.CreateInstanceGeneric

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 350.57538511034824 > 287.965885866338.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.160300007367674 (T) = (0 -353.54964811010575) / Math.Sqrt((48.42915931572142 / (51)) + (25.721615754815403 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.26373533870241694 = (279.7655785056425 - 353.54964811010575) / 279.7655785056425 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Reflection.Activator<EmptyStruct>.CreateInstanceType

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 434.38870313864334 > 371.7103634101547.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.54012763679372 (T) = (0 -432.0587791315115) / Math.Sqrt((79.25680954220935 / (50)) + (35.80389040021502 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.20385865775807463 = (358.8949386600975 - 432.0587791315115) / 358.8949386600975 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SystemTextJson_Reflection_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	96.32 μs	112.12 μs	1.16	0.02	False
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	156.43 μs	188.49 μs	1.20	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.SystemTextJson_Reflection_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 112.12409569506728 > 99.69098308833881.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -32.43296311868252 (T) = (0 -111589.02407283173) / Math.Sqrt((2605324.1023553633 / (48)) + (851911.898877866 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.17682343920318166 = (94822.23106330022 - 111589.02407283173) / 94822.23106330022 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 188.4875869448903 > 165.19627077171077.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -25.663875529344438 (T) = (0 -186698.6470838832) / Math.Sqrt((21571165.33650393 / (50)) + (2724335.7826773855 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.16957442717086363 = (159629.55648363224 - 186698.6470838832) / 159629.55648363224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
XmlSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.99 μs	4.96 μs	1.24	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>.XmlSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.955697201038025 > 4.348294075156738.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.954078901312158 (T) = (0 -4987.977678153509) / Math.Sqrt((13051.096827446157 / (51)) + (6106.928235363593 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.19109845224042016 = (4187.7123329068845 - 4987.977678153509) / 4187.7123329068845 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Devirtualization.EqualityComparer

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ValueTupleCompareNoOpt - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	346.07 ns	415.54 ns	1.20	0.14	False
ValueTupleCompareWrapped - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	362.65 ns	451.36 ns	1.24	0.16	False
ValueTupleCompare - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	329.78 ns	425.82 ns	1.29	0.13	True
ValueTupleCompareCached - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	326.20 ns	417.09 ns	1.28	0.06	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Devirtualization.EqualityComparer*'

Payloads

Baseline
Compare

Devirtualization.EqualityComparer.ValueTupleCompareNoOpt

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 415.54376076077494 > 357.9881410143185.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -30.16585283148647 (T) = (0 -421.784608628558) / Math.Sqrt((96.84220107073219 / (51)) + (21.839080658861445 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.24070980429132224 = (339.95428033993494 - 421.784608628558) / 339.95428033993494 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Devirtualization.EqualityComparer.ValueTupleCompareWrapped

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 451.36018724011717 > 378.07403913122323.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -30.226122663144153 (T) = (0 -446.7867121279275) / Math.Sqrt((115.53542213971389 / (50)) + (15.72822894490603 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.2283991089274323 = (363.71461757086104 - 446.7867121279275) / 363.71461757086104 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Devirtualization.EqualityComparer.ValueTupleCompare

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 425.8212968293487 > 352.5853127863135.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -38.648110875965905 (T) = (0 -424.1975786369058) / Math.Sqrt((98.14586785718605 / (49)) + (10.948601347534007 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.24744344250427014 = (340.053556083729 - 424.1975786369058) / 340.053556083729 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Devirtualization.EqualityComparer.ValueTupleCompareCached

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 417.09095840493785 > 347.3086270870023.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -62.43798767002045 (T) = (0 -417.5862681037397) / Math.Sqrt((58.58603231554622 / (51)) + (2.334040131898836 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.24500840263787077 = (335.4083934043945 - 417.5862681037397) / 335.4083934043945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DataContractSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	224.68 μs	265.07 μs	1.18	0.10	False
DataContractSerializer_BinaryXml_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	159.48 μs	191.88 μs	1.20	0.07	False
XmlSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	123.37 μs	154.10 μs	1.25	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.DataContractSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 265.0658883227176 > 234.7563087424584.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -37.317170069553256 (T) = (0 -265758.6867945879) / Math.Sqrt((35334900.93879297 / (51)) + (1494242.0006914067 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.16959687482825608 = (227222.4665730335 - 265758.6867945879) / 227222.4665730335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.DataContractSerializer_BinaryXml_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 191.87980725520046 > 166.57387253222763.
IsChangePoint: Marked as a change because one of 8/24/2023 10:59:30 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -28.009814311577752 (T) = (0 -195025.1571253953) / Math.Sqrt((19428071.482499324 / (50)) + (5015052.98648303 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.22557374579583253 = (159129.67929869835 - 195025.1571253953) / 159129.67929869835 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.XmlSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 154.1037939285715 > 130.21266984137097.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -7.906186696837685 (T) = (0 -148486.68069182392) / Math.Sqrt((7872768.069165785 / (51)) + (25960735.340025023 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.18767915195710297 = (125022.5538161059 - 148486.68069182392) / 125022.5538161059 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Threading.Channels.Tests.UnboundedChannelPerfTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TryWriteThenTryRead - Duration of single invocation 📈 - ADX Test Multi Config Graph	307.52 ns	459.10 ns	1.49	0.27	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.UnboundedChannelPerfTests*'

Payloads

Baseline
Compare

System.Threading.Channels.Tests.UnboundedChannelPerfTests.TryWriteThenTryRead

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 459.104474210873 > 325.19922474254275.
IsChangePoint: Marked as a change because one of 9/21/2023 9:38:47 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.936363280455781 (T) = (0 -467.6578847003093) / Math.Sqrt((511.77685270247747 / (50)) + (302.49943914195075 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.4203463934535756 = (329.2562200712166 - 467.6578847003093) / 329.2562200712166 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.00 ms	2.71 ms	1.35	0.11	True
SystemTextJson_Reflection_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	641.83 μs	762.35 μs	1.19	0.12	False
SystemTextJson_SourceGen_ - Duration of single invocation 📈 - ADX Test Multi Config Graph	484.69 μs	602.58 μs	1.24	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.7112761904761897 > 2.0429815915655283.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -34.84728707977958 (T) = (0 -2663338.386945828) / Math.Sqrt((1709004124.5761523 / (51)) + (1458037578.3199513 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.35323959214886674 = (1968120.355329391 - 2663338.386945828) / 1968120.355329391 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_Reflection_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 762.3481524347213 > 646.7839377066045.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -24.504524746268636 (T) = (0 -777621.5257143187) / Math.Sqrt((282386952.55952555 / (51)) + (147764755.86264217 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2584653008079083 = (617912.5679628211 - 777621.5257143187) / 617912.5679628211 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 602.5820755494506 > 493.064040426972.
IsChangePoint: Marked as a change because one of 8/11/2023 7:16:47 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -40.37575028660534 (T) = (0 -601392.3031294516) / Math.Sqrt((163712431.31194684 / (48)) + (25079515.373683054 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.264043035939705 = (475768.8512419747 - 601392.3031294516) / 475768.8512419747 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DeserializeFromReader - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	120.94 μs	155.23 μs	1.28	0.10	False
DeserializeFromString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	105.46 μs	138.70 μs	1.32	0.07	True
DeserializeFromUtf8Bytes - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	106.32 μs	136.01 μs	1.28	0.11	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromReader(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 155.22808017857145 > 128.68300274169923.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -33.78535347768506 (T) = (0 -153362.52680932276) / Math.Sqrt((7236672.326874238 / (49)) + (2652691.8159856778 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2474627075045832 = (122939.56836281561 - 153362.52680932276) / 122939.56836281561 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromString(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 138.69506173584298 > 111.50091870898576.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -20.02942754061871 (T) = (0 -136570.2862443796) / Math.Sqrt((7632563.12754147 / (51)) + (7532280.309398581 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2643807622212913 = (108013.57496491008 - 136570.2862443796) / 108013.57496491008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromUtf8Bytes(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 136.01224912207357 > 111.04249318380799.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.636930760360695 (T) = (0 -136553.18799083313) / Math.Sqrt((5792013.427429815 / (50)) + (15928575.799555605 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.27711767852760366 = (106922.94867319203 - 136553.18799083313) / 106922.94867319203 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.25 ms	2.84 ms	1.26	0.07	False
SystemTextJson_Reflection_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.59 ms	1.84 ms	1.16	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.842760798172757 > 2.457038108738994.
IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -15.762105407880956 (T) = (0 -2799099.640924672) / Math.Sqrt((4696260405.748839 / (49)) + (3199975109.6364326 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2027076594821721 = (2327331.682688235 - 2799099.640924672) / 2327331.682688235 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.SystemTextJson_Reflection_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8421544962962964 > 1.6215417276306239.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -18.32417667707505 (T) = (0 -1877161.4657530622) / Math.Sqrt((663737136.3735952 / (52)) + (1137753138.9455152 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.2023369696157834 = (1561260.7057678052 - 1877161.4657530622) / 1561260.7057678052 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.GetService

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Scoped - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	571.85 ns	708.47 ns	1.24	0.11	False
Transient - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.45 μs	3.43 μs	1.40	0.04	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'

Payloads

Baseline
Compare

Microsoft.Extensions.DependencyInjection.GetService.Scoped

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 708.4678965205258 > 599.5148870146672.
IsChangePoint: Marked as a change because one of 8/14/2023 10:44:26 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.55805643174902 (T) = (0 -716.7337896521078) / Math.Sqrt((221.57267085423436 / (50)) + (135.68948350969754 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.24210686124477912 = (577.0306984165856 - 716.7337896521078) / 577.0306984165856 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.GetService.Transient

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4317351365700057 > 2.540668420259039.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -60.94661624027439 (T) = (0 -3465.0072817273076) / Math.Sqrt((3630.6704472601728 / (49)) + (840.4007619025352 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.421504600886219 = (2437.5631845068197 - 3465.0072817273076) / 2437.5631845068197 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TryWriteThenTryRead - Duration of single invocation 📈 - ADX Test Multi Config Graph	401.42 ns	544.06 ns	1.36	0.19	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests*'

Payloads

Baseline
Compare

System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.TryWriteThenTryRead

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 544.0624314143822 > 418.10380759038094.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -27.90635303268526 (T) = (0 -546.9168605974861) / Math.Sqrt((138.4779106587325 / (49)) + (107.05763839037947 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.384199179902735 = (395.114278738351 - 546.9168605974861) / 395.114278738351 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DataContractJsonSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	250.56 μs	309.95 μs	1.24	0.07	False
SystemTextJson_Reflection_ - Duration of single invocation 📈 - ADX Test Multi Config Graph	83.89 μs	102.78 μs	1.23	0.03	False
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	148.35 μs	183.28 μs	1.24	0.06	False
SystemTextJson_SourceGen_ - Duration of single invocation 📈 - ADX Test Multi Config Graph	77.40 μs	91.29 μs	1.18	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.DataContractJsonSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 309.9530246975385 > 263.81624589563745.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -12.520792328381916 (T) = (0 -311871.8330786524) / Math.Sqrt((58267001.582678735 / (51)) + (77516442.82695965 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.22229976693171152 = (255151.67515864896 - 311871.8330786524) / 255151.67515864896 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.SystemTextJson_Reflection_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 102.78240398041385 > 90.11836385619178.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -46.93249226864946 (T) = (0 -102317.1184482423) / Math.Sqrt((2659813.898420149 / (49)) + (285428.4803405726 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.19415485970439156 = (85681.6162633802 - 102317.1184482423) / 85681.6162633802 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 183.2798733430799 > 154.7195874849443.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -12.503552120560467 (T) = (0 -177532.1548002376) / Math.Sqrt((11241407.913137902 / (48)) + (22454992.624787193 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.20527960336673567 = (147295.41120942633 - 177532.1548002376) / 147295.41120942633 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.SystemTextJson_SourceGen_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 91.29281691176469 > 83.31808089274702.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -16.329866199132095 (T) = (0 -93616.15053039262) / Math.Sqrt((3046838.0817533447 / (51)) + (2720622.1125726863 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17653100298199326 = (79569.64184804012 - 93616.15053039262) / 79569.64184804012 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Tests.Perf_SortedSet

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
EnumerateViewBetween - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	35.77 μs	45.97 μs	1.29	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_SortedSet*'

Payloads

Baseline
Compare

System.Collections.Tests.Perf_SortedSet.EnumerateViewBetween

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.96517613636364 > 37.721931704653535.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -29.968901388678546 (T) = (0 -46247.46044166744) / Math.Sqrt((945617.675594004 / (51)) + (345790.26121319603 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.26577298787443165 = (36536.931096411834 - 46247.46044166744) / 36536.931096411834 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<Location>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SystemTextJson_Reflection_ - Duration of single invocation 📈 - ADX Test Multi Config Graph	3.51 μs	4.16 μs	1.18	0.05	False
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.74 μs	6.78 μs	1.18	0.10	False
DataContractJsonSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	7.43 μs	8.68 μs	1.17	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream<Location>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_ToStream<Location>.SystemTextJson_Reflection_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.157648402650223 > 3.6880431543166994.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -40.524512430223695 (T) = (0 -4125.985682769535) / Math.Sqrt((5922.507941453579 / (51)) + (693.3523386831133 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.20063108034342325 = (3436.5141385390057 - 4125.985682769535) / 3436.5141385390057 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<Location>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.782444695224697 > 6.025702071690361.
IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -10.734380832615942 (T) = (0 -6970.074321826536) / Math.Sqrt((42518.31460675569 / (50)) + (43774.706704705015 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.200843263161144 = (5804.316462981402 - 6970.074321826536) / 5804.316462981402 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<Location>.DataContractJsonSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.683840315849876 > 8.003169111921734.
IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -40.64972944171845 (T) = (0 -8676.804041927862) / Math.Sqrt((33393.23870006933 / (50)) + (138.33399939341948 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.14176975144885626 = (7599.434151165219 - 8676.804041927862) / 7599.434151165219 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
XmlSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.52 ms	3.08 ms	1.23	0.13	False
DataContractSerializer_BinaryXml_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.05 ms	4.20 ms	1.38	0.09	True
DataContractSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	11.51 ms	12.87 ms	1.12	0.01	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.XmlSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.0832303562499996 > 2.6304744281875787.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -13.178846014562854 (T) = (0 -3121713.168816773) / Math.Sqrt((3692367507.712571 / (50)) + (8673539406.863163 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2498523157642922 = (2497665.627724846 - 3121713.168816773) / 2497665.627724846 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.DataContractSerializer_BinaryXml_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.203136903520209 > 3.192600598177897.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -70.94493700723106 (T) = (0 -4183344.9753759606) / Math.Sqrt((6767203236.7554655 / (50)) + (482242060.5044317 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.37229151121412396 = (3048437.5522186095 - 4183344.9753759606) / 3048437.5522186095 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.DataContractSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.865933347368419 > 12.044023451403847.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -21.52886425731566 (T) = (0 -12869167.881302213) / Math.Sqrt((18535534270.083176 / (49)) + (15073789358.862349 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.12073280133584828 = (11482815.409670275 - 12869167.881302213) / 11482815.409670275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
JsonNet_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.61 μs	3.31 μs	1.27	0.13	False
DataContractJsonSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.38 μs	4.09 μs	1.21	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.JsonNet_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3077785406680364 > 2.7975783835741743.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -22.15576205956138 (T) = (0 -3242.135774842575) / Math.Sqrt((6228.179323965448 / (51)) + (2138.573201964515 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2123121064277627 = (2674.340838182302 - 3242.135774842575) / 2674.340838182302 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.DataContractJsonSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.0889537673974 > 3.6579671460748893.
IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -23.792510518704585 (T) = (0 -4051.6600992164203) / Math.Sqrt((11368.815949321788 / (50)) + (1087.7531163454532 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.15103713234145552 = (3520.008160791892 - 4051.6600992164203) / 3520.008160791892 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DataContractSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	9.34 ms	10.81 ms	1.16	0.03	False
DataContractSerializer_BinaryXml_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.02 ms	5.50 ms	1.37	0.09	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>.DataContractSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.809025974025971 > 9.815579503.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -17.21053679561432 (T) = (0 -10867405.397573855) / Math.Sqrt((29828913625.98409 / (48)) + (23955459165.172623 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.14778864719143076 = (9468124.13954933 - 10867405.397573855) / 9468124.13954933 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>.DataContractSerializer_BinaryXml_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.50200740888889 > 4.2339673261408075.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -43.815272115038304 (T) = (0 -5443462.882295767) / Math.Sqrt((15667212738.494448 / (49)) + (2801290240.372464 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.34603475380016424 = (4044073.057495451 - 5443462.882295767) / 4044073.057495451 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<ClassImplementingIXmlSerialiable>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
XmlSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	8.64 μs	10.49 μs	1.21	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream<ClassImplementingIXmlSerialiable>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Xml_FromStream<ClassImplementingIXmlSerialiable>.XmlSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.49015046108028 > 9.382416725127841.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -11.924374730859702 (T) = (0 -10397.856560845174) / Math.Sqrt((98040.56014849625 / (49)) + (41497.84999309578 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.1462318453786398 = (9071.338056752737 - 10397.856560845174) / 9071.338056752737 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
DataContractSerializer_BinaryXml_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	323.00 μs	419.10 μs	1.30	0.10	True
XmlSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	299.70 μs	346.11 μs	1.15	0.05	False
DataContractSerializer_ - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	786.63 μs	893.06 μs	1.14	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>*'

Payloads

Baseline
Compare

MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>.DataContractSerializer_BinaryXml_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 419.1044698544699 > 341.60025691220847.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -32.24608768333953 (T) = (0 -411486.7365177518) / Math.Sqrt((40650242.09980125 / (50)) + (26885399.433013912 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2740507124704955 = (322975.1629900533 - 411486.7365177518) / 322975.1629900533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>.XmlSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 346.1058762820512 > 312.3305966574677.
IsChangePoint: Marked as a change because one of 8/22/2023 3:00:25 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -28.460711687048494 (T) = (0 -348799.88448259636) / Math.Sqrt((45623885.31413718 / (51)) + (7477968.30140682 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.15694909930780385 = (301482.4806824098 - 348799.88448259636) / 301482.4806824098 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>.DataContractSerializer_

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 893.0583712237582 > 827.1792987710337.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -14.351661746462263 (T) = (0 -888319.0138999769) / Math.Sqrt((110304630.55872983 / (49)) + (186939060.6304711 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.1274917432842993 = (787871.8573249768 - 888319.0138999769) / 787871.8573249768 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CtorFromCollectionNonGeneric<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Hashtable - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	185.93 μs	257.74 μs	1.39	0.16	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollectionNonGeneric<Int32>*'

Payloads

Baseline
Compare

System.Collections.CtorFromCollectionNonGeneric<Int32>.Hashtable(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 257.7379099043716 > 201.37288797882502.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -26.835683239488016 (T) = (0 -261658.92742559095) / Math.Sqrt((42384677.25215534 / (49)) + (25569554.8491088 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.3817712837720071 = (189364.8612462877 - 261658.92742559095) / 189364.8612462877 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Interop.StructureToPtr

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
MarshalPtrToStructure - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	852.55 ns	1.09 μs	1.28	0.18	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Interop.StructureToPtr*'

Payloads

Baseline
Compare

Interop.StructureToPtr.MarshalPtrToStructure

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0882190361964785 > 885.3745221326658.
IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -29.922449729265242 (T) = (0 -1077.7955573495597) / Math.Sqrt((687.6206531503506 / (50)) + (165.10561583352214 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.25936206284044755 = (855.8266039224884 - 1077.7955573495597) / 855.8266039224884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare	1be2db9f75035a01e1955a33b35743018b6d85a6
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SerializeObjectProperty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.68 μs	1.99 μs	1.18	0.20	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>*'

Payloads

Baseline
Compare

System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeObjectProperty(Mode: Reflection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9864880971215815 > 1.7657768884418295.
IsChangePoint: Marked as a change because one of 8/22/2023 3:00:25 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM.
IsRegressionStdDev: Marked as regression because -17.4731321317645 (T) = (0 -1981.1231869095236) / Math.Sqrt((2993.8071867506237 / (49)) + (854.0408550745024 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.17117388391698674 = (1691.5704953082325 - 1981.1231869095236) / 1691.5704953082325 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[lewing]

range is dotnet/runtime@aec0684...07e86f1

@SamMonoRT this one looks like it is worth figuring out

[lewing]

@radical were there v8 version changes?

[radical]

@radical were there v8 version changes?

No, that hasn't changed for few weeks.

[radical]

@radical were there v8 version changes?

No, that hasn't changed for few weeks.

err, that's true only for library tests! For perf pipeline, since no version change shows up in the diff, there was no version change for v8 used in perf runs.

[matouskozak]

I didn't see these kinds of regressions for Mono AOT-llvm Perf line. I will update this comment if something new shows up in this week Perf issues.

[SamMonoRT]

@kotlarmilos - any related regression on the interpreter lane?

[kotlarmilos]

No related regressions on the interpreter lane. There is #22539, but it has different range.

[radekdoulik]

@radical @lewing could this be dotnet/runtime#91480? range is dotnet/runtime@aec0684...07e86f1 maybe we are not using the "current" runtime for some reason?

or it can be the v8` version change as we suspected earlier

[radical]

@radical @lewing could this be dotnet/runtime#91480? range is dotnet/runtime@aec0684...07e86f1 maybe we are not using the "current" runtime for some reason?

That is possible. The net8/9 build is messed up on main right now.

[radical]

I compared the two builds.

• v8 version was the same - 11.7.439
• The older build used everything (tasks, targets, runtime pack, aot compiler, emsdk etc) from 9.0
• The newer regressed build used 9.0 runtime pack, and emsdk, but 8.0 for the tasks, targets, and aot compiler.

@lewing is working on a patch that would make this coherent.

[radekdoulik]

Did dotnet/runtime#93031 fixed it?

[radical]

Did dotnet/runtime#93031 fixed it?

Yeah, it should have fixed using the correct workload.

[lewing]

It is surprising that net9 already has that much of win over net8

[SamMonoRT]

It is surprising that net9 already has that much of win over net8

Any particular benchmarks? I know @matouskozak had a PR affecting many of them, but don't know of any other improvements

[matouskozak]

It is surprising that net9 already has that much of win over net8

Any particular benchmarks? I know @matouskozak had a PR affecting many of them, but don't know of any other improvements

It was this PR dotnet/runtime@169e22c, affecting scenarios where mini is used (llvm should be unaffected).

[lewing]

these are wasm benchmarks so it is all llvm, the regression and subsequent improvment is the difference between using the net8 wasm runtime and the net9 wasm runtime on the perf lane which is still targeting net8 which in theory but the two changes alter in practice