[Perf] Linux/arm64: 10 Regressions on 12/12/2022 1:43:17 AM

[performanceautofiler]

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	9b5b9775d4b53cee164d5df693ca236cb26492b4
Compare	4aae104908d98a46014b5d569d77d67b001c9d19
Diff	Diff

Regressions in System.Collections.CtorDefaultSizeNonGeneric

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Hashtable - Duration of single invocation	139.21 ns	155.49 ns	1.12	0.16	False
Queue - Duration of single invocation	132.49 ns	199.00 ns	1.50	0.11	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSizeNonGeneric*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSizeNonGeneric.Hashtable

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -11.93897479178275 (T) = (0 -153.27159245352206) / Math.Sqrt((0.7723100648184258 / (3)) + (9.312528874713824 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.1088353693275044 = (138.22754639084022 - 153.27159245352206) / 138.22754639084022 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSizeNonGeneric.Queue

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -44.88823076099701 (T) = (0 -194.1397926943852) / Math.Sqrt((2.0865150361980462 / (3)) + (8.779617144646712 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.4767858503187267 = (131.46103252037867 - 194.1397926943852) / 131.46103252037867 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	9b5b9775d4b53cee164d5df693ca236cb26492b4
Compare	4aae104908d98a46014b5d569d77d67b001c9d19
Diff	Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BitArrayLengthCtor - Duration of single invocation	96.06 ns	107.92 ns	1.12	0.11	False
BitArrayLengthValueCtor - Duration of single invocation	190.64 ns	211.88 ns	1.11	0.14	False
BitArrayLengthCtor - Duration of single invocation	104.02 ns	115.43 ns	1.11	0.02	False
BitArraySetLengthShrink - Duration of single invocation	283.97 ns	306.79 ns	1.08	0.09	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayLengthCtor(Size: 4)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -25.028982233283887 (T) = (0 -107.95728991853971) / Math.Sqrt((0.2555795159478787 / (3)) + (0.9509228884852138 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.12233428668263416 = (96.18995980033453 - 107.95728991853971) / 96.18995980033453 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayLengthValueCtor(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -9.406871506287928 (T) = (0 -207.72003161961578) / Math.Sqrt((5.755189556841562 / (3)) + (5.755512769587139 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.08104665525699398 = (192.14714796026553 - 207.72003161961578) / 192.14714796026553 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayLengthCtor(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -17.979160221882417 (T) = (0 -116.12863210073199) / Math.Sqrt((0.13905233127771308 / (3)) + (3.0603518763751327 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.1206471099986613 = (103.62640572987398 - 116.12863210073199) / 103.62640572987398 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArraySetLengthShrink(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -5.6810100252297655 (T) = (0 -304.0905256911271) / Math.Sqrt((16.08508220835678 / (3)) + (18.78982289520275 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.05595744390912837 = (287.9761182092637 - 304.0905256911271) / 287.9761182092637 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	9b5b9775d4b53cee164d5df693ca236cb26492b4
Compare	4aae104908d98a46014b5d569d77d67b001c9d19
Diff	Diff

Regressions in System.Collections.CtorGivenSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Stack - Duration of single invocation	283.66 ns	304.97 ns	1.08	0.13	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorGivenSize<Int32>.Stack(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -7.408477692071517 (T) = (0 -302.93291716867316) / Math.Sqrt((14.436954954224902 / (3)) + (6.57000787170882 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.06230140329016068 = (285.1666355993027 - 302.93291716867316) / 285.1666355993027 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	9b5b9775d4b53cee164d5df693ca236cb26492b4
Compare	4aae104908d98a46014b5d569d77d67b001c9d19
Diff	Diff

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ctor_capacity - Duration of single invocation	149.71 ns	167.60 ns	1.12	0.14	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -7.8196417501602875 (T) = (0 -168.3244699679797) / Math.Sqrt((25.867329562441533 / (3)) + (7.6051829170920415 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.16925148964564354 = (143.95916657672385 - 168.3244699679797) / 143.95916657672385 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	9b5b9775d4b53cee164d5df693ca236cb26492b4
Compare	4aae104908d98a46014b5d569d77d67b001c9d19
Diff	Diff

Regressions in System.Collections.CtorGivenSizeNonGeneric

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Queue - Duration of single invocation	500.02 ns	561.86 ns	1.12	0.11	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSizeNonGeneric*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorGivenSizeNonGeneric.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -9.310220351165572 (T) = (0 -562.9052274270364) / Math.Sqrt((122.66923792232944 / (3)) + (43.154719747324066 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.1279751814567803 = (499.04043695362526 - 562.9052274270364) / 499.04043695362526 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	9b5b9775d4b53cee164d5df693ca236cb26492b4
Compare	4aae104908d98a46014b5d569d77d67b001c9d19
Diff	Diff

Regressions in System.Tests.Perf_Random

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Next_int - Duration of single invocation	115.92 ns	127.00 ns	1.10	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Random.Next_int

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -18.260337243840215 (T) = (0 -126.89284153717858) / Math.Sqrt((0.36901356322964585 / (3)) + (1.423535890086374 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.089573868530759 = (116.46098094137282 - 126.89284153717858) / 116.46098094137282 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[kotlarmilos]

Likely caused by 3689fbe

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @BrzVlad
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	9b5b9775d4b53cee164d5df693ca236cb26492b4
Compare	4aae104908d98a46014b5d569d77d67b001c9d19
Diff	Diff

Regressions in System.Collections.CtorDefaultSizeNonGeneric

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Hashtable - Duration of single invocation	139.21 ns	155.49 ns	1.12	0.16	False
Queue - Duration of single invocation	132.49 ns	199.00 ns	1.50	0.11	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSizeNonGeneric*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSizeNonGeneric.Hashtable

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -11.93897479178275 (T) = (0 -153.27159245352206) / Math.Sqrt((0.7723100648184258 / (3)) + (9.312528874713824 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.1088353693275044 = (138.22754639084022 - 153.27159245352206) / 138.22754639084022 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSizeNonGeneric.Queue

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -44.88823076099701 (T) = (0 -194.1397926943852) / Math.Sqrt((2.0865150361980462 / (3)) + (8.779617144646712 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.4767858503187267 = (131.46103252037867 - 194.1397926943852) / 131.46103252037867 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	9b5b9775d4b53cee164d5df693ca236cb26492b4
Compare	4aae104908d98a46014b5d569d77d67b001c9d19
Diff	Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BitArrayLengthCtor - Duration of single invocation	96.06 ns	107.92 ns	1.12	0.11	False
BitArrayLengthValueCtor - Duration of single invocation	190.64 ns	211.88 ns	1.11	0.14	False
BitArrayLengthCtor - Duration of single invocation	104.02 ns	115.43 ns	1.11	0.02	False
BitArraySetLengthShrink - Duration of single invocation	283.97 ns	306.79 ns	1.08	0.09	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayLengthCtor(Size: 4)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -25.028982233283887 (T) = (0 -107.95728991853971) / Math.Sqrt((0.2555795159478787 / (3)) + (0.9509228884852138 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.12233428668263416 = (96.18995980033453 - 107.95728991853971) / 96.18995980033453 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayLengthValueCtor(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -9.406871506287928 (T) = (0 -207.72003161961578) / Math.Sqrt((5.755189556841562 / (3)) + (5.755512769587139 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.08104665525699398 = (192.14714796026553 - 207.72003161961578) / 192.14714796026553 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayLengthCtor(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -17.979160221882417 (T) = (0 -116.12863210073199) / Math.Sqrt((0.13905233127771308 / (3)) + (3.0603518763751327 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.1206471099986613 = (103.62640572987398 - 116.12863210073199) / 103.62640572987398 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArraySetLengthShrink(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -5.6810100252297655 (T) = (0 -304.0905256911271) / Math.Sqrt((16.08508220835678 / (3)) + (18.78982289520275 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.05595744390912837 = (287.9761182092637 - 304.0905256911271) / 287.9761182092637 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	9b5b9775d4b53cee164d5df693ca236cb26492b4
Compare	4aae104908d98a46014b5d569d77d67b001c9d19
Diff	Diff

Regressions in System.Collections.CtorGivenSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Stack - Duration of single invocation	283.66 ns	304.97 ns	1.08	0.13	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorGivenSize<Int32>.Stack(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -7.408477692071517 (T) = (0 -302.93291716867316) / Math.Sqrt((14.436954954224902 / (3)) + (6.57000787170882 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.06230140329016068 = (285.1666355993027 - 302.93291716867316) / 285.1666355993027 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	9b5b9775d4b53cee164d5df693ca236cb26492b4
Compare	4aae104908d98a46014b5d569d77d67b001c9d19
Diff	Diff

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ctor_capacity - Duration of single invocation	149.71 ns	167.60 ns	1.12	0.14	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -7.8196417501602875 (T) = (0 -168.3244699679797) / Math.Sqrt((25.867329562441533 / (3)) + (7.6051829170920415 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.16925148964564354 = (143.95916657672385 - 168.3244699679797) / 143.95916657672385 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	9b5b9775d4b53cee164d5df693ca236cb26492b4
Compare	4aae104908d98a46014b5d569d77d67b001c9d19
Diff	Diff

Regressions in System.Collections.CtorGivenSizeNonGeneric

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Queue - Duration of single invocation	500.02 ns	561.86 ns	1.12	0.11	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSizeNonGeneric*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorGivenSizeNonGeneric.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -9.310220351165572 (T) = (0 -562.9052274270364) / Math.Sqrt((122.66923792232944 / (3)) + (43.154719747324066 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.1279751814567803 = (499.04043695362526 - 562.9052274270364) / 499.04043695362526 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	9b5b9775d4b53cee164d5df693ca236cb26492b4
Compare	4aae104908d98a46014b5d569d77d67b001c9d19
Diff	Diff

Regressions in System.Tests.Perf_Random

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Next_int - Duration of single invocation	115.92 ns	127.00 ns	1.10	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Random.Next_int

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/19/2022 10:34:07 PM falls between 12/10/2022 11:25:25 PM and 12/19/2022 10:34:07 PM.
IsRegressionStdDev: Marked as regression because -18.260337243840215 (T) = (0 -126.89284153717858) / Math.Sqrt((0.36901356322964585 / (3)) + (1.423535890086374 / (7))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (3) + (7) - 2, .025) and -0.089573868530759 = (116.46098094137282 - 126.89284153717858) / 116.46098094137282 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	kotlarmilos
Labels:	untriaged, area-Codegen-Interpreter-mono, refs/heads/main, Regression, arm64
Milestone:	-

[BrzVlad]

On interpreter I see two problems with the change.

First is that we don't inline the throw helpers since they call other methods and are not marked as aggressive inlining.
Second is that the code is logically changed from if (uncommon) ldstr; throw ex (string) to ldstr; call throw_helper (string), which means that we always execute instructions for loading the string value, even though it is only used in the uncommon case.

@stephentoub I assume CoreCLR generates more or less identical code with your change and does not suffer from these 2 problems ?

[stephentoub]

@stephentoub I assume CoreCLR generates more or less identical code with your change and does not suffer from these 2 problems ?

Correct, e.g. this:

using System.Runtime.CompilerServices;

internal class Program
{
    static void Main()
    {
        while (true)
        {
            Test1(1);
            Test2(1);
        }
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static void Test1(int value)
    {
        if (value < 0)
            throw new ArgumentOutOfRangeException(nameof(value));
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static void Test2(int value)
    {
        ArgumentOutOfRangeException.ThrowIfNegative(value);
    }
}

produces this:

; Assembly listing for method Program:Test1(int)
; Emitting BLENDED_CODE for X64 CPU with AVX - Windows
; Tier-1 compilation
; optimized code
; rsp based frame
; partially interruptible
; No PGO data

G_M000_IG01:                ;; offset=0000H
       56                   push     rsi
       4883EC20             sub      rsp, 32

G_M000_IG02:                ;; offset=0005H
       85C9                 test     ecx, ecx
       7C06                 jl       SHORT G_M000_IG04

G_M000_IG03:                ;; offset=0009H
       4883C420             add      rsp, 32
       5E                   pop      rsi
       C3                   ret

G_M000_IG04:                ;; offset=000FH
       48B978DF9217FC7F0000 mov      rcx, 0x7FFC1792DF78
       E8D204B05F           call     CORINFO_HELP_NEWSFAST
       488BF0               mov      rsi, rax
       B901000000           mov      ecx, 1
       48BAB81D8C17FC7F0000 mov      rdx, 0x7FFC178C1DB8
       E83BD5AF5F           call     CORINFO_HELP_STRCNS
       488BD0               mov      rdx, rax
       488BCE               mov      rcx, rsi
       FF1547E65300         call     [System.ArgumentOutOfRangeException:.ctor(System.String):this]
       488BCE               mov      rcx, rsi
       E8A731A05F           call     CORINFO_HELP_THROW
       CC                   int3

; Total bytes of code 74

; Assembly listing for method Program:Test2(int)
; Emitting BLENDED_CODE for X64 CPU with AVX - Windows
; Tier-1 compilation
; optimized code
; rsp based frame
; partially interruptible
; No PGO data
; 0 inlinees with PGO data; 1 single block inlinees; 1 inlinees without PGO data

G_M000_IG01:                ;; offset=0000H
       56                   push     rsi
       4883EC20             sub      rsp, 32
       8BF1                 mov      esi, ecx

G_M000_IG02:                ;; offset=0007H
       85F6                 test     esi, esi
       7C06                 jl       SHORT G_M000_IG04

G_M000_IG03:                ;; offset=000BH
       4883C420             add      rsp, 32
       5E                   pop      rsi
       C3                   ret

G_M000_IG04:                ;; offset=0011H
       B901000000           mov      ecx, 1
       48BAB81D8C17FC7F0000 mov      rdx, 0x7FFC178C1DB8
       E8EBD4AF5F           call     CORINFO_HELP_STRCNS
       488BC8               mov      rcx, rax
       8BD6                 mov      edx, esi
       FF15D0295400         call     [System.ArgumentOutOfRangeException:ThrowNegative[int](System.String,int)]
       CC                   int3

; Total bytes of code 49

It's almost identical in the non-exceptional path: the only difference is an extra mov esi, ecx (@EgorBo, do you know why it's choosing to copy to another register rather than continue using ecx?). And the overall code is smaller because all of the exception-related goop is pushed off into a helper method.

We could mark all of these as AggressiveInlining if it would help the interpreter.

[EgorBo]

the only difference is an extra mov esi, ecx

Presumably, LSRA limitation? cc @kunalspathak
e.g.:

static void Foo(int a, int b)
{
    if (a < 0)
        Bar(b, a);
}

[MethodImpl(MethodImplOptions.NoInlining)]
static void Bar(int a, int b) { }

; Method Program:Foo(int,int)
       8BC1                 mov      eax, ecx  ;;  <--
       85C0                 test     eax, eax
       7D0A                 jge      SHORT G_M21919_IG05
       8BCA                 mov      ecx, edx
       8BD0                 mov      edx, eax
       FF2578779B00         tail.jmp [Program:Bar(int,int)]
G_M21919_IG05:
       C3                   ret      
; Total bytes of code: 17

[kotlarmilos]

@BrzVlad follow-up on the issue

[BrzVlad]

@stephentoub I agree that we should mark these throw helpers with aggressive inlining.

@EgorBo Do you know how exactly does coreclr optimizes operations from: ldstr; if (uncommon) throw ex (string) to if (uncommon) { ldstr; throw ex (string) }

[EgorBo]

@stephentoub I agree that we should mark these throw helpers with aggressive inlining.

@EgorBo Do you know how exactly does coreclr optimizes operations from: ldstr; if (uncommon) throw ex (string) to if (uncommon) { ldstr; throw ex (string) }

Not near the computer to check this, but presumably you're talking about constant propagation during inlining (of a throw helper in this case).
E.g. when we inline Foo(42) we'll try to propagate 42 by value at all uses inside Foo if it's possible (e.g. not address taken). Same happens with string literals. AFAIR, mono doesn't do it and saves 42 to a local instead.

[stephentoub]

mark these throw helpers with aggressive inlining

What about all of the myriad of other throw helpers we and others write? It sounds like the mono interpreter has a problem inlining the general pattern of:

static void ThrowIf(bool condition)
{
    if (condition) Throw();
}

static void Throw() => throw new Exception();

? If so, wouldn't it be more comprehensive to address that in the interpreter? Presumably this is a pattern that shows up even outside of throw helpers fairly frequently, with a simple wrapper function that conditional delegates to something else.

[kotlarmilos]

@BrzVlad could we address such pattern in the interpreter as @stephentoub suggested? If so, I can convert this issue to the dotnet/runtime repository and add it to the backlog.

[BrzVlad]

We recently tweaked the interpreter to inline by default methods doing calls #83548.
The second problem, propagating the string literal from the parent bblock to the slow path bblock where it is actually used, requires some more advanced optimizations that are planned for the near future.

I don't think this issue serves any purpose anymore.

[kotlarmilos]

Thanks, closing it.