Regression in IfStatements.IfStatements

[performanceautofiler]

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	0a92af9c695d9b1666fb60695b98e2802b8fe047
Compare	4f53c2f7e62df44f07cf410df8a0d439f42a0a71
Diff	Diff

Regressions in IfStatements.IfStatements

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
OrOr - Duration of single invocation	72.13 μs	78.65 μs	1.09	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'IfStatements.IfStatements*'

Payloads

Baseline
Compare

Histogram

IfStatements.IfStatements.OrOr

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.64729989530987 > 75.77867641516748.
IsChangePoint: Marked as a change because one of 10/7/2022 4:12:34 PM, 11/18/2022 7:31:41 AM, 11/25/2022 4:06:25 PM, 12/1/2022 1:41:45 AM falls between 11/22/2022 11:24:47 AM and 12/1/2022 1:41:45 AM.
IsRegressionStdDev: Marked as regression because -32.406495942474656 (T) = (0 -78642.9170985134) / Math.Sqrt((865287.3198576578 / (23)) + (13.154672428249054 / (16))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (16) - 2, .025) and -0.08687026888141984 = (72357.22546670704 - 78642.9170985134) / 72357.22546670704 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[dakersnar]

@SingleAccretion Could this have been caused by #76491?

[SingleAccretion]

It is hard for me to see how it could, though there is nothing else in the commit diff.

There are no Linux ARM64 diffs for this benchmark in #76491 and the change only affects code using vectors.

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	0a92af9c695d9b1666fb60695b98e2802b8fe047
Compare	4f53c2f7e62df44f07cf410df8a0d439f42a0a71
Diff	Diff

Regressions in IfStatements.IfStatements

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
OrOr - Duration of single invocation	72.13 μs	78.65 μs	1.09	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'IfStatements.IfStatements*'

Payloads

Baseline
Compare

Histogram

IfStatements.IfStatements.OrOr

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.64729989530987 > 75.77867641516748.
IsChangePoint: Marked as a change because one of 10/7/2022 4:12:34 PM, 11/18/2022 7:31:41 AM, 11/25/2022 4:06:25 PM, 12/1/2022 1:41:45 AM falls between 11/22/2022 11:24:47 AM and 12/1/2022 1:41:45 AM.
IsRegressionStdDev: Marked as regression because -32.406495942474656 (T) = (0 -78642.9170985134) / Math.Sqrt((865287.3198576578 / (23)) + (13.154672428249054 / (16))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (16) - 2, .025) and -0.08687026888141984 = (72357.22546670704 - 78642.9170985134) / 72357.22546670704 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	DrewScoggins
Labels:	tenet-performance, tenet-performance-benchmarks, area-CodeGen-coreclr, untriaged
Milestone:	-

[JulieLeeMSFT]

@AndyAyersMS PTAL.

[AndyAyersMS]

This test was bimodal, and then regressed when PGO was enabled

So this does not appear to be a regression per se (commit list shows nothing relevant).

That later regression seems possibly to be a similar problem to #87194 (comment) -- with PGO we inline the if-convertable method into a loop?

Looking cross-plat things looks similar elsewhere