[Perf] Linux/x64: 4 Regressions on 10/11/2022 1:45:10 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare	20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff	Diff

Regressions in System.Numerics.Tests.Perf_Matrix3x2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
AddBenchmark - Duration of single invocation	42.78 ns	53.10 ns	1.24	0.24	False
NegationOperatorBenchmark - Duration of single invocation	34.34 ns	41.02 ns	1.19	0.23	False
AddOperatorBenchmark - Duration of single invocation	42.23 ns	47.76 ns	1.13	0.24	False
MultiplyByScalarOperatorBenchmark - Duration of single invocation	34.89 ns	43.22 ns	1.24	0.22	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

Related Issues

Regressions

Improvements

• [Perf] Windows/x86: 1 Improvement on 10/11/2022 1:45:10 PM

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_Matrix3x2.AddBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 53.10044515593223 > 45.98237786553332.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -8.66022546073199 (T) = (0 -52.0575907195918) / Math.Sqrt((11.113246678532832 / (21)) + (8.410294885642573 / (25))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (25) - 2, .025) and -0.18312014504564122 = (44.00025723303323 - 52.0575907195918) / 44.00025723303323 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.NegationOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 41.01596109658645 > 36.85716946724475.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -12.070708638068094 (T) = (0 -40.19275368506249) / Math.Sqrt((2.5223873233775067 / (22)) + (2.5897820735120334 / (27))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (27) - 2, .025) and -0.1598390379858491 = (34.65373415509478 - 40.19275368506249) / 34.65373415509478 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.AddOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 47.75976263186404 > 43.67136377418964.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -8.132860926797328 (T) = (0 -47.88049571037708) / Math.Sqrt((9.449612644621302 / (20)) + (2.324761124928482 / (27))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (27) - 2, .025) and -0.14540816704513954 = (41.802125292939486 - 47.88049571037708) / 41.802125292939486 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 43.21828319830755 > 36.79250108652093.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 6:01:23 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -3.3210731791804227 (T) = (0 -41.47049716099626) / Math.Sqrt((32.45476689151834 / (22)) + (1.941579778672895 / (27))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (27) - 2, .025) and -0.11062962642298177 = (37.33962805815003 - 41.47049716099626) / 37.33962805815003 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[kotlarmilos]

Hey @BrzVlad looks like regression dotnet/runtime@94d7503