Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 98 Regressions on 11/2/2022 4:25:40 PM #9612

Open
performanceautofiler bot opened this issue Nov 8, 2022 · 6 comments
Open

[Perf] Linux/x64: 98 Regressions on 11/2/2022 4:25:40 PM #9612

performanceautofiler bot opened this issue Nov 8, 2022 · 6 comments
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InequalityOperatorBenchmark - Duration of single invocation 55.77 ns 191.93 ns 3.44 0.07 False
XorBenchmark - Duration of single invocation 75.00 ns 95.88 ns 1.28 0.08 False
BitwiseOrOperatorBenchmark - Duration of single invocation 50.19 ns 96.50 ns 1.92 0.10 False
BitwiseOrBenchmark - Duration of single invocation 51.05 ns 96.05 ns 1.88 0.05 False
BitwiseAndBenchmark - Duration of single invocation 47.95 ns 98.49 ns 2.05 0.08 False
BitwiseAndOperatorBenchmark - Duration of single invocation 47.95 ns 96.38 ns 2.01 0.09 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 48.65 ns 96.87 ns 1.99 0.09 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt16>.InequalityOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 191.92544769125442 > 57.5915730588647.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -227.69491497352482 (T) = (0 -192.48454118592326) / Math.Sqrt((0.9110207730284322 / (23)) + (4.571312882416045 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -2.5183345051877666 = (54.70899395782458 - 192.48454118592326) / 54.70899395782458 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.88174097822557 > 50.58945162203158.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -25.907177456886693 (T) = (0 -99.26573873094222) / Math.Sqrt((29.802349021198978 / (24)) + (34.168975718234634 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -1.0033079360310868 = (49.55091373900584 - 99.26573873094222) / 49.55091373900584 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.49937494580672 > 50.60726119301211.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -32.93695262215574 (T) = (0 -98.20874337912679) / Math.Sqrt((1.5557461599057532 / (24)) + (28.58064730396688 / (13))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (13) - 2, .025) and -1.018383150609948 = (48.6571359602603 - 98.20874337912679) / 48.6571359602603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.05228988697847 > 50.986522074937824.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -163.76630142911634 (T) = (0 -96.32295992470185) / Math.Sqrt((1.257178130511712 / (23)) + (0.38654080189238266 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.9759430871140337 = (48.747841247485766 - 96.32295992470185) / 48.747841247485766 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 98.49310516553548 > 50.947447975172274.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -73.2288887090014 (T) = (0 -97.92741780459922) / Math.Sqrt((1.132268900948006 / (24)) + (5.6900974952294 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -1.014670850175546 = (48.60715475982909 - 97.92741780459922) / 48.60715475982909 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.38412862655488 > 50.462239214490126.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -144.56793988266108 (T) = (0 -96.50929989346791) / Math.Sqrt((0.728847090054973 / (22)) + (1.071992629646582 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -0.9846896358608063 = (48.62689770212337 - 96.50929989346791) / 48.62689770212337 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.86824946514473 > 51.04206706052663.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -30.570791968623944 (T) = (0 -98.23047437382) / Math.Sqrt((17.08760412083286 / (24)) + (23.896062591312358 / (13))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (13) - 2, .025) and -0.9880094070351789 = (49.411473620900104 - 98.23047437382) / 49.411473620900104 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ExclusiveOrOperatorBenchmark - Duration of single invocation 49.20 ns 95.91 ns 1.95 0.08 False
InequalityOperatorBenchmark - Duration of single invocation 50.34 ns 279.90 ns 5.56 0.09 False
XorBenchmark - Duration of single invocation 48.69 ns 96.48 ns 1.98 0.10 False
BitwiseOrOperatorBenchmark - Duration of single invocation 47.92 ns 99.49 ns 2.08 0.10 False
BitwiseAndBenchmark - Duration of single invocation 49.39 ns 96.20 ns 1.95 0.07 False
BitwiseAndOperatorBenchmark - Duration of single invocation 47.95 ns 95.70 ns 2.00 0.05 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Byte>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.91222497568516 > 50.56757825417584.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -126.5905501464865 (T) = (0 -96.31659917443685) / Math.Sqrt((0.8225681538162195 / (23)) + (1.4960861831413956 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.9855652621243578 = (48.5084026255514 - 96.31659917443685) / 48.5084026255514 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 279.9017937097031 > 53.83747450847681.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -543.6443934994658 (T) = (0 -280.7780922028988) / Math.Sqrt((1.7872703536993064 / (24)) + (1.4595642120326853 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -4.511053861835897 = (50.94816694630583 - 280.7780922028988) / 50.94816694630583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.4787722398602 > 51.29263117502065.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -29.730888340959115 (T) = (0 -98.12707730094687) / Math.Sqrt((5.349514041041855 / (24)) + (34.472707914847064 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9860555291312795 = (49.408023019310356 - 98.12707730094687) / 49.408023019310356 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 99.4861827576989 > 50.26961540718703.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -118.60844825529325 (T) = (0 -97.4770021144049) / Math.Sqrt((0.3657512466813394 / (24)) + (2.193973772659016 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -1.0184283174078976 = (48.29351692785733 - 97.4770021144049) / 48.29351692785733 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.20217610508794 > 50.70361914818713.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -98.40771593022379 (T) = (0 -96.88068770999422) / Math.Sqrt((1.7524356950590516 / (23)) + (2.146122699670021 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.9957853258159508 = (48.54263955988644 - 96.88068770999422) / 48.54263955988644 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.696178643681 > 50.59674199240629.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -136.20362055050867 (T) = (0 -96.35138939842778) / Math.Sqrt((0.6732245800028778 / (24)) + (1.2415318018355708 / (13))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (13) - 2, .025) and -0.9876273532187982 = (48.4755803155932 - 96.35138939842778) / 48.4755803155932 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseAndOperatorBenchmark - Duration of single invocation 47.59 ns 94.85 ns 1.99 0.07 False
XorBenchmark - Duration of single invocation 48.21 ns 96.03 ns 1.99 0.05 False
BitwiseOrOperatorBenchmark - Duration of single invocation 48.23 ns 95.97 ns 1.99 0.07 False
BitwiseAndBenchmark - Duration of single invocation 50.80 ns 95.75 ns 1.88 0.07 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 47.96 ns 95.67 ns 1.99 0.07 False
InequalityOperatorBenchmark - Duration of single invocation 58.63 ns 86.37 ns 1.47 0.11 False
BitwiseOrBenchmark - Duration of single invocation 48.11 ns 98.20 ns 2.04 0.10 False
ConditionalSelectBenchmark - Duration of single invocation 206.99 ns 357.36 ns 1.73 0.10 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Double>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 94.85178397828227 > 50.19563965006899.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -33.160270169460546 (T) = (0 -97.8049901761651) / Math.Sqrt((0.7253487602272041 / (24)) + (30.623309361420226 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -1.0197630983150279 = (48.42399104021565 - 97.8049901761651) / 48.42399104021565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.02650143467453 > 50.80128958611014.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -132.0188588039626 (T) = (0 -96.31368133422473) / Math.Sqrt((1.4988293535530235 / (24)) + (0.9543835702997202 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9817645383387759 = (48.59996203936526 - 96.31368133422473) / 48.59996203936526 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.96631804816252 > 50.84174598459494.
IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -243.21496692127042 (T) = (0 -96.00631489392745) / Math.Sqrt((0.6462508815541488 / (23)) + (0.14745461130252913 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.991692511737875 = (48.20338196188527 - 96.00631489392745) / 48.20338196188527 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.75127033898963 > 50.95922427778627.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -259.31079760737623 (T) = (0 -96.0825789569291) / Math.Sqrt((0.4953062981423236 / (24)) + (0.1737607552309884 / (13))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (13) - 2, .025) and -0.9907201767093713 = (48.2652359086208 - 96.0825789569291) / 48.2652359086208 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.66613803784648 > 50.10105788754565.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -129.41616901452196 (T) = (0 -96.61965398339706) / Math.Sqrt((0.5300149856167049 / (24)) + (1.5321594758028105 / (13))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (13) - 2, .025) and -1.0042858720449763 = (48.20652349598007 - 96.61965398339706) / 48.20652349598007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 86.37350290839403 > 60.892697344778206.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -40.91668100054353 (T) = (0 -81.10869698801167) / Math.Sqrt((1.099290189101042 / (23)) + (3.4601839421617218 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.3940492166707745 = (58.1820899994571 - 81.10869698801167) / 58.1820899994571 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 98.19846109897627 > 51.27420679196366.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -35.23407164873306 (T) = (0 -98.34268861612944) / Math.Sqrt((0.9074382531253558 / (24)) + (27.19654002479333 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -1.0169260614068574 = (48.758697950252525 - 98.34268861612944) / 48.758697950252525 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 357.3632937436377 > 219.73349129652124.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -71.61071909334534 (T) = (0 -359.601302609402) / Math.Sqrt((58.46368958556225 / (23)) + (23.988241876254754 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.6971995407155152 = (211.87921277529878 - 359.601302609402) / 211.87921277529878 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d3a3eda6f4660e85f7a603b624b43993529be377
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseOrBenchmark - Duration of single invocation 48.17 ns 95.12 ns 1.97 0.09 False
ConditionalSelectBenchmark - Duration of single invocation 205.03 ns 357.62 ns 1.74 0.07 False
BitwiseOrOperatorBenchmark - Duration of single invocation 48.58 ns 96.40 ns 1.98 0.12 False
XorBenchmark - Duration of single invocation 49.57 ns 95.83 ns 1.93 0.06 False
BitwiseAndOperatorBenchmark - Duration of single invocation 48.23 ns 96.55 ns 2.00 0.12 False
InequalityOperatorBenchmark - Duration of single invocation 54.55 ns 73.76 ns 1.35 0.08 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 48.04 ns 96.23 ns 2.00 0.09 False
BitwiseAndBenchmark - Duration of single invocation 47.96 ns 96.77 ns 2.02 0.07 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int64>.BitwiseOrBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.12280163610497 > 50.50921380197422.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -54.40617609858815 (T) = (0 -96.20472513223616) / Math.Sqrt((16.09064026126927 / (23)) + (0.5407218537144862 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.9488353563526991 = (49.36524002329578 - 96.20472513223616) / 49.36524002329578 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 357.61534339885924 > 215.29653769705814.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -208.36738852921047 (T) = (0 -358.4218354671053) / Math.Sqrt((6.109060046244078 / (24)) + (3.8811341225457676 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.7359067749083873 = (206.4752788847322 - 358.4218354671053) / 206.4752788847322 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.39946582196255 > 50.449655088029914.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -39.61192128774731 (T) = (0 -97.94107069531891) / Math.Sqrt((0.9400435550677619 / (24)) + (21.28189409259217 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -1.0203683609624832 = (48.47683847546532 - 97.94107069531891) / 48.47683847546532 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.8299661581851 > 51.54287318450783.
IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -91.98061059170884 (T) = (0 -96.44272659283362) / Math.Sqrt((3.788117259963684 / (23)) + (1.370775277281959 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.9559832089941114 = (49.30652070496581 - 96.44272659283362) / 49.30652070496581 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.5500125465877 > 50.8461302803367.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -58.854508356721915 (T) = (0 -96.5533826194992) / Math.Sqrt((12.894974996044343 / (24)) + (1.3852971153482008 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9463161708503928 = (49.60827231749952 - 96.5533826194992) / 49.60827231749952 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 73.76074241146526 > 57.2512061665098.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -16.57429248712799 (T) = (0 -72.89879554414694) / Math.Sqrt((23.661891589174743 / (23)) + (0.7683160541340209 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.3108566570938809 = (55.61156908319845 - 72.89879554414694) / 55.61156908319845 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.23269709829253 > 51.114859871060645.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -102.89769304507946 (T) = (0 -97.45536598687315) / Math.Sqrt((0.7378341772008161 / (24)) + (2.70484608186861 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9986286220543725 = (48.76111795431992 - 97.45536598687315) / 48.76111795431992 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.7694828084752 > 50.863614177367566.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -35.47229848569621 (T) = (0 -96.79384752998043) / Math.Sqrt((31.157328952242896 / (23)) + (5.5063191599798555 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.9557694246859573 = (49.4914412242245 - 96.79384752998043) / 49.4914412242245 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline d3a3eda6f4660e85f7a603b624b43993529be377
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsPrefix_FirstHalf - Duration of single invocation 7.91 μs 8.54 μs 1.08 0.14 False
IsPrefix_FirstHalf - Duration of single invocation 7.81 μs 8.59 μs 1.10 0.09 False
IsPrefix_FirstHalf - Duration of single invocation 11.86 μs 12.54 μs 1.06 0.13 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, True))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 8.544743974076116 > 8.213318171032084.
IsChangePoint: Marked as a change because one of 9/7/2022 8:46:47 AM, 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -32.19517882915821 (T) = (0 -8628.567755296535) / Math.Sqrt((2261.8290808726492 / (23)) + (6970.628476930153 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.1037408049479922 = (7817.5670561560055 - 8628.567755296535) / 7817.5670561560055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreCase, True))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 8.58529605933823 > 8.182214775557586.
IsChangePoint: Marked as a change because one of 9/7/2022 8:46:47 AM, 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -24.523274138646258 (T) = (0 -8654.602485497342) / Math.Sqrt((15764.779920229059 / (24)) + (6298.239919573801 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.1040766693505792 = (7838.76946751171 - 8654.602485497342) / 7838.76946751171 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (pl-PL, None, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 12.539350361291195 > 12.458545956520487.
IsChangePoint: Marked as a change because one of 9/7/2022 8:46:47 AM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -3.699219862158057 (T) = (0 -13097.781620894912) / Math.Sqrt((223038.61128945267 / (24)) + (2158620.126187853 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.12891602940603003 = (11602.08667405157 - 13097.781620894912) / 11602.08667405157 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseAndBenchmark - Duration of single invocation 48.73 ns 97.22 ns 2.00 0.08 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 50.15 ns 96.08 ns 1.92 0.11 False
BitwiseAndOperatorBenchmark - Duration of single invocation 48.23 ns 96.00 ns 1.99 0.08 False
XorBenchmark - Duration of single invocation 50.80 ns 95.85 ns 1.89 0.10 False
BitwiseOrOperatorBenchmark - Duration of single invocation 48.03 ns 95.90 ns 2.00 0.11 False
BitwiseOrBenchmark - Duration of single invocation 47.85 ns 95.36 ns 1.99 0.10 False
ConditionalSelectBenchmark - Duration of single invocation 220.97 ns 357.34 ns 1.62 0.08 False
InequalityOperatorBenchmark - Duration of single invocation 54.19 ns 111.20 ns 2.05 0.14 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int32>.BitwiseAndBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 97.21844173395623 > 51.77852504517287.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -61.08701317966335 (T) = (0 -97.05088903912326) / Math.Sqrt((3.806472463065016 / (23)) + (6.331790617951153 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.9791147869452129 = (49.03752408869748 - 97.05088903912326) / 49.03752408869748 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.07717599693252 > 50.396008731129236.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -28.231699395276436 (T) = (0 -98.53719220591266) / Math.Sqrt((0.4086997247448984 / (24)) + (44.03371486609595 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -1.038705782676049 = (48.333208765695765 - 98.53719220591266) / 48.333208765695765 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.99890703318678 > 50.76659074266164.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -104.66481524657493 (T) = (0 -96.82704958249724) / Math.Sqrt((0.5603836971690419 / (23)) + (2.457505746428133 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.9974017144233416 = (48.47650269012191 - 96.82704958249724) / 48.47650269012191 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.8545512840364 > 52.035638928022095.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -40.77471666550106 (T) = (0 -97.59877926841985) / Math.Sqrt((12.726559280056975 / (24)) + (11.74290564439484 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9562908740356306 = (49.88970738644986 - 97.59877926841985) / 49.88970738644986 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.90448474223166 > 51.49541773733721.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -56.80561443537422 (T) = (0 -96.20729068961894) / Math.Sqrt((15.292801898105534 / (23)) + (0.07539578499425563 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.9363266413036139 = (49.685465580770135 - 96.20729068961894) / 49.685465580770135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.36034080057541 > 50.42359148222783.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -114.0527186571468 (T) = (0 -96.41522879985577) / Math.Sqrt((1.0892758202255386 / (24)) + (1.8211571761230738 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9822355567919491 = (48.63964248320428 - 96.41522879985577) / 48.63964248320428 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 357.3387400805788 > 233.02901997151903.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -78.77148376708456 (T) = (0 -359.5637517090755) / Math.Sqrt((39.73357023087309 / (24)) + (17.054930421050848 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.5907882713400185 = (226.0286665341032 - 359.5637517090755) / 226.0286665341032 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 111.19835874915604 > 58.54636578062326.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -42.79949042372026 (T) = (0 -109.98404582091904) / Math.Sqrt((31.02480938523274 / (23)) + (2.8810324596510393 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.9425458147583938 = (56.6185080348277 - 109.98404582091904) / 56.6185080348277 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InequalityOperatorBenchmark - Duration of single invocation 54.18 ns 191.97 ns 3.54 0.10 False
BitwiseOrOperatorBenchmark - Duration of single invocation 47.82 ns 98.68 ns 2.06 0.08 False
BitwiseOrBenchmark - Duration of single invocation 48.00 ns 95.91 ns 2.00 0.06 False
BitwiseAndBenchmark - Duration of single invocation 48.96 ns 96.04 ns 1.96 0.07 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 47.64 ns 95.99 ns 2.02 0.08 False
BitwiseAndOperatorBenchmark - Duration of single invocation 48.64 ns 95.87 ns 1.97 0.06 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int16>.InequalityOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 191.97069259099823 > 57.35011622508773.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -96.93158026656864 (T) = (0 -193.6297902693425) / Math.Sqrt((0.8750048408313574 / (24)) + (28.28905122657787 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -2.5460350334537263 = (54.6045903220401 - 193.6297902693425) / 54.6045903220401 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 98.67666023176041 > 50.525458727873676.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -82.117223974872 (T) = (0 -96.24046935341616) / Math.Sqrt((5.638143209616878 / (24)) + (1.420109368210182 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9796300654144746 = (48.61538073946473 - 96.24046935341616) / 48.61538073946473 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.9100674786617 > 50.41300738251568.
IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -224.33794941175893 (T) = (0 -96.29130627564007) / Math.Sqrt((0.3072751791302606 / (23)) + (0.4541402962748317 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.994362964626863 = (48.281736064857064 - 96.29130627564007) / 48.281736064857064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.03738354738924 > 50.956652136975706.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -119.26270265835856 (T) = (0 -96.65960155505455) / Math.Sqrt((1.141151620686749 / (24)) + (1.623539673746009 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9957244042850627 = (48.43334147115436 - 96.65960155505455) / 48.43334147115436 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.99218999878573 > 50.66972268542233.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -181.64059226039365 (T) = (0 -96.33614232714483) / Math.Sqrt((0.6630168193229992 / (24)) + (0.5905744953439952 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9927232792539309 = (48.3439639261969 - 96.33614232714483) / 48.3439639261969 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.87061723831863 > 50.997593927338244.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -138.50592452536134 (T) = (0 -96.53869718553851) / Math.Sqrt((0.6539044740474286 / (23)) + (1.2860028631402116 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.9904352158156491 = (48.5013008303204 - 96.53869718553851) / 48.5013008303204 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseOrBenchmark - Duration of single invocation 46.67 ns 95.83 ns 2.05 0.08 False
ConditionalSelectBenchmark - Duration of single invocation 206.55 ns 356.26 ns 1.72 0.07 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 48.29 ns 95.84 ns 1.98 0.05 False
BitwiseOrOperatorBenchmark - Duration of single invocation 48.46 ns 96.00 ns 1.98 0.09 False
InequalityOperatorBenchmark - Duration of single invocation 54.40 ns 72.84 ns 1.34 0.12 False
BitwiseAndBenchmark - Duration of single invocation 50.96 ns 96.33 ns 1.89 0.09 False
XorBenchmark - Duration of single invocation 48.28 ns 98.05 ns 2.03 0.07 False
BitwiseAndOperatorBenchmark - Duration of single invocation 49.50 ns 96.09 ns 1.94 0.09 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt64>.BitwiseOrBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.82805312138979 > 50.2834865881927.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -130.05705056368055 (T) = (0 -96.61341779599285) / Math.Sqrt((0.49592086319126916 / (24)) + (1.6610468014468271 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -1.0098205867491303 = (48.070667816307086 - 96.61341779599285) / 48.070667816307086 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 356.2582001621826 > 217.00732114854452.
IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -175.5643448436053 (T) = (0 -359.193754348334) / Math.Sqrt((4.671148648908278 / (23)) + (7.629892001251282 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.7323505147365131 = (207.34473265819798 - 359.193754348334) / 207.34473265819798 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.83663486729915 > 51.05149901340364.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -145.31358697574504 (T) = (0 -96.56120773728261) / Math.Sqrt((0.4211683415216381 / (24)) + (1.2881631091106298 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9924909736322798 = (48.46255717849153 - 96.56120773728261) / 48.46255717849153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.99516096752849 > 50.41516752336444.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -232.5035098524497 (T) = (0 -96.28244178221402) / Math.Sqrt((0.5666784353703554 / (23)) + (0.24005971343398397 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.9748082317040832 = (48.75533747351806 - 96.28244178221402) / 48.75533747351806 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 72.84044147250422 > 57.05955282106516.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -34.99518547588376 (T) = (0 -73.15969692540806) / Math.Sqrt((0.5585647610866854 / (24)) + (3.671744846766932 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.3433741983738073 = (54.459656150884804 - 73.15969692540806) / 54.459656150884804 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.32877029126215 > 51.55514211085966.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -150.8729253965234 (T) = (0 -96.76524464662647) / Math.Sqrt((1.0729800704328352 / (24)) + (0.7914826623168196 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9845342249963245 = (48.75967540786841 - 96.76524464662647) / 48.75967540786841 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 98.05160903482309 > 51.07621295932212.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -50.51820655169287 (T) = (0 -97.63605769469903) / Math.Sqrt((1.6820710793390183 / (23)) + (11.11930537621121 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.9942700436339722 = (48.95829328950153 - 97.63605769469903) / 48.95829328950153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.0869354974656 > 51.64298761899575.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -242.8961085318722 (T) = (0 -96.16228113903318) / Math.Sqrt((0.45473595440623976 / (24)) + (0.2541675880003706 / (13))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (13) - 2, .025) and -0.9825869295935106 = (48.50343745519866 - 96.16228113903318) / 48.50343745519866 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseAndOperatorBenchmark - Duration of single invocation 49.25 ns 94.78 ns 1.92 0.09 False
BitwiseAndBenchmark - Duration of single invocation 48.14 ns 96.08 ns 2.00 0.05 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 48.65 ns 95.67 ns 1.97 0.09 False
InequalityOperatorBenchmark - Duration of single invocation 52.43 ns 279.84 ns 5.34 0.10 False
XorBenchmark - Duration of single invocation 48.74 ns 98.21 ns 2.02 0.08 False
BitwiseOrBenchmark - Duration of single invocation 51.55 ns 97.79 ns 1.90 0.11 False
BitwiseOrOperatorBenchmark - Duration of single invocation 48.24 ns 97.68 ns 2.02 0.10 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<SByte>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 94.78098526595384 > 51.585698226625844.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -110.94429751145776 (T) = (0 -96.69490246397972) / Math.Sqrt((1.1728023592370136 / (24)) + (1.9329955799371215 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9844345107990673 = (48.72667852618821 - 96.69490246397972) / 48.72667852618821 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.07664164068149 > 50.588238357978675.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -55.98823294941862 (T) = (0 -96.61668989259412) / Math.Sqrt((16.500431799595514 / (24)) + (0.45758572497523026 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.967656681039757 = (49.10241243992806 - 96.61668989259412) / 49.10241243992806 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.66515843791537 > 51.02977172015098.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -44.3136118943051 (T) = (0 -97.04930834150457) / Math.Sqrt((7.491469895147304 / (22)) + (11.594492075864512 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -0.974816432242316 = (49.14345797260225 - 97.04930834150457) / 49.14345797260225 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 279.83979693843514 > 53.78883744030381.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -687.5134995684481 (T) = (0 -280.09285047326244) / Math.Sqrt((1.305721217216351 / (24)) + (0.7404681733721664 / (13))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (13) - 2, .025) and -4.52877869981527 = (50.660890167772685 - 280.09285047326244) / 50.660890167772685 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 98.21180919929215 > 51.333197469819304.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -161.23382061735686 (T) = (0 -96.74572903246624) / Math.Sqrt((0.7931802378957273 / (24)) + (0.7706724794669859 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9788596029463965 = (48.88963769254675 - 96.74572903246624) / 48.88963769254675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 97.7858394941456 > 50.45912902626121.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -24.58275827255607 (T) = (0 -99.31011767953918) / Math.Sqrt((1.2269969565042431 / (24)) + (58.94196026476686 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -1.0449183315421537 = (48.564344183195566 - 99.31011767953918) / 48.564344183195566 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 97.67669497961151 > 50.51128230381015.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -37.12866357873168 (T) = (0 -97.64717243223701) / Math.Sqrt((0.9329909505035636 / (23)) + (23.936728669309392 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -1.0122683287804446 = (48.52592024415405 - 97.64717243223701) / 48.52592024415405 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseAndBenchmark - Duration of single invocation 49.27 ns 96.63 ns 1.96 0.08 False
InequalityOperatorBenchmark - Duration of single invocation 54.88 ns 109.17 ns 1.99 0.13 False
BitwiseOrBenchmark - Duration of single invocation 47.56 ns 95.85 ns 2.02 0.07 False
XorBenchmark - Duration of single invocation 49.95 ns 102.49 ns 2.05 0.07 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 47.43 ns 97.15 ns 2.05 0.09 False
BitwiseAndOperatorBenchmark - Duration of single invocation 48.81 ns 95.81 ns 1.96 0.08 False
ConditionalSelectBenchmark - Duration of single invocation 223.43 ns 356.74 ns 1.60 0.09 False
BitwiseOrOperatorBenchmark - Duration of single invocation 47.64 ns 95.98 ns 2.01 0.09 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt32>.BitwiseAndBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.6344286317765 > 51.20055362903157.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -104.78752837817132 (T) = (0 -96.83698463871512) / Math.Sqrt((1.2645357910443085 / (22)) + (2.172800035208752 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -0.9961453195272291 = (48.5119914323924 - 96.83698463871512) / 48.5119914323924 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 109.16542711583479 > 56.96699107361023.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -46.86039578599727 (T) = (0 -109.67690302613423) / Math.Sqrt((17.697300704477453 / (23)) + (7.262585685115628 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.9700088441474961 = (55.67330489503256 - 109.67690302613423) / 55.67330489503256 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.84759430728384 > 50.311222019526895.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -101.65469818931707 (T) = (0 -96.57398321864956) / Math.Sqrt((1.0635401712715848 / (24)) + (2.3059815092963856 / (13))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (13) - 2, .025) and -0.9826265575556221 = (48.71012286737221 - 96.57398321864956) / 48.71012286737221 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 102.49314511070918 > 51.957274391795856.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -76.35046376154997 (T) = (0 -97.21348023047032) / Math.Sqrt((0.8532041709107413 / (23)) + (5.123785584307693 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.9945472521617933 = (48.73962255098511 - 97.21348023047032) / 48.73962255098511 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 97.14945151553614 > 50.609408118445714.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -116.8577270747861 (T) = (0 -96.76903969007495) / Math.Sqrt((0.824609548968563 / (24)) + (1.8967956456615487 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9907451116317457 = (48.609457395957975 - 96.76903969007495) / 48.609457395957975 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.81487192908797 > 49.999782078029384.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -143.61673249884836 (T) = (0 -96.41826001170088) / Math.Sqrt((1.1859050702801408 / (24)) + (0.8757719557055821 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9936860512609406 = (48.36180699098512 - 96.41826001170088) / 48.36180699098512 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 356.7391093029363 > 234.31841473882423.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -58.798592462381066 (T) = (0 -360.92008145023647) / Math.Sqrt((30.216258867654822 / (24)) + (57.43996386954703 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.6057462884345882 = (224.7678129787805 - 360.92008145023647) / 224.7678129787805 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.98375275315452 > 50.8286685294356.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -222.3840585867757 (T) = (0 -96.0132514157012) / Math.Sqrt((0.5820772604914952 / (24)) + (0.3009713840967245 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9818465885401787 = (48.446359052657165 - 96.0132514157012) / 48.446359052657165 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseOrOperatorBenchmark - Duration of single invocation 47.91 ns 96.02 ns 2.00 0.07 False
XorBenchmark - Duration of single invocation 48.94 ns 96.24 ns 1.97 0.08 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 47.77 ns 99.05 ns 2.07 0.08 False
BitwiseAndBenchmark - Duration of single invocation 49.37 ns 96.12 ns 1.95 0.08 False
ConditionalSelectBenchmark - Duration of single invocation 227.72 ns 357.20 ns 1.57 0.09 False
BitwiseOrBenchmark - Duration of single invocation 48.54 ns 95.82 ns 1.97 0.07 False
InequalityOperatorBenchmark - Duration of single invocation 58.32 ns 120.86 ns 2.07 0.08 False
BitwiseAndOperatorBenchmark - Duration of single invocation 48.03 ns 96.53 ns 2.01 0.07 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Single>.BitwiseOrOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.01618198230142 > 50.40456094994668.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -32.15713708416459 (T) = (0 -97.94892752016837) / Math.Sqrt((0.8810699954372153 / (23)) + (30.437375304904464 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -1.0261233219647146 = (48.34302357528175 - 97.94892752016837) / 48.34302357528175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.23865299198427 > 51.29279418750577.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -33.91842543022786 (T) = (0 -97.84364436667404) / Math.Sqrt((0.620101867607661 / (23)) + (26.882337285384548 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -1.0069798647041488 = (48.75168211072077 - 97.84364436667404) / 48.75168211072077 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 99.0462853489888 > 50.499070588185276.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -144.08434844474354 (T) = (0 -96.37383573759335) / Math.Sqrt((1.1063906384649833 / (24)) + (0.907442431295834 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9917008454327614 = (48.387706396064225 - 96.37383573759335) / 48.387706396064225 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.11525484072004 > 51.01162229353459.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -50.425474907637955 (T) = (0 -96.28145875829368) / Math.Sqrt((19.666157716470305 / (24)) + (0.6888485638408383 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.953571383563913 = (49.284842913007246 - 96.28145875829368) / 49.284842913007246 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 357.19757426823946 > 239.33213152463426.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -91.87204262910005 (T) = (0 -358.53026429177936) / Math.Sqrt((25.780513175367265 / (24)) + (12.159678934289461 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.5556004045977396 = (230.47709632377678 - 358.53026429177936) / 230.47709632377678 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.82312447626032 > 51.18658852928413.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -98.60902533002611 (T) = (0 -96.57565014667016) / Math.Sqrt((0.5747311438127005 / (23)) + (2.9810662367444007 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.9921985740455807 = (48.476919622802896 - 96.57565014667016) / 48.476919622802896 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 120.8636534489909 > 60.78389311425647.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -174.2242625119833 (T) = (0 -121.78157929265535) / Math.Sqrt((0.8835758912829271 / (24)) + (1.3411557997533614 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -1.0875862070590452 = (58.336072005485754 - 121.78157929265535) / 58.336072005485754 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.52743215827873 > 51.02438014872784.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -148.1379852345087 (T) = (0 -96.70226527463792) / Math.Sqrt((0.9398177175754058 / (24)) + (0.9251651948086226 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9879293895258744 = (48.64471836079733 - 96.70226527463792) / 48.64471836079733 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Convert_uint_float - Duration of single invocation 240.79 μs 320.39 μs 1.33 0.10 False
Convert_double_ulong - Duration of single invocation 122.16 μs 200.29 μs 1.64 0.07 False
Convert_double_long - Duration of single invocation 117.44 μs 197.19 μs 1.68 0.10 False
Narrow_ushort - Duration of single invocation 378.89 μs 471.56 μs 1.24 0.05 False
Widen_short - Duration of single invocation 401.70 μs 529.98 μs 1.32 0.08 False
Convert_long_double - Duration of single invocation 337.38 μs 396.37 μs 1.17 0.07 False
Convert_float_uint - Duration of single invocation 166.05 μs 244.78 μs 1.47 0.10 False
Narrow_ulong - Duration of single invocation 178.62 μs 282.80 μs 1.58 0.11 False
Widen_float - Duration of single invocation 348.38 μs 482.51 μs 1.39 0.09 False
Narrow_int - Duration of single invocation 246.97 μs 341.55 μs 1.38 0.16 False
Narrow_long - Duration of single invocation 179.76 μs 290.99 μs 1.62 0.13 False
Widen_int - Duration of single invocation 353.74 μs 461.35 μs 1.30 0.07 False
Convert_ulong_double - Duration of single invocation 315.87 μs 416.94 μs 1.32 0.07 False
Widen_ushort - Duration of single invocation 398.43 μs 514.68 μs 1.29 0.05 False
Narrow_short - Duration of single invocation 373.25 μs 469.32 μs 1.26 0.05 False
Widen_sbyte - Duration of single invocation 536.78 μs 651.49 μs 1.21 0.04 False
Convert_float_int - Duration of single invocation 146.44 μs 205.32 μs 1.40 0.08 False
Narrow_double - Duration of single invocation 169.96 μs 247.16 μs 1.45 0.14 False
Widen_byte - Duration of single invocation 518.46 μs 648.35 μs 1.25 0.02 False
Convert_int_float - Duration of single invocation 154.13 μs 232.07 μs 1.51 0.06 False
Widen_uint - Duration of single invocation 317.06 μs 491.44 μs 1.55 0.08 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorConvert.Convert_uint_float

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 320.3942307692308 > 255.37860640333628.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -57.48000000217138 (T) = (0 -321846.77160015336) / Math.Sqrt((15708841.299879313 / (24)) + (17599042.8880998 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.32789112913752777 = (242374.36679706906 - 321846.77160015336) / 242374.36679706906 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_double_ulong

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 200.28995705128207 > 129.7628247318209.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -46.5150092769702 (T) = (0 -204770.87984022478) / Math.Sqrt((24748310.13385887 / (24)) + (27833800.750540312 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.6521019121451291 = (123945.6708662393 - 204770.87984022478) / 123945.6708662393 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_double_long

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 197.19031260416668 > 124.0972177323499.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -143.66450385230414 (T) = (0 -196963.44460098867) / Math.Sqrt((2918614.6350038475 / (23)) + (2389427.2117089797 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.6608138311771459 = (118594.53534378721 - 196963.44460098867) / 118594.53534378721 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_ushort

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 471.56118644957985 > 397.0092675560898.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -49.30775951392731 (T) = (0 -473131.9331248125) / Math.Sqrt((78193121.32809801 / (24)) + (4577869.0804457925 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.24583264822571563 = (379771.66018135374 - 473131.9331248125) / 379771.66018135374 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_short

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 529.9777085416667 > 420.46400472656256.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -33.64567297663295 (T) = (0 -517895.70061127795) / Math.Sqrt((164014844.52927956 / (24)) + (54825870.67601795 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.27065853788181676 = (407580.54596997256 - 517895.70061127795) / 407580.54596997256 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_long_double

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 396.37209832589286 > 332.0713808702806.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -34.30215590974856 (T) = (0 -400185.7455005335) / Math.Sqrt((39420884.775853 / (24)) + (55250357.193886094 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.2541396320813213 = (319091.8580863287 - 400185.7455005335) / 319091.8580863287 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_float_uint

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 244.77854778645835 > 174.77796713799648.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -21.124063044770704 (T) = (0 -250757.13433370425) / Math.Sqrt((6499791.71833524 / (24)) + (199594356.0160648 / (13))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (13) - 2, .025) and -0.49921474795456144 = (167258.98319491735 - 250757.13433370425) / 167258.98319491735 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_ulong

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 282.801042485119 > 185.82097256320222.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -54.625544744763204 (T) = (0 -287486.2617669724) / Math.Sqrt((12253363.542199068 / (24)) + (49456130.12985031 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.6182947711696173 = (177647.64917283435 - 287486.2617669724) / 177647.64917283435 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_float

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 482.5109371323528 > 367.3647213958333.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -45.462555314608444 (T) = (0 -469627.3275032988) / Math.Sqrt((90219664.96132152 / (24)) + (40688661.830581084 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.333206847722362 = (352253.9119159234 - 469627.3275032988) / 352253.9119159234 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_int

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 341.55347399068324 > 260.9546811147836.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -30.453638249216564 (T) = (0 -348786.58568163984) / Math.Sqrt((100413619.70676722 / (23)) + (68679387.18995418 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.37215060559115165 = (254189.7254284089 - 348786.58568163984) / 254189.7254284089 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_long

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 290.9928076223544 > 186.04319995978042.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -53.02760999678919 (T) = (0 -273636.71828216076) / Math.Sqrt((28437966.490212206 / (24)) + (27737641.582588494 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.5263031414724332 = (179280.7148507746 - 273636.71828216076) / 179280.7148507746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_int

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 461.3537984068628 > 372.16497741840277.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -49.94106050954364 (T) = (0 -463866.57250069187) / Math.Sqrt((55740919.822265655 / (23)) + (36184284.85145476 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.3256829328361013 = (349907.6295025677 - 463866.57250069187) / 349907.6295025677 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_ulong_double

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 416.9375 > 331.51317121875.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -34.18040161754264 (T) = (0 -426553.30895168154) / Math.Sqrt((70379426.71940236 / (24)) + (99346605.27917725 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.34005374747704037 = (318310.597432951 - 426553.30895168154) / 318310.597432951 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_ushort

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 514.6834683179725 > 419.8582933999399.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -33.147837086976324 (T) = (0 -521326.3533673121) / Math.Sqrt((54155028.27083483 / (23)) + (138412542.04760572 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.29746954578885576 = (401802.3814580927 - 521326.3533673121) / 401802.3814580927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_short

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 469.31801458333337 > 391.28338573878483.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -19.444761327858274 (T) = (0 -479405.0992517889) / Math.Sqrt((18884681.90586513 / (24)) + (366665620.28488284 / (13))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (13) - 2, .025) and -0.2794150813080799 = (374706.46255134256 - 479405.0992517889) / 374706.46255134256 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_sbyte

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 651.4934026041666 > 558.5322709375.
IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -35.254654129687246 (T) = (0 -635534.3523060244) / Math.Sqrt((83728073.79736117 / (23)) + (82778439.51029661 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.20693468113071253 = (526568.9703361795 - 635534.3523060244) / 526568.9703361795 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_float_int

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 205.32056174242425 > 154.52152450642524.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -39.11661432491716 (T) = (0 -203323.26511106975) / Math.Sqrt((1612780.163656739 / (23)) + (27656334.409742378 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.37961048098142003 = (147377.29809535132 - 203323.26511106975) / 147377.29809535132 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_double

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 247.15703131510415 > 179.1910769256898.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -107.1955624776692 (T) = (0 -243799.51582895423) / Math.Sqrt((6431256.239070942 / (24)) + (2602158.735563821 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.42087072741982084 = (171584.5862147313 - 243799.51582895423) / 171584.5862147313 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_byte

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 648.3526782142858 > 545.7066565416667.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -35.24430760680096 (T) = (0 -636518.4395292425) / Math.Sqrt((89256582.1127473 / (24)) + (77379787.0311196 / (13))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (13) - 2, .025) and -0.20801409999090736 = (526913.0877975957 - 636518.4395292425) / 526913.0877975957 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_int_float

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 232.0690045248869 > 161.82599562500002.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -70.97152423174124 (T) = (0 -232863.30118897697) / Math.Sqrt((24752037.80421241 / (24)) + (2245693.002553358 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.4986130374344302 = (155385.87705578105 - 232863.30118897697) / 155385.87705578105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_uint

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 491.441796484375 > 335.47355930165816.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -44.38644739922427 (T) = (0 -477859.8409115784) / Math.Sqrt((31142991.83548839 / (24)) + (156268963.98452777 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.4878105120600195 = (321183.26698064164 - 477859.8409115784) / 321183.26698064164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added CompliationMode=wasm PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged labels Nov 8, 2022
@lewing lewing mentioned this issue Nov 8, 2022
Open
@lewing
Copy link
Member

lewing commented Nov 8, 2022

range here is dotnet/runtime@9a138bf...13d4e6d which is the same as #9628

This was referenced Nov 9, 2022
Open
Closed
@tannergooding
Copy link
Member

As per dotnet/runtime#78163 (comment), the fixes taken here are required for correctness.

These tests should effectively be ignored on Mono until the SIMD work is completed and IsHardwareAccelerated starts reporting true. The software fallbacks are not expected to be used in production scenarios and exist purely for validation purposes and to support scenarios like the debugger.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lewing @tannergooding