Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 105 Regressions on 1/13/2023 4:37:08 PM #11790

Closed
performanceautofiler bot opened this issue Jan 17, 2023 · 7 comments
Closed

[Perf] Linux/x64: 105 Regressions on 1/13/2023 4:37:08 PM #11790

performanceautofiler bot opened this issue Jan 17, 2023 · 7 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Doubles

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDoubles - Duration of single invocation 514.57 ms 615.87 ms 1.20 0.01 False
WriteDoubles - Duration of single invocation 481.38 ms 575.82 ms 1.20 0.01 False
WriteDoubles - Duration of single invocation 480.35 ms 579.65 ms 1.21 0.01 False
WriteDoubles - Duration of single invocation 516.77 ms 607.39 ms 1.18 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Doubles*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: False)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 615.8681597692307 > 546.2032953375001.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -63.57822528638588 (T) = (0 -622380354.7688208) / Math.Sqrt((26312403806020.914 / (33)) + (39377286997513.734 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.20047532198819165 = (518444938.7414752 - 622380354.7688208) / 518444938.7414752 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 575.8163878461538 > 503.5985055096154.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -55.52668952458229 (T) = (0 -584876828.8641113) / Math.Sqrt((22030719141357.797 / (33)) + (59173838307099.086 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.21542240968128276 = (481212806.51512927 - 584876828.8641113) / 481212806.51512927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 579.6458008666667 > 497.4965405925001.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -53.82662678210934 (T) = (0 -578067669.976051) / Math.Sqrt((27331991511702.957 / (33)) + (61101223757395.44 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.21954635703880807 = (474002211.2645742 - 578067669.976051) / 474002211.2645742 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 607.3901055 > 537.9534553373077.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -59.66494639057844 (T) = (0 -613296797.5577533) / Math.Sqrt((30959405915255.91 / (33)) + (41791282482531.01 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.19972502782081097 = (511197802.27620155 - 613296797.5577533) / 511197802.27620155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_Reflection_ - Duration of single invocation 6.48 ms 6.89 ms 1.06 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.89443239074074 > 6.860951825174254.
IsChangePoint: Marked as a change because one of 12/8/2022 3:49:24 AM, 12/21/2022 9:23:17 AM, 1/3/2023 9:28:36 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -21.455866021838688 (T) = (0 -6911873.656301252) / Math.Sqrt((3812839256.0262275 / (33)) + (3029041010.2068048 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.05266777480778267 = (6566054.192704209 - 6911873.656301252) / 6566054.192704209 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_Reflection_ - Duration of single invocation 6.98 ms 7.40 ms 1.06 0.00 False
SystemTextJson_SourceGen_ - Duration of single invocation 7.14 ms 7.59 ms 1.06 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.395414017316017 > 7.338300797542734.
IsChangePoint: Marked as a change because one of 12/8/2022 9:08:34 PM, 12/21/2022 9:23:17 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -20.50772411383373 (T) = (0 -7380952.875742476) / Math.Sqrt((3222059733.9036493 / (33)) + (5359696850.0217905 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.05506646478694644 = (6995723.134118323 - 7380952.875742476) / 6995723.134118323 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### MicroBenchmarks.Serializers.Json_FromStream&lt;MyEventsListerViewModel&gt;.SystemTextJson_SourceGen_

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.5927165221445225 > 7.532171772362637.
IsChangePoint: Marked as a change because one of 12/7/2022 12:36:41 PM, 12/21/2022 9:23:17 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -27.264105106639864 (T) = (0 -7559366.856102841) / Math.Sqrt((2302218077.398748 / (34)) + (2655977263.523647 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.05291895986229648 = (7179438.441389141 - 7559366.856102841) / 7179438.441389141 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Floor - Duration of single invocation 62.73 ns 177.40 ns 2.83 0.17 False
Round - Duration of single invocation 75.90 ns 192.22 ns 2.53 0.15 False
Divide - Duration of single invocation 510.78 ns 979.43 ns 1.92 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.Floor

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 177.40164467265936 > 66.66540588129249.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -106.19998473671771 (T) = (0 -181.3915888158736) / Math.Sqrt((4.32275583865001 / (34)) + (23.147049834619345 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -1.8502862338727417 = (63.639779984276586 - 181.3915888158736) / 63.639779984276586 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Round

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 192.2228718873296 > 79.73301796738347.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -166.53860595119724 (T) = (0 -194.18085845199258) / Math.Sqrt((2.7336598847004927 / (33)) + (8.847023432935561 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -1.5571810213699049 = (75.93551525263868 - 194.18085845199258) / 75.93551525263868 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Divide

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 979.4262402389465 > 545.0223648764676.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -94.10962253122975 (T) = (0 -1004.0635892564443) / Math.Sqrt((74.15276698421921 / (33)) + (514.1400900663796 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.940190991138659 = (517.5076030361215 - 1004.0635892564443) / 517.5076030361215 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CtorInvoke - Duration of single invocation 178.26 μs 203.96 μs 1.14 0.01 False
Ctor - Duration of single invocation 176.10 μs 200.41 μs 1.14 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 203.96454611013985 > 186.69571647914807.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -88.87122423263594 (T) = (0 -203760.9281283042) / Math.Sqrt((1194198.4333806904 / (33)) + (981467.855244424 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.1436382415474858 = (178169.0404586245 - 203760.9281283042) / 178169.0404586245 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 200.41448179945053 > 184.50449909234553.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/16/2022 11:20:37 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -99.29211015484401 (T) = (0 -201158.2876771209) / Math.Sqrt((1085491.508913958 / (33)) + (656018.9091117901 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.14285958045837524 = (176013.1262988939 - 201158.2876771209) / 176013.1262988939 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 1.60 μs 2.34 μs 1.46 0.05 False
ToStringWithFormat - Duration of single invocation 2.39 μs 3.38 μs 1.42 0.24 False
ToStringWithFormat - Duration of single invocation 2.34 μs 3.39 μs 1.45 0.21 False
ToStringWithFormat - Duration of single invocation 2.08 μs 2.92 μs 1.40 0.06 False
ToStringWithCultureInfo - Duration of single invocation 2.23 μs 3.26 μs 1.46 0.20 False
ToStringWithFormat - Duration of single invocation 1.56 μs 2.33 μs 1.49 0.05 False
ToString - Duration of single invocation 2.28 μs 3.27 μs 1.43 0.26 False
ToStringWithFormat - Duration of single invocation 2.24 μs 3.52 μs 1.57 0.25 False
ToString - Duration of single invocation 2.19 μs 3.16 μs 1.44 0.23 False
Parse - Duration of single invocation 1.05 μs 1.20 μs 1.14 0.04 False
ToStringWithFormat - Duration of single invocation 1.62 μs 2.40 μs 1.49 0.04 False
Parse - Duration of single invocation 1.02 μs 1.18 μs 1.15 0.03 False
ToStringWithFormat - Duration of single invocation 1.55 μs 2.19 μs 1.41 0.03 False
ToString - Duration of single invocation 1.49 μs 2.16 μs 1.45 0.05 False
ToStringWithCultureInfo - Duration of single invocation 1.49 μs 2.23 μs 1.49 0.03 False
ToStringWithCultureInfo - Duration of single invocation 2.31 μs 3.19 μs 1.38 0.24 False
TryParse - Duration of single invocation 1.03 μs 1.15 μs 1.12 0.02 False
ToStringWithFormat - Duration of single invocation 2.29 μs 3.29 μs 1.44 0.21 False
ToStringWithFormat - Duration of single invocation 1.53 μs 2.19 μs 1.43 0.06 False
ToStringWithFormat - Duration of single invocation 2.35 μs 3.30 μs 1.41 0.20 False
TryParse - Duration of single invocation 1.02 μs 1.17 μs 1.14 0.02 False
ToStringWithFormat - Duration of single invocation 2.11 μs 2.90 μs 1.37 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "E")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.340315493222892 > 1.6451133577131594.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -96.50445471172159 (T) = (0 -2402.0791947546504) / Math.Sqrt((855.978717606429 / (33)) + (1036.832109865361 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.5352759498630107 = (1564.5911700557692 - 2402.0791947546504) / 1564.5911700557692 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.378340657018921 > 2.49912122951212.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -59.81986604864159 (T) = (0 -3420.1916803116146) / Math.Sqrt((4381.2606344695805 / (34)) + (4013.087537410816 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.45529652480231914 = (2350.16824545513 - 3420.1916803116146) / 2350.16824545513 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3887423991275907 > 2.4398404757608825.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -62.67081912389819 (T) = (0 -3319.660682133462) / Math.Sqrt((5246.571002299898 / (33)) + (2649.935758481247 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.46801132704278703 = (2261.331790144087 - 3319.660682133462) / 2261.331790144087 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.924227832773823 > 2.1569761577595874.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -48.06321100029179 (T) = (0 -2971.598489645902) / Math.Sqrt((790.4343853926653 / (33)) + (6969.4069671518955 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.43905767217961905 = (2064.9613612393123 - 2971.598489645902) / 2064.9613612393123 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 3.4028235E+38, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.2553055065186753 > 2.3208847199247646.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -54.961704310680844 (T) = (0 -3198.522243678653) / Math.Sqrt((4836.213416694299 / (33)) + (4772.661265788775 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.4975286545676139 = (2135.8671394519474 - 3198.522243678653) / 2135.8671394519474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3262494533382974 > 1.6097045145221347.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -95.68880186262965 (T) = (0 -2368.2680734856613) / Math.Sqrt((609.5527820521087 / (33)) + (1169.382555924092 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.53360039658474 = (1544.253691352512 - 2368.2680734856613) / 1544.253691352512 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: -3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.2654234958104493 > 2.3252171775382218.
IsChangePoint: Marked as a change because one of 12/7/2022 12:36:41 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -50.28698126569101 (T) = (0 -3279.409010331947) / Math.Sqrt((4462.761061537548 / (33)) + (6671.608860715232 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.4844376506674461 = (2209.192827234899 - 3279.409010331947) / 2209.192827234899 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5221303531657653 > 2.421167107006209.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -47.762683755966094 (T) = (0 -3372.0551623583183) / Math.Sqrt((7398.289577623102 / (34)) + (5093.286065229668 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.43643890809344654 = (2347.510321085616 - 3372.0551623583183) / 2347.510321085616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: 3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.1555232646501894 > 2.3167488180507307.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -55.25884820648584 (T) = (0 -3203.1380158218262) / Math.Sqrt((4468.590134891561 / (34)) + (4919.042463726475 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.4922944071366575 = (2146.4517996605327 - 3203.1380158218262) / 2146.4517996605327 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.Parse(value: "-3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2015647097760982 > 1.0896643340031615.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 1/3/2023 9:28:36 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -23.983162288546065 (T) = (0 -1194.9496002118788) / Math.Sqrt((325.2609302459254 / (33)) + (738.7060343994506 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.15564953109408317 = (1034.0069095866713 - 1194.9496002118788) / 1034.0069095866713 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.402961958026852 > 1.6546129776442402.
IsChangePoint: Marked as a change because one of 12/7/2022 12:36:41 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -140.40019344810446 (T) = (0 -2397.742954700364) / Math.Sqrt((389.0946887263021 / (34)) + (475.1424839093693 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.5192546796251837 = (1578.2363463194417 - 2397.742954700364) / 1578.2363463194417 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.Parse(value: "3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1768157635546388 > 1.0663969937882225.
IsChangePoint: Marked as a change because one of 1/3/2023 9:28:36 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -33.268547984890795 (T) = (0 -1178.2667158407664) / Math.Sqrt((235.54285692958075 / (34)) + (351.3001416506906 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.15919109386910013 = (1016.4559769934019 - 1178.2667158407664) / 1016.4559769934019 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1875687939621393 > 1.619398696468143.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -105.75539979206324 (T) = (0 -2211.072966868562) / Math.Sqrt((338.19830602336486 / (33)) + (627.1365473372288 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.4345686549714686 = (1541.2806903358257 - 2211.072966868562) / 1541.2806903358257 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1618079751738306 > 1.5821147302842267.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -82.94281650051555 (T) = (0 -2194.980202133663) / Math.Sqrt((586.0365550424684 / (33)) + (1021.2012480615698 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.4448498351010258 = (1519.175314146184 - 2194.980202133663) / 1519.175314146184 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.23234616600604 > 1.5852054840979883.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -72.80966418546085 (T) = (0 -2181.6615471060345) / Math.Sqrt((807.1487722238583 / (33)) + (1275.7167545291518 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.44521968193543265 = (1509.5708800369794 - 2181.6615471060345) / 1509.5708800369794 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: -3.4028235E+38, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.1887535116888346 > 2.337363918957621.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -56.37700691147937 (T) = (0 -3281.5680344299462) / Math.Sqrt((5117.751735648225 / (34)) + (4344.354712092439 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.4810090476372095 = (2215.7650148493926 - 3281.5680344299462) / 2215.7650148493926 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.TryParse(value: "3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1505461355338462 > 1.070796634264635.
IsChangePoint: Marked as a change because one of 1/3/2023 9:28:36 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -28.47930006104355 (T) = (0 -1178.8224140619689) / Math.Sqrt((251.5801991512097 / (34)) + (430.87249009897187 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.1463272163761079 = (1028.3472268839505 - 1178.8224140619689) / 1028.3472268839505 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.2905802366926493 > 2.4612692639450975.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -55.77915600913537 (T) = (0 -3370.7840904730206) / Math.Sqrt((3705.2545187493115 / (34)) + (6009.216376927166 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.4901712994116112 = (2262.011147177484 - 3370.7840904730206) / 2262.011147177484 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1858469344080045 > 1.619362711938782.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -67.39302947718514 (T) = (0 -2213.2473904883) / Math.Sqrt((2056.3293246772255 / (33)) + (671.4877793886478 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.41979916577673015 = (1558.8453943607565 - 2213.2473904883) / 1558.8453943607565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.303181450734416 > 2.5039753916368244.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -43.47424684696269 (T) = (0 -3300.085429473698) / Math.Sqrt((6938.22458875095 / (34)) + (5405.252207032525 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.394683310337153 = (2366.189804534142 - 3300.085429473698) / 2366.189804534142 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.TryParse(value: "-3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1657787374507185 > 1.0741422041049284.
IsChangePoint: Marked as a change because one of 1/5/2023 3:38:13 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -24.61150867341268 (T) = (0 -1197.5847130299057) / Math.Sqrt((377.980368068297 / (33)) + (639.6954971009166 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.1534711653078098 = (1038.2441703346126 - 1197.5847130299057) / 1038.2441703346126 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.8950772935008557 > 2.134790889736494.
IsChangePoint: Marked as a change because one of 12/8/2022 3:49:24 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -60.870078478706716 (T) = (0 -2903.117885383896) / Math.Sqrt((1326.4406177222725 / (33)) + (3414.917736376461 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.42571215004930835 = (2036.2580800644025 - 2903.117885383896) / 2036.2580800644025 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Xml.Tests.Perf_XmlConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DateTime_ToString_Unspecified - Duration of single invocation 841.02 ns 971.23 ns 1.15 0.05 False
DateTime_ToString_RoundtripKind - Duration of single invocation 776.59 ns 943.44 ns 1.21 0.04 False
TimeSpan_ToString - Duration of single invocation 1.45 μs 2.00 μs 1.38 0.03 False
DateTime_ToString - Duration of single invocation 829.09 ns 971.22 ns 1.17 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'

Payloads

Baseline
Compare

Histogram

System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Unspecified

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 971.2264866222603 > 837.1399494011185.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -34.465208535399334 (T) = (0 -952.920002946145) / Math.Sqrt((396.1246807207159 / (34)) + (154.37479669206556 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.18716986229579094 = (802.6821040616334 - 952.920002946145) / 802.6821040616334 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_RoundtripKind

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 943.4367825118946 > 817.4135432446083.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -37.17716069864901 (T) = (0 -935.6117682530132) / Math.Sqrt((213.7070465067619 / (33)) + (228.82648381295365 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.19849389747878746 = (780.6562638501653 - 935.6117682530132) / 780.6562638501653 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.TimeSpan_ToString

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.00496530575155 > 1.5249058602450478.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -44.477593881412645 (T) = (0 -1993.7742331988359) / Math.Sqrt((363.25443846569385 / (45)) + (1201.946204414277 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.361431848578345 = (1464.4686293190548 - 1993.7742331988359) / 1464.4686293190548 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 971.2203615756822 > 874.8234296992295.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -32.16747633674908 (T) = (0 -1008.1781156384492) / Math.Sqrt((321.57786505214995 / (34)) + (307.79364147158157 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.18579452459835 = (850.2131648650826 - 1008.1781156384492) / 850.2131648650826 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromReader - Duration of single invocation 3.30 μs 3.66 μs 1.11 0.11 False
DeserializeFromUtf8Bytes - Duration of single invocation 1.80 μs 1.99 μs 1.11 0.09 False
DeserializeFromString - Duration of single invocation 4.05 μs 4.35 μs 1.07 0.08 False
DeserializeFromReader - Duration of single invocation 3.36 μs 3.58 μs 1.07 0.09 False
DeserializeFromString - Duration of single invocation 4.03 μs 4.32 μs 1.07 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromReader(Mode: Reflection)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.657490271317829 > 3.46988132964959.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -13.636665270764365 (T) = (0 -3667.467004884197) / Math.Sqrt((8101.076602291632 / (33)) + (8560.875891952272 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.10500553571558202 = (3318.958038078253 - 3667.467004884197) / 3318.958038078253 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;.DeserializeFromUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9917718731182794 > 1.883763904550333.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -21.989238365721057 (T) = (0 -2013.8194958044346) / Math.Sqrt((462.0332265835432 / (33)) + (1759.6027895153566 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.12105020203648684 = (1796.3687015498087 - 2013.8194958044346) / 1796.3687015498087 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;.DeserializeFromString(Mode: SourceGen)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.349322050678483 > 4.219362682545603.
IsChangePoint: Marked as a change because one of 12/17/2022 11:39:38 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -21.177591243684073 (T) = (0 -4484.748786611996) / Math.Sqrt((4264.001843274256 / (34)) + (6037.749263525426 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.1061411770039088 = (4054.4090391421605 - 4484.748786611996) / 4054.4090391421605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;.DeserializeFromReader(Mode: SourceGen)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5820557425949104 > 3.46306336860067.
IsChangePoint: Marked as a change because one of 12/16/2022 10:05:39 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -19.451406635561906 (T) = (0 -3637.181131840976) / Math.Sqrt((3756.0939588949964 / (33)) + (3852.396885079807 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.10157184891530352 = (3301.8101682812953 - 3637.181131840976) / 3301.8101682812953 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;.DeserializeFromString(Mode: Reflection)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.315075352794686 > 4.236864091836265.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -21.809323780310816 (T) = (0 -4511.838209902741) / Math.Sqrt((4836.309766788183 / (33)) + (5509.054249237874 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.10833350421968037 = (4070.830839927816 - 4511.838209902741) / 4070.830839927816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Append_Primitives - Duration of single invocation 26.53 μs 30.36 μs 1.14 0.01 False
Insert_Primitives - Duration of single invocation 373.67 μs 413.98 μs 1.11 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.Append_Primitives

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.358881583518617 > 28.135623005209727.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -14.948925887436879 (T) = (0 -29653.44696145758) / Math.Sqrt((46395.20913506054 / (33)) + (740979.4409945459 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.1068845240069868 = (26790.00954328132 - 29653.44696145758) / 26790.00954328132 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_StringBuilder.Insert_Primitives

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 413.97504747807017 > 395.4463920238095.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/14/2022 5:20:21 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -19.619852105078095 (T) = (0 -409069.88325053296) / Math.Sqrt((5116917.648188215 / (34)) + (48588550.99854522 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.08142057899849371 = (378270.8515029126 - 409069.88325053296) / 378270.8515029126 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Format_OneArg - Duration of single invocation 32.50 μs 36.08 μs 1.11 0.00 False
Format_OneArg - Duration of single invocation 10.60 μs 11.30 μs 1.07 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:E} - {0:F4}{0:G}{0:N} , !!", o: 3.14159)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.07657755278786 > 34.08574178941257.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -44.598118677353256 (T) = (0 -36140.38914430161) / Math.Sqrt((67501.92909254623 / (33)) + (87964.08808817089 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.10795395205139714 = (32619.03536458985 - 36140.38914430161) / 32619.03536458985 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:D5}, {0:E} - {0:F4}{0:G}{0:N}  {0:X} !!", o: 8)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.29772112503315 > 11.053582552815548.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/3/2022 2:55:34 AM, 1/5/2023 6:34:45 PM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -20.457273377298645 (T) = (0 -11431.251384477964) / Math.Sqrt((24909.501181060074 / (45)) + (10655.23318553071 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.08060826517925283 = (10578.534102348118 - 11431.251384477964) / 10578.534102348118 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetDateTime - Duration of single invocation 57.00 μs 70.86 μs 1.24 0.07 False
GetDateTimeOffset - Duration of single invocation 86.47 μs 102.70 μs 1.19 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetDateTime

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.86293189461882 > 59.721920969277015.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 1/3/2023 9:28:36 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -45.169142280768796 (T) = (0 -71406.12622519433) / Math.Sqrt((1893522.1332391622 / (33)) + (716056.4031578966 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.23658500233718655 = (57744.616092087796 - 71406.12622519433) / 57744.616092087796 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetDateTimeOffset

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 102.70410016339869 > 91.5640550751036.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 1/5/2023 3:38:13 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -35.034584092943966 (T) = (0 -102764.71345503305) / Math.Sqrt((3032270.044324202 / (33)) + (1758516.3714181662 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.1666869075152132 = (88082.51193449947 - 102764.71345503305) / 88082.51193449947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 10.33 μs 10.88 μs 1.05 0.00 False
Parse - Duration of single invocation 10.20 μs 11.00 μs 1.08 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTimeOffset.TryParse(value: "12/30/2017 3:45:22 AM -08:00")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.883160861797087 > 10.658183456129414.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 1/3/2023 11:24:32 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -26.081182953501024 (T) = (0 -10979.536604305524) / Math.Sqrt((9541.363313657428 / (33)) + (12125.081276591674 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.07518198882128044 = (10211.79364838725 - 10979.536604305524) / 10211.79364838725 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTimeOffset.Parse(value: "12/30/2017 3:45:22 AM -08:00")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.997321262265704 > 10.703284341279238.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 1/3/2023 11:24:32 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -26.911521376746734 (T) = (0 -10938.692037225946) / Math.Sqrt((9628.331046638134 / (34)) + (10302.312733079232 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.07346262377597644 = (10190.1005167263 - 10938.692037225946) / 10190.1005167263 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseDouble - Duration of single invocation 973.98 ns 1.15 μs 1.18 0.04 False
TryParseSingle - Duration of single invocation 827.55 ns 1.01 μs 1.22 0.06 False
TryParseSingle - Duration of single invocation 833.73 ns 992.00 ns 1.19 0.08 False
TryParseDouble - Duration of single invocation 982.20 ns 1.14 μs 1.16 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: 1.7976931348623157e+308)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1514183806538842 > 1021.6476191588872.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 11:24:47 AM, 12/17/2022 10:44:50 PM, 1/5/2023 6:34:45 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -24.185612613117403 (T) = (0 -1145.1016270104337) / Math.Sqrt((884.9466050177157 / (33)) + (242.3511042801734 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.15049442798931226 = (995.3126231229965 - 1145.1016270104337) / 995.3126231229965 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0106807346376048 > 872.1290358844491.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 11:24:47 AM, 12/17/2022 10:44:50 PM, 1/5/2023 6:34:45 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -25.089367609557343 (T) = (0 -1010.24878485752) / Math.Sqrt((604.664351713938 / (33)) + (477.3163818946162 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.18923369064741716 = (849.4955977134671 - 1010.24878485752) / 849.4955977134671 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: -3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 991.998032740285 > 903.2993338062081.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 11:24:47 AM, 12/17/2022 10:44:50 PM, 1/5/2023 6:34:45 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -21.961296116985807 (T) = (0 -1010.5117357946613) / Math.Sqrt((928.6434233191981 / (34)) + (440.65457629405466 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.17790353583595517 = (857.8900606470322 - 1010.5117357946613) / 857.8900606470322 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: -1.7976931348623157e+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1365373757779258 > 1040.527815505652.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 11:24:47 AM, 12/17/2022 10:44:50 PM, 1/5/2023 6:34:45 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -23.74415728170744 (T) = (0 -1147.8169880550233) / Math.Sqrt((758.3448918650896 / (34)) + (319.89465193730337 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.14513514992829005 = (1002.3419402739504 - 1147.8169880550233) / 1002.3419402739504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Globalization.Tests.Perf_NumberCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 1.72 μs 2.49 μs 1.45 0.03 False
ToString - Duration of single invocation 1.73 μs 2.51 μs 1.45 0.04 False
ToString - Duration of single invocation 1.73 μs 2.44 μs 1.41 0.04 False
ToString - Duration of single invocation 1.71 μs 2.50 μs 1.46 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_NumberCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: )

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4863074617922774 > 1.7997876419585237.
IsChangePoint: Marked as a change because one of 12/8/2022 3:49:24 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -101.94912253680597 (T) = (0 -2467.9693027442227) / Math.Sqrt((538.2268290048355 / (33)) + (747.25853612637 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.4236456161779725 = (1733.5559318265744 - 2467.9693027442227) / 1733.5559318265744 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: ja)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.510184626015211 > 1.8194716471714152.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -86.6228207275651 (T) = (0 -2460.1517230515906) / Math.Sqrt((844.8764156940675 / (33)) + (939.5850991014115 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.4190736104280369 = (1733.6322125739002 - 2460.1517230515906) / 1733.6322125739002 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: da)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4448904987129985 > 1.817666692492967.
IsChangePoint: Marked as a change because one of 12/7/2022 6:04:36 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -65.4579348636394 (T) = (0 -2456.4830985717695) / Math.Sqrt((854.9055733678093 / (33)) + (1998.8918532621985 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.4148834562549499 = (1736.1734549316352 - 2456.4830985717695) / 1736.1734549316352 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: fr)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.495673533225284 > 1.8323297960136156.
IsChangePoint: Marked as a change because one of 12/7/2022 8:33:47 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -81.25093640825834 (T) = (0 -2448.408435662744) / Math.Sqrt((606.4659998388091 / (34)) + (1218.5563803818445 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.4065537028148354 = (1740.714507212145 - 2448.408435662744) / 1740.714507212145 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToDateTime_String - Duration of single invocation 6.95 μs 7.38 μs 1.06 0.01 False
ToDateTime_String - Duration of single invocation 2.73 μs 3.23 μs 1.18 0.14 False
ToDateTime_String - Duration of single invocation 6.63 μs 7.35 μs 1.11 0.01 False
ToDateTime_String - Duration of single invocation 4.43 μs 4.78 μs 1.08 0.03 False
ChangeType - Duration of single invocation 301.27 ns 423.14 ns 1.40 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ToDateTime_String(value: "12/12/1999 11:59:59 PM")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.375679000559828 > 7.1486622891321.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 1/3/2023 9:28:36 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -19.721234336302807 (T) = (0 -7492.724330020423) / Math.Sqrt((7967.836831771656 / (34)) + (16756.440932421916 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.0923771175785648 = (6859.100405388654 - 7492.724330020423) / 6859.100405388654 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "12/12/1999")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.2281732359221045 > 2.832948380120819.
IsChangePoint: Marked as a change because one of 11/15/2022 10:02:59 PM, 1/3/2023 3:01:43 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -26.897161104839487 (T) = (0 -3261.0873265190467) / Math.Sqrt((5534.346205809127 / (33)) + (5570.26805318796 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.20717496891766504 = (2701.4205980785773 - 3261.0873265190467) / 2701.4205980785773 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "Thursday, February 26, 2009")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.350101665205958 > 6.922613871493697.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 1/3/2023 9:28:36 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -25.027529280792216 (T) = (0 -7226.539762940959) / Math.Sqrt((7401.238695063486 / (33)) + (7834.497814103689 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.09247259240374062 = (6614.847652187394 - 7226.539762940959) / 6614.847652187394 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "February 26, 2009")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.775803894979805 > 4.477474383861492.
IsChangePoint: Marked as a change because one of 11/9/2022 9:25:09 AM, 1/3/2023 9:28:36 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -26.94305031616901 (T) = (0 -4890.2729953089365) / Math.Sqrt((6159.67546772585 / (33)) + (6317.572349579523 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.13849324110752168 = (4295.390450058095 - 4890.2729953089365) / 4295.390450058095 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ChangeType

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 423.1444379713029 > 315.9651806462595.
IsChangePoint: Marked as a change because one of 12/1/2022 11:48:13 PM, 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -37.759940449676584 (T) = (0 -431.08543770294847) / Math.Sqrt((43.2105718087722 / (45)) + (99.66989371671062 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.43650921108907387 = (300.09235887608804 - 431.08543770294847) / 300.09235887608804 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseO - Duration of single invocation 577.68 ns 720.97 ns 1.25 0.12 False
ParseR - Duration of single invocation 622.53 ns 735.50 ns 1.18 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTime.ParseO

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 720.9728898990369 > 605.9179161467429.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -37.73778692789979 (T) = (0 -732.0938885304172) / Math.Sqrt((278.43534452364145 / (34)) + (150.77023649416213 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.253263173529643 = (584.150164141962 - 732.0938885304172) / 584.150164141962 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTime.ParseR

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 735.5015395742922 > 618.1407929540762.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -50.96108309771596 (T) = (0 -744.2267586935426) / Math.Sqrt((145.3501901902806 / (34)) + (93.46957851281725 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.25356278872349775 = (593.6892554471789 - 744.2267586935426) / 593.6892554471789 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 5.90 ms 6.65 ms 1.13 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.XmlSerializer_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.645512308880308 > 6.182010172479675.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/8/2022 9:50:09 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -18.481392209442433 (T) = (0 -6457721.734604342) / Math.Sqrt((3846376274.4833603 / (33)) + (14980840422.57446 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.08985558982363824 = (5925300.3745655315 - 6457721.734604342) / 5925300.3745655315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Where01LinqQueryX - Duration of single invocation 3.47 secs 5.49 secs 1.58 0.17 False
Where01ForX - Duration of single invocation 3.32 secs 5.35 secs 1.61 0.19 False
Where01LinqMethodNestedX - Duration of single invocation 4.32 secs 6.35 secs 1.47 0.15 False
Where01LinqMethodX - Duration of single invocation 3.48 secs 5.60 secs 1.61 0.19 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

Histogram

LinqBenchmarks.Where01LinqQueryX

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.4946738384666665 > 3.6502130646288466.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -79.10159948331534 (T) = (0 -5645675192.2515) / Math.Sqrt((5137121997528366 / (34)) + (12826606483960404 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.6306162854939971 = (3462295355.7348633 - 5645675192.2515) / 3462295355.7348633 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01ForX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.346267377933334 > 3.4775568290076926.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -75.62442234043604 (T) = (0 -5435699323.824752) / Math.Sqrt((13613588950318956 / (33)) + (8601245270864738 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.6636400565182846 = (3267352996.5374513 - 5435699323.824752) / 3267352996.5374513 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01LinqMethodNestedX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.3457501852857146 > 4.264203843011538.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -93.84250470478794 (T) = (0 -6262327723.475581) / Math.Sqrt((9960942754969272 / (33)) + (5196262404395134 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.541324847465161 = (4062951255.067683 - 6262327723.475581) / 4062951255.067683 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01LinqMethodX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.602590774 > 3.6502452946211545.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -62.43093293999073 (T) = (0 -5690194825.9569235) / Math.Sqrt((23660185813061200 / (33)) + (11180488541782820 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.633481899799326 = (3483475896.88993 - 5690194825.9569235) / 3483475896.88993 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringHebrewIsrael - Duration of single invocation 4.42 μs 5.90 μs 1.34 0.05 False
Parse - Duration of single invocation 6.84 μs 7.55 μs 1.10 0.01 False
Parse - Duration of single invocation 6.84 μs 7.61 μs 1.11 0.01 False
Parse - Duration of single invocation 6.88 μs 7.57 μs 1.10 0.01 False
Parse - Duration of single invocation 6.79 μs 7.56 μs 1.11 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.903529176082863 > 4.73820778873615.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -49.70073175410676 (T) = (0 -5794.29141905619) / Math.Sqrt((6257.5026064316635 / (33)) + (11011.306687894894 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.29734236966656363 = (4466.277795695063 - 5794.29141905619) / 4466.277795695063 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: fr)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.551957796089836 > 7.191165026399045.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 1/3/2023 9:28:36 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -28.725556560551073 (T) = (0 -7567.181802132175) / Math.Sqrt((12326.365568266034 / (33)) + (4310.11708346636 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.10050286183078895 = (6876.112788605987 - 7567.181802132175) / 6876.112788605987 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: )

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.605570214018842 > 7.186218762932612.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 1/3/2023 3:01:43 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -25.792607885204855 (T) = (0 -7519.306628209537) / Math.Sqrt((13246.951220854997 / (33)) + (4754.120690728978 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.09402862254541618 = (6873.043788118432 - 7519.306628209537) / 6873.043788118432 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: ja)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.565254097392012 > 7.227791615335746.
IsChangePoint: Marked as a change because one of 11/15/2022 2:18:08 PM, 1/3/2023 9:28:36 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -34.27846679852832 (T) = (0 -7554.953654548108) / Math.Sqrt((6948.10237039154 / (34)) + (3314.6135832581576 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.09451061851309116 = (6902.585983872522 - 7554.953654548108) / 6902.585983872522 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: da)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.559788761401782 > 7.13311003334121.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 1/3/2023 3:01:43 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -25.508941304219725 (T) = (0 -7575.344779490956) / Math.Sqrt((11555.760948471192 / (33)) + (7629.758666360346 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.0988369120273477 = (6893.966426295681 - 7575.344779490956) / 6893.966426295681 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 2.00 μs 2.17 μs 1.08 0.02 False
ToStringD - Duration of single invocation 382.40 ns 498.18 ns 1.30 0.16 False
Parse - Duration of single invocation 1.36 μs 1.61 μs 1.19 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Parse(numberString: -2147483648)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1700604499206575 > 2.1513989463920415.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/18/2022 12:43:31 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -20.4905555861702 (T) = (0 -2239.0128839009767) / Math.Sqrt((895.4422407155911 / (34)) + (1592.6837566542235 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.10193753689463023 = (2031.8872975420531 - 2239.0128839009767) / 2031.8872975420531 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 123)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 498.17577932351344 > 401.3595028304053.
IsChangePoint: Marked as a change because one of 12/1/2022 11:48:13 PM, 12/8/2022 9:11:55 AM, 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -37.34585350455427 (T) = (0 -507.9045087649082) / Math.Sqrt((103.4166436309522 / (45)) + (91.99922827323566 / (9))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (9) - 2, .025) and -0.3516730280352402 = (375.75989032139387 - 507.9045087649082) / 375.75989032139387 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 123)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6073081051759945 > 1.428053739741653.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -30.644516651077247 (T) = (0 -1561.2221449589672) / Math.Sqrt((343.9501513887239 / (33)) + (753.639280169571 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.1541687123687663 = (1352.6810493370435 - 1561.2221449589672) / 1352.6810493370435 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithCultureInfo - Duration of single invocation 2.91 μs 3.61 μs 1.24 0.15 False
ToStringWithCultureInfo - Duration of single invocation 1.54 μs 2.14 μs 1.39 0.05 False
ToString - Duration of single invocation 2.85 μs 3.77 μs 1.33 0.13 False
ToStringWithFormat - Duration of single invocation 1.57 μs 2.20 μs 1.41 0.05 False
ToStringWithFormat - Duration of single invocation 2.91 μs 3.78 μs 1.30 0.11 False
ToStringWithFormat - Duration of single invocation 1.54 μs 2.19 μs 1.43 0.03 False
ToStringWithFormat - Duration of single invocation 4.64 μs 5.11 μs 1.10 0.04 False
ToStringWithCultureInfo - Duration of single invocation 2.91 μs 3.75 μs 1.29 0.15 False
ToStringWithFormat - Duration of single invocation 3.01 μs 3.92 μs 1.30 0.14 False
Parse - Duration of single invocation 1.31 μs 1.50 μs 1.14 0.02 False
ToStringWithFormat - Duration of single invocation 1.58 μs 2.37 μs 1.50 0.03 False
TryParse - Duration of single invocation 1.33 μs 1.46 μs 1.09 0.02 False
ToStringWithFormat - Duration of single invocation 1.52 μs 2.23 μs 1.47 0.04 False
ToString - Duration of single invocation 1.49 μs 2.14 μs 1.44 0.05 False
TryParse - Duration of single invocation 1.31 μs 1.47 μs 1.12 0.04 False
ToStringWithFormat - Duration of single invocation 2.10 μs 2.76 μs 1.31 0.04 False
ToString - Duration of single invocation 2.86 μs 3.59 μs 1.26 0.17 False
ToStringWithFormat - Duration of single invocation 2.91 μs 3.81 μs 1.31 0.14 False
ToStringWithFormat - Duration of single invocation 3.06 μs 4.01 μs 1.31 0.13 False
Parse - Duration of single invocation 1.33 μs 1.51 μs 1.13 0.04 False
ToStringWithFormat - Duration of single invocation 3.12 μs 3.83 μs 1.23 0.16 False
ToStringWithFormat - Duration of single invocation 1.54 μs 2.19 μs 1.42 0.02 False
ToStringWithFormat - Duration of single invocation 2.04 μs 2.72 μs 1.33 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithCultureInfo(value: 1.7976931348623157E+308, culture: zh)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6149603525930005 > 2.987887805475598.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -45.81655135145215 (T) = (0 -3751.757427951718) / Math.Sqrt((5106.810555104612 / (33)) + (5389.740990869273 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.3292539603608313 = (2822.4534512075393 - 3751.757427951718) / 2822.4534512075393 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.135871976226887 > 1.5721136465838146.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -69.85073144709118 (T) = (0 -2187.3746821582354) / Math.Sqrt((1024.1650530149473 / (33)) + (1307.1513117780244 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.4459656824391104 = (1512.7431506316857 - 2187.3746821582354) / 1512.7431506316857 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: -1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7721155087022775 > 3.0557630791715162.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -51.15023737943894 (T) = (0 -3794.7743897799046) / Math.Sqrt((6075.907982865658 / (33)) + (2517.3123022755017 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.30721154184968985 = (2902.9535528812276 - 3794.7743897799046) / 2902.9535528812276 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2015154376584847 > 1.6448122553613997.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -82.54043825293827 (T) = (0 -2241.9666979904473) / Math.Sqrt((990.834266815539 / (33)) + (757.6698941444813 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.4272108137072579 = (1570.8728356442427 - 2241.9666979904473) / 1570.8728356442427 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7818317656565656 > 3.1209261564705013.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -41.083870267078105 (T) = (0 -3815.9169878919306) / Math.Sqrt((4957.61764431303 / (33)) + (6348.378291346426 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.29707259770971917 = (2941.945573925324 - 3815.9169878919306) / 2941.945573925324 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.193159313277035 > 1.609860837686602.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -103.2804840003917 (T) = (0 -2224.3602386950683) / Math.Sqrt((507.87197018056764 / (33)) + (584.4212464781756 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.4393638471799427 = (1545.377315855974 - 2224.3602386950683) / 1545.377315855974 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "F50")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.1094402222586295 > 4.809588938366102.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -18.602758844327234 (T) = (0 -5024.244388939323) / Math.Sqrt((7321.267422368998 / (34)) + (5682.301207040437 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.08887191031530908 = (4614.173936661137 - 5024.244388939323) / 4614.173936661137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: -1.7976931348623157E+308, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7472852884918755 > 3.0762540639555933.
IsChangePoint: Marked as a change because one of 12/7/2022 6:04:36 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -45.94101913994282 (T) = (0 -3770.2702999167223) / Math.Sqrt((6228.853660335278 / (33)) + (3918.9075641776644 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.3090352126441905 = (2880.190130486216 - 3770.2702999167223) / 2880.190130486216 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.918347478567827 > 3.209885531745207.
IsChangePoint: Marked as a change because one of 12/8/2022 3:49:24 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -53.0729089681851 (T) = (0 -3932.6936129603732) / Math.Sqrt((4896.754674947991 / (34)) + (2660.7774734924137 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.2854270303832474 = (3059.445242712726 - 3932.6936129603732) / 3059.445242712726 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.498631624959897 > 1.3808123708105726.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 1/3/2023 3:01:43 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -26.53216778634138 (T) = (0 -1481.4414175168408) / Math.Sqrt((422.18926989810757 / (33)) + (473.6009941724674 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.11916624467100717 = (1323.7009466384764 - 1481.4414175168408) / 1323.7009466384764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.368030186033544 > 1.6421365284347598.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -89.59020357634165 (T) = (0 -2384.0637254357985) / Math.Sqrt((753.563945866336 / (34)) + (1298.819288722853 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.5254137893229702 = (1562.8964036662642 - 2384.0637254357985) / 1562.8964036662642 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4573057938299474 > 1.3806190395708273.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 1/3/2023 3:01:43 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -30.11973449547766 (T) = (0 -1487.3631040276061) / Math.Sqrt((372.11433029312366 / (34)) + (361.98684082456367 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.12045179514451636 = (1327.4672863867074 - 1487.3631040276061) / 1327.4672863867074 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2271800540726017 > 1.6111073522759285.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -84.47112101214985 (T) = (0 -2222.731412603207) / Math.Sqrt((611.9882983976509 / (33)) + (908.7761195729029 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.4261346387270351 = (1558.5705250012106 - 2222.731412603207) / 1558.5705250012106 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.144432951805859 > 1.5902960421271926.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -60.84208224838863 (T) = (0 -2196.9056618466166) / Math.Sqrt((685.4957780908849 / (34)) + (2184.097392451984 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.4463386336518269 = (1518.9428054616092 - 2196.9056618466166) / 1518.9428054616092 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4712319086980272 > 1.3938742849595345.
IsChangePoint: Marked as a change because one of 1/4/2023 9:39:29 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -27.37905368702369 (T) = (0 -1499.6181369022343) / Math.Sqrt((608.726502938443 / (33)) + (307.9052382397749 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.11738414117958461 = (1342.079309734196 - 1499.6181369022343) / 1342.079309734196 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.7578162927728607 > 2.174213566592773.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -56.72724757956534 (T) = (0 -2812.4109801128434) / Math.Sqrt((631.0596667693615 / (33)) + (3051.881229881824 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.34891037386397145 = (2084.950219529156 - 2812.4109801128434) / 2084.950219529156 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: 1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.59273352064602 > 3.000120486725063.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -42.1296853176546 (T) = (0 -3709.5183547621896) / Math.Sqrt((5311.15121672587 / (34)) + (5712.969525944709 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.30724022501942705 = (2837.6715187960667 - 3709.5183547621896) / 2837.6715187960667 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.810863294684193 > 3.1744476393175436.
IsChangePoint: Marked as a change because one of 12/7/2022 8:33:47 PM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -41.21058243450495 (T) = (0 -3884.349129079365) / Math.Sqrt((6829.782981345381 / (34)) + (5462.821027905113 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.2949943383758915 = (2999.510510564004 - 3884.349129079365) / 2999.510510564004 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.008803502399932 > 3.160699351779334.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -52.395056442351695 (T) = (0 -3962.5657755185625) / Math.Sqrt((5638.53309194115 / (33)) + (3715.6792622571047 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.3273027977256303 = (2985.427125075399 - 3962.5657755185625) / 2985.427125075399 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5079188377971904 > 1.3844477110473867.
IsChangePoint: Marked as a change because one of 1/3/2023 9:28:36 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -26.696897938371908 (T) = (0 -1507.3851166346274) / Math.Sqrt((364.4139856242867 / (33)) + (650.9636035438241 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.12973267751892087 = (1334.2847796038736 - 1507.3851166346274) / 1334.2847796038736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.828441133365901 > 3.1986658980213836.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -44.71448534942195 (T) = (0 -3880.892097402836) / Math.Sqrt((4104.681774510975 / (33)) + (4957.033255266395 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.27998446995009485 = (3031.9837377043705 - 3880.892097402836) / 3031.9837377043705 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1893624608022093 > 1.6156571596609903.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -106.0711055923929 (T) = (0 -2212.4635021689764) / Math.Sqrt((482.99309353556544 / (33)) + (538.1809698930407 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.4372224573736033 = (1539.4022622023715 - 2212.4635021689764) / 1539.4022622023715 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.7206259764817107 > 2.150103097009645.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -68.0945180084486 (T) = (0 -2753.785252154162) / Math.Sqrt((1179.2565013935775 / (33)) + (1377.1121970303464 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.3313676748060616 = (2068.3882478634628 - 2753.785252154162) / 2068.3882478634628 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 414018530f0788c418d92ff817bb5fd6932324f6
Compare 32cb6bb23d58c00975a2d80d62fe90998e780772
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation 6.43 ms 6.77 ms 1.05 0.01 False
DeserializeFromString - Duration of single invocation 6.52 ms 6.99 ms 1.07 0.00 False
DeserializeFromUtf8Bytes - Duration of single invocation 6.18 ms 6.57 ms 1.06 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.768963607068606 > 6.748420194071162.
IsChangePoint: Marked as a change because one of 12/10/2022 12:36:40 AM, 12/21/2022 9:23:17 AM, 1/3/2023 9:28:36 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -27.755469600764897 (T) = (0 -6782154.620623343) / Math.Sqrt((1846638630.35874 / (33)) + (2370012946.0385 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.05615876572857346 = (6421529.452482257 - 6782154.620623343) / 6421529.452482257 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;MyEventsListerViewModel&gt;.DeserializeFromString(Mode: Reflection)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.985613515873016 > 6.8827931590127625.
IsChangePoint: Marked as a change because one of 12/10/2022 5:39:01 AM, 12/21/2022 9:23:17 AM, 1/3/2023 6:29:42 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -20.787322612337906 (T) = (0 -6907647.535586787) / Math.Sqrt((3463946559.2175627 / (33)) + (3080720106.5285134 / (21))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (21) - 2, .025) and -0.05013347167458564 = (6577875.786181323 - 6907647.535586787) / 6577875.786181323 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;MyEventsListerViewModel&gt;.DeserializeFromUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.56795272863248 > 6.5474932589833.
IsChangePoint: Marked as a change because one of 12/8/2022 3:49:24 AM, 12/21/2022 9:23:17 AM, 1/13/2023 11:01:49 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -22.75920061070816 (T) = (0 -6579500.627593135) / Math.Sqrt((3143540766.694672 / (34)) + (2630220050.4764104 / (21))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (21) - 2, .025) and -0.05378366441483597 = (6243691.992745702 - 6579500.627593135) / 6243691.992745702 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added LLVM=false PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged labels Jan 17, 2023
@kotlarmilos kotlarmilos mentioned this issue Jan 18, 2023
Closed
@kotlarmilos
Copy link
Member

Could be dotnet/runtime#79461. Similar to #11831.

@kotlarmilos kotlarmilos mentioned this issue Jan 18, 2023
Closed
@kotlarmilos
Copy link
Member

Mitigated by dotnet/runtime#82093.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kotlarmilos