[Perf] Windows/x64: 2 Regressions on 5/3/2023 1:06:54 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	442141d696ee7d4ae54420974559e99738d753e6
Compare	c8f43d52f699c1da289a5cb36d371aec15901ab8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
MeasureReverseEndianness - Duration of single invocation	614.26 ns	941.83 ns	1.53	0.00	True
MeasureReverseUsingNtoH - Duration of single invocation	605.61 ns	946.10 ns	1.56	0.01	True	6023.187212732387	8077.543601682801	1.3410746364661752)	Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.MeasureReverseEndianness

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 941.8282251056002 > 645.1990668248882.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -172.69616839909932 (T) = (0 -942.899495537063) / Math.Sqrt((71.28264791595957 / (21)) + (0.9109916109751146 / (13))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (13) - 2, .025) and -0.5172382205961809 = (621.4577794952738 - 942.899495537063) / 621.4577794952738 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Compare

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.MeasureReverseUsingNtoH

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 946.1020211179717 > 643.259435520526.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -145.33095665139552 (T) = (0 -943.5256872067196) / Math.Sqrt((100.76093042903165 / (21)) + (1.9921542971535473 / (13))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (13) - 2, .025) and -0.5214730941818898 = (620.1395810512588 - 943.5256872067196) / 620.1395810512588 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[DrewScoggins]

@EgorBo @jakobbotsch This looks like a regression from dotnet/runtime#85559

[jakobbotsch]

Yep, dotnet/runtime#85988 (comment) has an analysis.

[DrewScoggins]

Sorry, I tried to track something down, but couldn't find it. Thanks!