Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Regressions in PerfLabTests.CastingPerf #79305

Closed
performanceautofiler bot opened this issue Dec 6, 2022 · 3 comments
Closed

Regressions in PerfLabTests.CastingPerf #79305

performanceautofiler bot opened this issue Dec 6, 2022 · 3 comments
Assignees
@EgorBo
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Milestone
8.0.0

Comments

@performanceautofiler
Copy link

Run Information

Architecture x86
OS Windows 10.0.18362
Baseline 02c144afac6efe0f716e64ea0a701e2eec995866
Compare 4ce23f46ed20683a1b6649e9a43e09459007c9d8
Diff Diff

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ObjInt - Duration of single invocation 155.93 μs 236.75 μs 1.52 0.01 False 1011338.2352941176 911208.3333333333 0.9009926664824804 Trace Trace
ObjScalarValueType - Duration of single invocation 156.57 μs 221.40 μs 1.41 0.01 False Trace Trace
ObjFooIsObj - Duration of single invocation 155.93 μs 219.89 μs 1.41 0.01 False Trace Trace
FooObjIsNull - Duration of single invocation 155.92 μs 235.46 μs 1.51 0.01 False Trace Trace
ObjObjrefValueType - Duration of single invocation 155.93 μs 220.45 μs 1.41 0.01 False 1011541.6666666665 911375 0.9009762326481857 Trace Trace

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf.ObjInt

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 236.75046717171713 > 163.72450742574262.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -70.45756980147726 (T) = (0 -230173.21543620728) / Math.Sqrt((60155.2292099663 / (43)) + (8847041.346048357 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.47516110639698356 = (156032.59497425015 - 230173.21543620728) / 156032.59497425015 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; PerfLabTests.CastingPerf.ObjInt()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0A76F8A0]
       test      ecx,ecx
       jle       short M00_L01
M00_L00:
       mov       esi,ds:[6EE4138]
       mov       edx,6EE4158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 40

Compare Jit Disasm

; PerfLabTests.CastingPerf.ObjInt()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0AE82918]
       test      ecx,ecx
       jle       short M00_L01
       mov       esi,ds:[7104138]
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
M00_L00:
       mov       edx,7104158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 50

PerfLabTests.CastingPerf.ObjScalarValueType

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 221.39642939814814 > 163.7248382996954.
IsChangePoint: Marked as a change because one of 11/13/2022 8:13:15 AM, 11/13/2022 11:44:30 PM, 12/3/2022 7:24:45 AM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -84.21816907425399 (T) = (0 -226724.70643478556) / Math.Sqrt((91621.1101554736 / (43)) + (5607293.229763678 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.45234151903737774 = (156109.77408747518 - 226724.70643478556) / 156109.77408747518 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; PerfLabTests.CastingPerf.ObjScalarValueType()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0A36F8A0]
       test      ecx,ecx
       jle       short M00_L01
M00_L00:
       mov       esi,ds:[6924150]
       mov       edx,6924158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 40

Compare Jit Disasm

; PerfLabTests.CastingPerf.ObjScalarValueType()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0B0D2918]
       test      ecx,ecx
       jle       short M00_L01
       mov       esi,ds:[7284150]
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
M00_L00:
       mov       edx,7284158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 50

PerfLabTests.CastingPerf.ObjFooIsObj

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.88900905432595 > 163.72572633044555.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -63.61631832795094 (T) = (0 -226552.94826143753) / Math.Sqrt((51162.83030810417 / (43)) + (9825638.824581709 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.45210982198774213 = (156016.40098495936 - 226552.94826143753) / 156016.40098495936 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; PerfLabTests.CastingPerf.ObjFooIsObj()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0AD3F8A0]
       test      ecx,ecx
       jle       short M00_L01
M00_L00:
       mov       esi,ds:[7374140]
       mov       edx,7374158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 40

Compare Jit Disasm

; PerfLabTests.CastingPerf.ObjFooIsObj()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0B532918]
       test      ecx,ecx
       jle       short M00_L01
       mov       esi,ds:[7934140]
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
M00_L00:
       mov       edx,7934158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 50

PerfLabTests.CastingPerf.FooObjIsNull

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 235.4583084577114 > 163.72059560643564.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -73.6443613694094 (T) = (0 -229436.471536146) / Math.Sqrt((85374.47803562158 / (43)) + (7916438.017468861 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.46976413269129824 = (156104.27988606773 - 229436.471536146) / 156104.27988606773 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; PerfLabTests.CastingPerf.FooObjIsNull()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0A94F8A0]
       test      ecx,ecx
       jle       short M00_L01
M00_L00:
       mov       esi,ds:[7094148]
       mov       edx,7094158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 40

Compare Jit Disasm

; PerfLabTests.CastingPerf.FooObjIsNull()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0B512918]
       test      ecx,ecx
       jle       short M00_L01
       mov       esi,ds:[7804148]
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
M00_L00:
       mov       edx,7804158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 50

PerfLabTests.CastingPerf.ObjObjrefValueType

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 220.44800920910075 > 163.72261014851486.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -65.81211461747941 (T) = (0 -227663.18376580504) / Math.Sqrt((73433.08585642751 / (43)) + (9455613.775383344 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.45879811064601056 = (156062.15973572055 - 227663.18376580504) / 156062.15973572055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; PerfLabTests.CastingPerf.ObjObjrefValueType()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0B05F8A0]
       test      ecx,ecx
       jle       short M00_L01
M00_L00:
       mov       esi,ds:[77F4154]
       mov       edx,77F4158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 40

Compare Jit Disasm

; PerfLabTests.CastingPerf.ObjObjrefValueType()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0AE52918]
       test      ecx,ecx
       jle       short M00_L01
       mov       esi,ds:[6F84154]
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
M00_L00:
       mov       edx,6F84158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 50

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler performanceautofiler bot added CoreClr untriaged New issue has not been triaged by the area owner labels Dec 6, 2022
@LoopedBard3
Copy link
Member

This seems likely related to #79060. @EgorBo I am not familiar with the changes made, could they have bubbled through to these tests?

@LoopedBard3 LoopedBard3 transferred this issue from dotnet/perf-autofiling-issues Dec 6, 2022
@dotnet-issue-labeler dotnet-issue-labeler bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Dec 6, 2022
@ghost
Copy link

ghost commented Dec 6, 2022

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture x86
OS Windows 10.0.18362
Baseline 02c144afac6efe0f716e64ea0a701e2eec995866
Compare 4ce23f46ed20683a1b6649e9a43e09459007c9d8
Diff Diff

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ObjInt - Duration of single invocation 155.93 μs 236.75 μs 1.52 0.01 False 1011338.2352941176 911208.3333333333 0.9009926664824804 Trace Trace
ObjScalarValueType - Duration of single invocation 156.57 μs 221.40 μs 1.41 0.01 False Trace Trace
ObjFooIsObj - Duration of single invocation 155.93 μs 219.89 μs 1.41 0.01 False Trace Trace
FooObjIsNull - Duration of single invocation 155.92 μs 235.46 μs 1.51 0.01 False Trace Trace
ObjObjrefValueType - Duration of single invocation 155.93 μs 220.45 μs 1.41 0.01 False 1011541.6666666665 911375 0.9009762326481857 Trace Trace

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf.ObjInt

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 236.75046717171713 > 163.72450742574262.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -70.45756980147726 (T) = (0 -230173.21543620728) / Math.Sqrt((60155.2292099663 / (43)) + (8847041.346048357 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.47516110639698356 = (156032.59497425015 - 230173.21543620728) / 156032.59497425015 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; PerfLabTests.CastingPerf.ObjInt()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0A76F8A0]
       test      ecx,ecx
       jle       short M00_L01
M00_L00:
       mov       esi,ds:[6EE4138]
       mov       edx,6EE4158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 40

Compare Jit Disasm

; PerfLabTests.CastingPerf.ObjInt()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0AE82918]
       test      ecx,ecx
       jle       short M00_L01
       mov       esi,ds:[7104138]
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
M00_L00:
       mov       edx,7104158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 50

PerfLabTests.CastingPerf.ObjScalarValueType

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 221.39642939814814 > 163.7248382996954.
IsChangePoint: Marked as a change because one of 11/13/2022 8:13:15 AM, 11/13/2022 11:44:30 PM, 12/3/2022 7:24:45 AM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -84.21816907425399 (T) = (0 -226724.70643478556) / Math.Sqrt((91621.1101554736 / (43)) + (5607293.229763678 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.45234151903737774 = (156109.77408747518 - 226724.70643478556) / 156109.77408747518 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; PerfLabTests.CastingPerf.ObjScalarValueType()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0A36F8A0]
       test      ecx,ecx
       jle       short M00_L01
M00_L00:
       mov       esi,ds:[6924150]
       mov       edx,6924158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 40

Compare Jit Disasm

; PerfLabTests.CastingPerf.ObjScalarValueType()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0B0D2918]
       test      ecx,ecx
       jle       short M00_L01
       mov       esi,ds:[7284150]
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
M00_L00:
       mov       edx,7284158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 50

PerfLabTests.CastingPerf.ObjFooIsObj

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.88900905432595 > 163.72572633044555.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -63.61631832795094 (T) = (0 -226552.94826143753) / Math.Sqrt((51162.83030810417 / (43)) + (9825638.824581709 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.45210982198774213 = (156016.40098495936 - 226552.94826143753) / 156016.40098495936 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; PerfLabTests.CastingPerf.ObjFooIsObj()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0AD3F8A0]
       test      ecx,ecx
       jle       short M00_L01
M00_L00:
       mov       esi,ds:[7374140]
       mov       edx,7374158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 40

Compare Jit Disasm

; PerfLabTests.CastingPerf.ObjFooIsObj()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0B532918]
       test      ecx,ecx
       jle       short M00_L01
       mov       esi,ds:[7934140]
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
M00_L00:
       mov       edx,7934158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 50

PerfLabTests.CastingPerf.FooObjIsNull

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 235.4583084577114 > 163.72059560643564.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -73.6443613694094 (T) = (0 -229436.471536146) / Math.Sqrt((85374.47803562158 / (43)) + (7916438.017468861 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.46976413269129824 = (156104.27988606773 - 229436.471536146) / 156104.27988606773 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; PerfLabTests.CastingPerf.FooObjIsNull()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0A94F8A0]
       test      ecx,ecx
       jle       short M00_L01
M00_L00:
       mov       esi,ds:[7094148]
       mov       edx,7094158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 40

Compare Jit Disasm

; PerfLabTests.CastingPerf.FooObjIsNull()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0B512918]
       test      ecx,ecx
       jle       short M00_L01
       mov       esi,ds:[7804148]
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
M00_L00:
       mov       edx,7804158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 50

PerfLabTests.CastingPerf.ObjObjrefValueType

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 220.44800920910075 > 163.72261014851486.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -65.81211461747941 (T) = (0 -227663.18376580504) / Math.Sqrt((73433.08585642751 / (43)) + (9455613.775383344 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.45879811064601056 = (156062.15973572055 - 227663.18376580504) / 156062.15973572055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; PerfLabTests.CastingPerf.ObjObjrefValueType()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0B05F8A0]
       test      ecx,ecx
       jle       short M00_L01
M00_L00:
       mov       esi,ds:[77F4154]
       mov       edx,77F4158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 40

Compare Jit Disasm

; PerfLabTests.CastingPerf.ObjObjrefValueType()
       push      ebp
       mov       ebp,esp
       push      esi
       xor       eax,eax
       mov       ecx,ds:[0AE52918]
       test      ecx,ecx
       jle       short M00_L01
       mov       esi,ds:[6F84154]
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
       nop
M00_L00:
       mov       edx,6F84158
       call      CORINFO_HELP_ASSIGN_REF_ESI
       inc       eax
       cmp       eax,ecx
       jl        short M00_L00
M00_L01:
       pop       esi
       pop       ebp
       ret
; Total bytes of code 50

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author: performanceautofiler[bot]
Assignees: EgorBo
Labels:

area-CodeGen-coreclr, untriaged
Milestone: -

@LoopedBard3 LoopedBard3 changed the title [Perf] Windows/x86: 5 Regressions on 12/3/2022 11:06:27 AM Regressions in PerfLabTests.CastingPerf Dec 6, 2022
@LoopedBard3 LoopedBard3 added tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark labels Dec 6, 2022
@JulieLeeMSFT JulieLeeMSFT removed the untriaged New issue has not been triaged by the area owner label Dec 27, 2022
@JulieLeeMSFT JulieLeeMSFT added this to the 8.0.0 milestone Dec 27, 2022
@EgorBo
Copy link
Member

EgorBo commented Mar 27, 2023

My change is not related and the benchmarks are just bi-modal:

image

@EgorBo EgorBo closed this as completed Mar 27, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Apr 26, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@EgorBo EgorBo

Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Projects
None yet
Milestone
8.0.0
Development

No branches or pull requests
3 participants
@EgorBo @LoopedBard3 @JulieLeeMSFT