-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Regressions in System.Reflection.Attributes #91620
Comments
Run Information
Regressions in Exceptions.Handling
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Exceptions.Handling*' PayloadsExceptions.Handling.CatchAndRethrowDeep(kind: Software)ETL FilesHistogramDescription of detection logic
JIT DisasmsExceptions.Handling.CatchAndThrowOtherDeep(kind: Software)ETL FilesHistogramDescription of detection logic
JIT DisasmsExceptions.Handling.CatchAndThrowOtherDeep(kind: Hardware)ETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Collections.CtorFromCollectionNonGeneric<Int32>
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollectionNonGeneric<Int32>*' PayloadsSystem.Collections.CtorFromCollectionNonGeneric<Int32>.ArrayList(Size: 512)ETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Collections.CtorFromCollectionNonGeneric<Int32>.Queue(Size: 512)ETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Collections.CtorFromCollectionNonGeneric<Int32>.Stack(Size: 512)ETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Tests.Perf_Array
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*' PayloadsSystem.Tests.Perf_Array.ArrayRetrieve2DETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Tests.Perf_Array.ArrayRetrieve3DETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Tests.Perf_Array.ArrayRetrieve1DETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Cache
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Cache*' PayloadsSystem.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 40000, unique: 7, cacheSize: 0)ETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Memory.ReadOnlyMemory<Char>
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlyMemory<Char>*' PayloadsSystem.Memory.ReadOnlyMemory<Char>.Pin(Size: 512)ETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.ComponentModel.Tests.Perf_TypeDescriptorTests
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.ComponentModel.Tests.Perf_TypeDescriptorTests*' PayloadsSystem.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeEnum))ETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository |
@DrewScoggins is there a reason that this perf regression, and recent filed issues, do not have a commit range to help identify the cause? Without the range, these will be assigned the test owner, but that is often not the best first place to start. |
Tagging subscribers to this area: @dotnet/area-system-reflection Issue DetailsRun Information
Regressions in System.Reflection.Attributes
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*' PayloadsSystem.Reflection.Attributes.GetCustomAttributesClassHitInheritETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesMethodOverrideMissETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesMethodOverrideMissInheritETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesMethodBaseHitInheritETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.IsDefinedMethodOverrideHitInheritETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesMethodBaseHitETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesClassMissETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesClassHitETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesMethodOverrideHitETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.IsDefinedMethodOverrideHitETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Memory.ReadOnlyMemory<Byte>
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlyMemory<Byte>*' PayloadsSystem.Memory.ReadOnlyMemory<Byte>.Pin(Size: 512)ETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Memory.Memory<Char>
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Memory<Char>*' PayloadsSystem.Memory.Memory<Char>.Pin(Size: 512)ETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Memory.Memory<Byte>
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Memory<Byte>*' PayloadsSystem.Memory.Memory<Byte>.Pin(Size: 512)ETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Collections.Tests.Add_Remove_SteadyState<String>
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Add_Remove_SteadyState<String>*' PayloadsSystem.Collections.Tests.Add_Remove_SteadyState<String>.ConcurrentStack(Count: 512)ETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Threading.Tests.Perf_CancellationToken
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_CancellationToken*' PayloadsSystem.Threading.Tests.Perf_CancellationToken.CancelAfterETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository
|
Also, the apparent regression was from August 29-30 but the graph pics show a perf improvement, not a regression. However, there was a regression in late July or early August but this issue isn't for that so this is confusing. |
Those were always found "manually" using the graphs in the issue, and then added. I assume that the triage v-team has just not been grabbing them lately. For this filed issue this is the range, 87f3817...9c3f8b3.
So first, the regression in late July was already filed here, dotnet/perf-autofiling-issues#20306. In the case here we are seeing more regressions. It can sometimes be clearer to see if you click on the Test Report link below the tables. In the case of the Hope this helps. |
Where did this range come from? The various "Diff" links above show 86252f7...03d25cd. If I understand correctly this issue lists the following regressions:
but this issue is currently assigned to |
Also the "test report" for at least the first couple of areas have this larger commit range: 976c2ab...9eb0266 |
If the smaller commit range 87f3817...9c3f8b3 is used, then I suspect #89311. |
Assigning to area-Host based on #89311 and also because that calls out PGO-related issues. |
Tagging subscribers to this area: @vitek-karas, @agocke, @VSadov Issue DetailsRun Information
Regressions in System.Reflection.Attributes
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*' PayloadsSystem.Reflection.Attributes.GetCustomAttributesClassHitInheritETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesMethodOverrideMissETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesMethodOverrideMissInheritETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesMethodBaseHitInheritETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.IsDefinedMethodOverrideHitInheritETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesMethodBaseHitETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesClassMissETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesClassHitETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.GetCustomAttributesMethodOverrideHitETL FilesHistogramDescription of detection logic
JIT DisasmsSystem.Reflection.Attributes.IsDefinedMethodOverrideHitETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Memory.ReadOnlyMemory<Byte>
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlyMemory<Byte>*' PayloadsSystem.Memory.ReadOnlyMemory<Byte>.Pin(Size: 512)ETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Memory.Memory<Char>
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Memory<Char>*' PayloadsSystem.Memory.Memory<Char>.Pin(Size: 512)ETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Memory.Memory<Byte>
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Memory<Byte>*' PayloadsSystem.Memory.Memory<Byte>.Pin(Size: 512)ETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Collections.Tests.Add_Remove_SteadyState<String>
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Add_Remove_SteadyState<String>*' PayloadsSystem.Collections.Tests.Add_Remove_SteadyState<String>.ConcurrentStack(Count: 512)ETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Threading.Tests.Perf_CancellationToken
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_CancellationToken*' PayloadsSystem.Threading.Tests.Perf_CancellationToken.CancelAfterETL FilesHistogramDescription of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository
|
We try and narrow down the diff range as much as we can automatically, but it is difficult to always be correct. So we bias towards giving a slightly larger range automatically, to ensure that we don't miss it entirely. But manual inspection can often give a much better range. |
Thanks for the reminder on that. We have had updated PGO data for the last couple weeks. |
#89311 was reverted:
So this should no longer be an issue from PGO being disabled. |
Run Information
Regressions in System.Reflection.Attributes
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Reflection.Attributes.GetCustomAttributesClassHitInherit
ETL Files
Histogram
Description of detection logic
JIT Disasms
System.Reflection.Attributes.GetCustomAttributesMethodOverrideMiss
ETL Files
Histogram
Description of detection logic
JIT Disasms
System.Reflection.Attributes.GetCustomAttributesMethodOverrideMissInherit
ETL Files
Histogram
Description of detection logic
JIT Disasms
System.Reflection.Attributes.GetCustomAttributesMethodBaseHitInherit
ETL Files
Histogram
Description of detection logic
JIT Disasms
System.Reflection.Attributes.IsDefinedMethodOverrideHitInherit
ETL Files
Histogram
Description of detection logic
JIT Disasms
System.Reflection.Attributes.GetCustomAttributesMethodBaseHit
ETL Files
Histogram
Description of detection logic
JIT Disasms
System.Reflection.Attributes.GetCustomAttributesClassMiss
ETL Files
Histogram
Description of detection logic
JIT Disasms
System.Reflection.Attributes.GetCustomAttributesClassHit
ETL Files
Histogram
Description of detection logic
JIT Disasms
System.Reflection.Attributes.GetCustomAttributesMethodOverrideHit
ETL Files
Histogram
Description of detection logic
JIT Disasms
System.Reflection.Attributes.IsDefinedMethodOverrideHit
ETL Files
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Memory.ReadOnlyMemory<Byte>
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Memory.ReadOnlyMemory<Byte>.Pin(Size: 512)
ETL Files
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Memory.Memory<Char>
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Memory.Memory<Char>.Pin(Size: 512)
ETL Files
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Memory.Memory<Byte>
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Memory.Memory<Byte>.Pin(Size: 512)
ETL Files
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Collections.Tests.Add_Remove_SteadyState<String>
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Collections.Tests.Add_Remove_SteadyState<String>.ConcurrentStack(Count: 512)
ETL Files
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Threading.Tests.Perf_CancellationToken
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Threading.Tests.Perf_CancellationToken.CancelAfter
ETL Files
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
The text was updated successfully, but these errors were encountered: