.NET 9 per-preview performance overview

[kotlarmilos]

Description

This issue provides an overview of the all performance regressions from microbenchmark automated runs in WASM, Mono AOT, and Interpreter during the timeframe of .NET 9 preview releases. WASM regressions are tracked separately at #96444.

List of queries for autofilling issues:

• Mono AOT autofilling issues: https://github.com/dotnet/perf-autofiling-issues/issues?q=is%3Aissue+is%3Aopen+label%3Amono-aot
• Interpreter autofilling issues: https://github.com/dotnet/perf-autofiling-issues/issues?q=is%3Aopen+is%3Aissue+label%3Amono-interpreter
• WASM autofilling issues: https://github.com/dotnet/perf-autofiling-issues/issues?q=is%3Aopen+is%3Aissue+label%3Aarch-wasm
• WASM AOT autofilling issues: https://github.com/dotnet/perf-autofiling-issues/issues?q=is%3Aopen+is%3Aissue+label%3Awasm-aot
• Mono JIT autofilling issues: https://github.com/dotnet/perf-autofiling-issues/issues?q=is%3Aopen+is%3Aissue+label%3Amono-jit

Mono AOT

Summary of the most severe unresolved regressions from .NET 9

• [Perf] Linux/arm64: 22 Regressions on 4/8/2024 7:16:22 PM · Issue #101127 · dotnet/runtime (github.com) (Generic methods related. IO, Text regressions, affecting arm64/x64, WASM, ...)
• [Perf] Linux/x64: 44 Regressions on 6/3/2024 6:35:27 PM · Issue #104076 · dotnet/runtime (github.com) (a lot of regressions caused by Use generic math for floating point formatting by huoyaoyuan · Pull Request #102683 · dotnet/runtime (github.com), affecting both x64 and arm64)
• [Perf] Linux/x64: 26 Regressions on 6/14/2024 3:42:32 AM · Issue #103653 · dotnet/runtime (github.com) (Generic methods related. Base64 regressions, x64/arm64)

Details

RC1

Issue	Status	Comment
dotnet/perf-autofiling-issues#39525	Resolved	ConcurrentDictionary, affecting CoreCLR as well, x64/arm64
dotnet/perf-autofiling-issues#39828	Closed	Regex_Cache.IsMatch regression, CoreCLR as well (#106346)

Preview 7

Issue	Status	Comment
dotnet/perf-autofiling-issues#39238	Closed	Likely caused LLVM 19 bump, no action
#105335	Resolved	ConcurrentBag (threading)
#106475	Closed	Perf_FloatingPointTensorPrimitives.MaxMagnitude
dotnet/perf-autofiling-issues#38390	Closed	Base64EncodeDecodeInPlaceTests
dotnet/perf-autofiling-issues#37939	Resolved	Tagged causing PR, Tanner says by design.
#104302	Resolved	CredentialCacheTests, no action expected regression
dotnet/perf-autofiling-issues#37022	Closed	JSON and Buffer regressions, tagged author
#104022, dotnet/perf-autofiling-issues#37021	Resolved	Vector SIMD intrinsics (both x64 and arm64), tagged Tanner

Preview 6

Issue	Status	Comment
dotnet/perf-autofiling-issues#37020	Closed	System.Collections.IndexerSetReverse
#103653	Investigation	Perf_Base64
#103652	Investigation	Vector SIMD intrinsics
#104076	Resolving	Perf_Double/Single/NumberCultureInfo, tagged author, looks like Mono perf measurements weren't accurate before merging
#103460, dotnet/perf-autofiling-issues#35612	closed	BigInteger, tagged author

Preview 5

Issue	Status	Comment
dotnet/perf-autofiling-issues#35132	false pos.	JSON regressions, tagged author
#102557	Backlog	VectorConvert, x64 only
#102281	Resolved	CredentialCacheTests, Assigned
dotnet/perf-autofiling-issues#33792	Resolved	UnboundedChannelPerfTests, Tagged author, PR reverted
#106402	closed	GetType_Resolvers, Tagged author

Preview 4

Issue	Status	Comment
#102382	Resolved	Authors notified
dotnet/perf-autofiling-issues#33133, dotnet/perf-autofiling-issues#33094	Resolved	Need intrinsifying BitCast API #101495
#104777	Closed	System.Reflection.Invoke, Authors notified #100846
#101124, #101127	Resolving	Authors notified #99982
dotnet/perf-autofiling-issues#32511	Resolved

Preview 3

Issue	Status	Comment
#106462	Closed	EventSourceLogger.NetScopes_TwoMessages regressions, long range of commits, x64/arm64, caused by BDN change.
#106479	Investigation	Reflection.Invoke regression, nothing suspicious in the range, x64/arm64
#102865	Resolved	Perf_Single/Double.TryParse, affecting CoreCLR as well

Preview 2

Issue	Status
dotnet/perf-autofiling-issues#30456, dotnet/perf-autofiling-issues#30503	Resolved

Preview 1

Issue	Status
#102381	Investigating
dotnet/perf-autofiling-issues#27018, dotnet/perf-autofiling-issues#27047	Resolved (no action)
dotnet/perf-autofiling-issues#25809	Archived (#97988)
dotnet/perf-autofiling-issues#24512	No action
dotnet/perf-autofiling-issues#27020	Resolved
dotnet/perf-autofiling-issues#27019	Resolved
dotnet/perf-autofiling-issues#26504	Resolved
dotnet/perf-autofiling-issues#26154	Resolved
dotnet/perf-autofiling-issues#25812	Resolved
dotnet/perf-autofiling-issues#25806	Resolved
dotnet/perf-autofiling-issues#25089	Resolved
dotnet/perf-autofiling-issues#25426	Resolved
dotnet/perf-autofiling-issues#25421	Resolved
dotnet/perf-autofiling-issues#24200	Resolved
dotnet/perf-autofiling-issues#24203	Resolved
dotnet/perf-autofiling-issues#23314	Resolved
dotnet/perf-autofiling-issues#23579	Resolved
dotnet/perf-autofiling-issues#24786	Resolved
dotnet/perf-autofiling-issues#24198	Resolved
dotnet/perf-autofiling-issues#22570	Resolved
dotnet/perf-autofiling-issues#22569	Resolved
dotnet/perf-autofiling-issues#23306	Resolved
dotnet/perf-autofiling-issues#23307	Resolved
dotnet/perf-autofiling-issues#23309	Resolved
dotnet/perf-autofiling-issues#23311	Resolved
dotnet/perf-autofiling-issues#23580	Resolved
dotnet/perf-autofiling-issues#23582	Resolved
dotnet/perf-autofiling-issues#23870	Resolved

Mono Interpreter

Summary of the most severe unresolved regressions from .NET 9:

• [mono][interp][Perf] Interpreter: missing vector instrinsics causing 117 Regressions on 6/15/2024 2:08:36 PM #109375 - Missing intrinsics
• [Perf] Linux/arm64: 4 Regressions on 7/30/2024 1:22:57 AM #106144 - Missing intrinsics

Details

RC1

Issue	Status

Preview 7

Issue	Status
dotnet/perf-autofiling-issues#38386	Resolving
dotnet/perf-autofiling-issues#38426	Resolving
dotnet/perf-autofiling-issues#38837	Resolving
#106144	Resolving
dotnet/perf-autofiling-issues#39545	Resolved
dotnet/perf-autofiling-issues#38827	Resolved

Preview 6

Issue	Status
#109375	Resolving
dotnet/perf-autofiling-issues#37950	Resolved
dotnet/perf-autofiling-issues#37898	Resolved
dotnet/perf-autofiling-issues#37892	Resolved
dotnet/perf-autofiling-issues#37889	Resolved
dotnet/perf-autofiling-issues#36078, dotnet/perf-autofiling-issues#36098	Resolved
dotnet/perf-autofiling-issues#36973	Resolved
dotnet/perf-autofiling-issues#36976, dotnet/perf-autofiling-issues#36978, dotnet/perf-autofiling-issues#37047	Resolved

Preview 5

Issue	Status
dotnet/perf-autofiling-issues#35112	Resolved
dotnet/perf-autofiling-issues#35186	Resolved

Preview 4

Issue	Status
dotnet/perf-autofiling-issues#33948	Resolved
dotnet/perf-autofiling-issues#32813	Resolved
dotnet/perf-autofiling-issues#32770, dotnet/perf-autofiling-issues#32770, dotnet/perf-autofiling-issues#32772, dotnet/perf-autofiling-issues#32812	Resolved
dotnet/perf-autofiling-issues#34651	Resolved
dotnet/perf-autofiling-issues#34665	Resolved

Preview 3

Issue	Status

Preview 2

Issue	Status
dotnet/perf-autofiling-issues#30463	Resolved
dotnet/perf-autofiling-issues#29872	Resolved
dotnet/perf-autofiling-issues#30464	Resolved

Preview 1

Issue	Status
dotnet/perf-autofiling-issues#29871	Resolving
dotnet/perf-autofiling-issues#27704	Resolving
dotnet/perf-autofiling-issues#29026	Resolved
dotnet/perf-autofiling-issues#23222, dotnet/perf-autofiling-issues#23611, dotnet/perf-autofiling-issues#23635	Resolved
dotnet/perf-autofiling-issues#26185	Resolved
dotnet/perf-autofiling-issues#29868	Resolved
dotnet/perf-autofiling-issues#27348	Resolved
dotnet/perf-autofiling-issues#27350, dotnet/perf-autofiling-issues#27351, dotnet/perf-autofiling-issues#27352	Resolved
dotnet/perf-autofiling-issues#26192	Resolved
dotnet/perf-autofiling-issues#22914, dotnet/perf-autofiling-issues#22938	Resolved
dotnet/perf-autofiling-issues#25465	Resolved
dotnet/perf-autofiling-issues#23877, dotnet/perf-autofiling-issues#23880	Resolved
dotnet/perf-autofiling-issues#22915	Resolved
dotnet/perf-autofiling-issues#25084	Resolved
dotnet/perf-autofiling-issues#24816	Resolved
dotnet/perf-autofiling-issues#24812	Resolved
dotnet/perf-autofiling-issues#23615	Resolved
dotnet/perf-autofiling-issues#23614	Resolved
dotnet/perf-autofiling-issues#22539, dotnet/perf-autofiling-issues#22533	Resolved

Mono JIT

Summary of the most severe unresolved regressions from .NET 9

• [Perf] Linux/x64: 113 Regressions on 2/25/2024 4:37:10 PM perf-autofiling-issues#30027
• [Perf] Linux/x64: 120 Regressions on 2/25/2024 4:37:10 PM perf-autofiling-issues#30432
• [Perf] Linux/x64: 9 Regressions on 7/10/2024 6:50:30 PM perf-autofiling-issues#38367
• [Perf] Linux/x64: 183 Regressions on 4/15/2024 6:25:38 PM perf-autofiling-issues#33182

Mono Android

Issue	Status	Comment
#106818	By design	Android size regression on 07/08/24, expected by #105714

Mono Size - iOS

Summary of the most severe unresolved regressions from .NET 9

• [mono] [iOS] Cumulative Mono iOS HelloWorld size regressions from .NET 9 #105701
• [mono][size][Perf] iOS and Android size regression on 8/9/2024 8:00:01 AM #106265

Details

Preview 7

Issue	Status	Comment
#104952	Tracked at #105701	Probably caused by new APIs/generics

Preview 6

Issue	Status	Comment
dotnet/perf-autofiling-issues#35656	Closed	Tagged author, only 0.03MB
dotnet/perf-autofiling-issues#35768	Improvement

Preview 5

Issue	Status	Comment
#104073	Tracked at #105701	Tagged author
dotnet/perf-autofiling-issues#35226	MAUI improvement	Long range, not clear cause

Preview 4 (NAOT)

Issue	Status	Comment
#104074	Resolved	No action, triggered by our build setup (disabled full stripping of symbols)

Preview 4

Date	Diff	Change	Size increase (bytes)	Size increase (%)	Tracking issue	Status
08 Apr 2024	700d724...8508806	#99982	202000	3.63	#100975	Tracked at #105701
19 Apr 2024	a3bf7d4...b067ce0	#101258	290000	1.4	#101488	Tracked at #105701

Preview 1

Date	Diff	Change	Size increase (bytes)	Size increase (%)	Tracking issue	Status
03 Oct 2023	88f2547...d2e24fa	#90764	066800	5.01	#93072	Tracked at #105701

Mono Startup - iOS

Preview 1

Date	Diff	Change	Slowdown (ms)	Slowdown (%)	Tracking issue	Status
16 Dec 2023	2284495...4dffd80	4dffd80	8.3	18.8	#96449	Resolved (no action)

MAUI

Issue	Status	Comment
#104083	Investigating	Incrementally slower startup times for MAUI-NAOT

[vitek-karas]

We may do some specific perf improvements still, but overall we're not going to address all of the changes in the release. Moving this to 10.