Performance Fundamental: Net 8 -> Net 9 Manual Comparison Report

[DrewScoggins]

The full comparison reports are included as links in the details section below.

Investigated

Top level issue	Issue Date	Notes	x64 CoreCLR Win	x64 CoreCLR Lin	x64 MonoJit Lin	Arm64 CoreCLR Lin	Wasm	Arm64 MonoAOT Lin
#97529	5-Apr	Scalar Conversions - Closed as expected	dotnet/perf-autofiling-issues#32430	dotnet/perf-autofiling-issues#32435
#101247, #93020 (Task Issue)	16-Apr	Flow Graph Modernization and Improved Block Layout, Likelihood edgebased weights - Closed as unregressed by further updates	dotnet/perf-autofiling-issues#33045	dotnet/perf-autofiling-issues#33063
#100824 (PR), #102900 (Potential Fix)	20-Apr	GetNetworkInterfaces update - Potentially fixed	dotnet/perf-autofiling-issues#33051
#104975, #102655 (PR)	11-Jul	Regex Regressions SymbolicRegexMatcher - Fixed by #105668	dotnet/perf-autofiling-issues#38297	dotnet/perf-autofiling-issues#38275
#103971, #103527 (PR)	20-Jun	System.Numerics Vector Cleanup - Moved to net10, also potentially poorly designed microbenchmarks	dotnet/perf-autofiling-issues#36989	dotnet/perf-autofiling-issues#36882		dotnet/perf-autofiling-issues#37168	dotnet/perf-autofiling-issues#37140
#103974	22-Jun	Avx512 IndexOfAnyAsciiSearcher - Commit Reverted	dotnet/perf-autofiling-issues#36991
#105328	15-Jul	Related to JCC, might be old hardware issue	dotnet/perf-autofiling-issues#38716	dotnet/perf-autofiling-issues#38692 (potentially overlooked in parent), #105328 (https://github.com/dotnet/perf-autofiling-issues/issues/38696), dotnet/perf-autofiling-issues#38697		dotnet/perf-autofiling-issues#38904, #105328
#102763 (tracking), #102343 (PR)	22-May	Enable RPO Block Layout - Moved to net10	dotnet/perf-autofiling-issues#35156	dotnet/perf-autofiling-issues#35095, #102343		dotnet/perf-autofiling-issues#35370
#106027	2-Aug	More ArgumentNullException.ThrowIfNull - Commit reverted and tests stated to be back around pre-commit levels	#106027
#98505, #98040 (PR)	7-Feb	Closed as by design	#98505	#98040
#103972, #103785 (PR)	21-Jun	BBJ_Always compaction - Moved to net10	dotnet/perf-autofiling-issues#36990	dotnet/perf-autofiling-issues#36883		dotnet/perf-autofiling-issues#37169
#98044, #97722 (PR)	3-Feb	Most regressed, only EightQueens still regressed, moved to net10	#98044	#98044 (Missed, other hardware)
#105236 (PGO)	24-Jul	PGO Update, closed	dotnet/perf-autofiling-issues#39166	#105236		dotnet/perf-autofiling-issues#39262 (PGO)
dotnet/perf-autofiling-issues#35582	31-May	Marked as likely alignment related	dotnet/perf-autofiling-issues#35581 (closed, date mismatch), dotnet/perf-autofiling-issues#35582
#102301 (PR)	5-Jun	Marked as resolved, but DotBenchmark appears to not be resolved	#102301 (PR)	dotnet/perf-autofiling-issues#36054		dotnet/perf-autofiling-issues#36174
dotnet/perf-autofiling-issues#33485 (PGO?)	23-Apr	Closed as Look Again/PGO	dotnet/perf-autofiling-issues#33485 (PGO?)	dotnet/perf-autofiling-issues#33489 (PGO)
#98705	13-Feb	Marked as essentially by design and due to update to fix bug	#98705	dotnet/perf-autofiling-issues#29444, dotnet/perf-autofiling-issues#29445
#96014 (PGO Update)	15-Dec	Likely due to PGO update	#96014 (PGO Update)			#96014 (PGO)
#97637 (PGO)	8-Feb	Likely due to PGO update	#97637 (PGO)			dotnet/perf-autofiling-issues#29276
dotnet/perf-autofiling-issues#27330, #96751 (PR), #97024 (Tracking, related?)	10-Jan	Linked, mostly (~90%?) fix by improvements from later updates, maybe linked to related? FannKuchRedux is called out in #97042 as the only one not to have returned.	dotnet/perf-autofiling-issues#27330	#97042 (Missed)
#102763 (tracking), #99634 (PR)	14-Mar	Remove bbFallsThrough restriction - Moved to net10	#100085			#102763
#97399 (tracking)	17-Jan	Reverted	#97399			#97399
#98668 (tracking)	13-Feb	Set to future milestone in june		dotnet/perf-autofiling-issues#29443
#105707	26-Jul	Moved to 10.0	dotnet/perf-autofiling-issues#39168
PGO 10526	24-Jul	PGO	dotnet/perf-autofiling-issues#39166
dotnet/perf-autofiling-issues#34579	13-May	Roslyn update	dotnet/perf-autofiling-issues#34579
#104974	9-Jul	Won't Fix	dotnet/perf-autofiling-issues#38295
dotnet/perf-autofiling-issues#38722	19-Jul	Seems like two regressions, but only one was looked at	dotnet/perf-autofiling-issues#38722
PGO #105236	24-Jul	PGO		dotnet/perf-autofiling-issues#39084
#99315	27-Feb	Not fully understood it seems, but moved to net10		#99315 (https://github.com/dotnet/perf-autofiling-issues/issues/30418)
#99002	25-Feb	By design/acceptable given improvements		#99002 (https://github.com/dotnet/perf-autofiling-issues/issues/29817)
#94475	2-Nov	Moved to 10.0	#106027
dotnet/perf-autofiling-issues#30843	5-Mar	Memory Randomization - Expected		dotnet/perf-autofiling-issues#30843		dotnet/perf-autofiling-issues#30843
#93926, #93371 (PR)	18-Oct	Defer throw helper blocks - Seems most have unregressed but some Perf_BitArray tests have not		#93926, #93371 (PR)
#106345, #106014 (PR)	8-Aug	EventSource Startup. Mix of noise and not noise. Initial linked PR dismissed, not fully investigated		#106345, #106014 (PR)
#104303, #102739 (PR)	27-Jun	Closed as by design due to the bi-modal/unstable behaviour		dotnet/perf-autofiling-issues#37353
#105616 (PGO)	29-Jul	PGO		#105616		#105616 (PGO)
#94775	2-Nov	Moved to 10.0		#94775
#94689, #95645 (Tracking)	28-Nov	Closed based on latest		#94689		#95645, dotnet/perf-autofiling-issues#25601
#100090	18-Mar	Closed as largest regression have resolved		dotnet/perf-autofiling-issues#31796
#99964	15-Mar	Closed as nothing actionable for net 9 and clear regressions tracked in other issues		dotnet/perf-autofiling-issues#31340
#96619 (PGO)	8-Jan	PGO		#96619 (PGO)
#91497 (PGO)	11-Sep	PGO		#91497 (PGO)
#92709 (PR), #94717	9-Nov	Closed as mostly unregressed		#94717
#93116 (PR)	15-Oct	x64LinCore closed as edge case		dotnet/perf-autofiling-issues#23222 (Mentioned in #98209 per-preview perf overview)
#97400, #97075 (PR)	19-Jan	Most recovered, SelectToList regressed on only one machine, Closed		#97400		#97400
dotnet/perf-autofiling-issues#27270, #96756 (PR), dotnet/perf-autofiling-issues#27332 (Related Issue)	12-Jan	dotnet/perf-autofiling-issues#27332 (x64WinCore) closed due to the tests fixing themselves, x64 Linux CoreCLR mostly fixed itself but still has potentially related regression		dotnet/perf-autofiling-issues#27270
#99121	23-Feb	Moved to net10, possibly a tail-call + special intrinsic problem				#99121, (#99122 Closed as dupe of #99121)
#99419	1-Mar	Closed as fixed, but the single direct regression was not fixed				#99419
#98176, #97921 (PR)	5-Feb	More Addressing modes for arm64. Regression was considered acceptable (diff regression than the Arm64 one in sheet)				#98176 (Missed, Likely related)
#103714 (PR), #104107	25-Jun	Closed as by design. Seemingly only the one test was affected.				#104107
#102319	11-May	Closed as BigInteger needs modernization anyways and there were still good improvements				#102319
dotnet/perf-autofiling-issues#32308	26-Mar	Closed with minimal/no followup.				dotnet/perf-autofiling-issues#32317
dotnet/perf-autofiling-issues#34038	2-May	Known issue with Volatiles				dotnet/perf-autofiling-issues#34038
#101990, #101739 (Poss. PR)	3-May	Closed as many unregressed from this change but Arm64 still seems to have some lingering issues		#101990		dotnet/perf-autofiling-issues#34039 (Newly linked to #101990, potentially initially missed?)
#102044	3-May	Marked as complete/fixed				#102044
dotnet/perf-autofiling-issues#22914	3-Oct	Closed as improvements reported				dotnet/perf-autofiling-issues#22938
#92774	22-Sep	Closed as completed after triaging and being put on backlog. No clear comment on why marked as completed				#92774
#101127	8-Apr	Issue due to generics change that Mono does not handle performantly due to lack of generic specialization						#101127
#104076	3-Jun	Still under discussion						dotnet/perf-autofiling-issues#36103
dotnet/perf-autofiling-issues#39238	22-Jul	Attributed to LLVM upgrade						dotnet/perf-autofiling-issues#39238
#103653	14-Jun	Moved to 10.0						#103653
dotnet/perf-autofiling-issues#24014	26-Oct	Triaged - Unknown (9 cases)					dotnet/perf-autofiling-issues#24014
dotnet/perf-autofiling-issues#38086	2-Jul	Triaged - PR made, FMA regressions not fixed (7 cases)					dotnet/perf-autofiling-issues#38086
dotnet/perf-autofiling-issues#26200	13-Dec	Known - Chrome Bump (1 case)					dotnet/perf-autofiling-issues#26200
#99553	22-Feb	Investigated - Unknown (3 cases)					#99553
dotnet/perf-autofiling-issues#31842	21-Mar	Assigned - Unknown (1 case)					dotnet/perf-autofiling-issues#31842
dotnet/perf-autofiling-issues#31405	12-Mar	Investigated - Unknown (1 case)					dotnet/perf-autofiling-issues#31405
dotnet/perf-autofiling-issues#30899	7-Mar	Investigated - Unfixed (3 cases)					dotnet/perf-autofiling-issues#30899
dotnet/perf-autofiling-issues#34714	13-May	Investigated - Likely chrome bump (4 cases)					dotnet/perf-autofiling-issues#34714
dotnet/perf-autofiling-issues#23013	3-Oct	Fix merged - regression not fully recovered (3 cases)					dotnet/perf-autofiling-issues#23013
dotnet/perf-autofiling-issues#40068	15-Aug	Investigated - Unfixed, Filed for Mono but not wasm (4 cases)					dotnet/perf-autofiling-issues#40068
#100334 (PR)	15-Jul	Cause found but still open/no discussion on fix (235 cases)					dotnet/perf-autofiling-issues#38866

Un/partially investigated

Top Level Issues	Issue Date	Notes/Status	x64 CoreCLR Win	x64 CoreCLR Lin	x64 MonoJit Lin	Arm64 CoreCLR Lin	Wasm	Arm64 MonoAOT Lin
#100922, #99982 (PR)	8-Apr	Use iSimdVector and Align WidenAsciiToUtf16 - In discussion (> 1 week since update)	dotnet/perf-autofiling-issues#32733	dotnet/perf-autofiling-issues#32721		#100922
#103143 (PR)	15-Jun	Closed incorrectly/not fully followed up on	dotnet/perf-autofiling-issues#36447
#96402 (PR), #96496 (Tracking)	2-Jan	Regex Search Values - In discussion, not fixed	Missed			dotnet/perf-autofiling-issues#27119
#102927 (PR)	17-Jun	Hot jump fallthrough - Seemingly uninvestigated	dotnet/perf-autofiling-issues#36986	dotnet/perf-autofiling-issues#36878, #102927 (Missed), #102927		dotnet/perf-autofiling-issues#36657
#99314	1-Mar	Improve isinst - At least one case fixed, not all seem to be unregressed and are uninvestigated (could be due to other issue). Some tests had been removed from the issue so are untracked.	Missed	#99314 (https://github.com/dotnet/perf-autofiling-issues/issues/30419) (Deleted)
dotnet/perf-autofiling-issues#30481 (Noise?)	28-Feb	Potential noise or incorrectly closed	dotnet/perf-autofiling-issues#30481 (Noise?)
#102461 (cause?)	21-May	Marked as noise but should be investigated/incorrectly closed, though it seems some of the regressions really occurred on May 22 as part of Enable RPO and some of the regressions (Crc32) seem to be real. Some may be related to #102461. X64Linux has some that point to #102461 as the cause, but not correctly tracked to it.	dotnet/perf-autofiling-issues#35151 (Noise?)	dotnet/perf-autofiling-issues#35092, #102461		dotnet/perf-autofiling-issues#34815 (Untriaged)
dotnet/perf-autofiling-issues#23165	10-Oct	PGO Update but not closed or tracked	dotnet/perf-autofiling-issues#23165
#103820 (PR)	24-Jun	Open, issue was only made for linux, potentially missing windows?	dotnet/perf-autofiling-issues#37463 (missed)
#103296	3-Jun	In discussion/Still Open	dotnet/perf-autofiling-issues#36066
dotnet/perf-autofiling-issues#37376	27-Jun	Closed as Noise Incorrectly - Cause is unclear in commits	dotnet/perf-autofiling-issues#37376
#100456 (PR), #93020 (tracking)	9-Apr	Initial Profile Repair as part of Flow Graph modernization and Block Layout - Closed, Uninvestigated?	dotnet/perf-autofiling-issues#32734			#100456
#91903 (PR)	22-Sep	BitArrayCopyToBoolArray - Open, needs discussion	dotnet/perf-autofiling-issues#22213
dotnet/performance#4318	9-Jul	zlib-ng changes - Open, In Discussion	dotnet/perf-autofiling-issues#38293
dotnet/perf-autofiling-issues#37373	24-Jun	Open, needs investigation	dotnet/perf-autofiling-issues#37373
dotnet/perf-autofiling-issues#36448	15-Jun	Look Again that was not looked at again	dotnet/perf-autofiling-issues#36448
dotnet/performance#4318	9-Jul	Tracked, but not fixed	dotnet/perf-autofiling-issues#38293
dotnet/perf-autofiling-issues#37373	24-Jun	Look Again that was not looked at again	dotnet/perf-autofiling-issues#37373
dotnet/perf-autofiling-issues#36445	13-Jun	Closed as Noise Incorrectly - Cause is unclear in commits	dotnet/perf-autofiling-issues#36445
#103462 (PR)	15-Jun	Expose new Vector API's. Noted as regression because we are no longer constant folding on sincos, though other regressions seem uninvestigated/followed up.	dotnet/perf-autofiling-issues#36447	dotnet/perf-autofiling-issues#36405, dotnet/perf-autofiling-issues#36406 (noise/missed?), #103462		dotnet/perf-autofiling-issues#36655, dotnet/perf-autofiling-issues#36654	dotnet/perf-autofiling-issues#36484
#97011 (PR)	2-Feb	Jump-to-next-block JIT stuff. Only improvements linked so no followup or connected issues. (x64 Lin only had 1 seemingly affected test)		#97011 (PR)
dotnet/perf-autofiling-issues#29816	24-Feb	Marked as noise but should be investigated/was incorrectly closed		dotnet/perf-autofiling-issues#29816
dotnet/perf-autofiling-issues#36880	19-Jun	Marked as look again and not followed up on		dotnet/perf-autofiling-issues#36880
#102884 (PR)	22-Jul	No regressions linked, dotnet/perf-autofiling-issues#39080 closed as noise		#102884, dotnet/perf-autofiling-issues#39080
#104328	4-Jul	Utf8JsonReader, Linked but seemingly unfollowed up		dotnet/perf-autofiling-issues#37804
dotnet/perf-autofiling-issues#31338	14-Mar	Marked look again		dotnet/perf-autofiling-issues#31338
dotnet/perf-autofiling-issues#23287, #93214 (Likely PR)	10-Oct	Missing autofile on Linux x64 Coreclr, Not followed up on		dotnet/perf-autofiling-issues#23287		dotnet/perf-autofiling-issues#23287
dotnet/perf-autofiling-issues#24755, #94631 (PR)	11-Nov	Improve forwardsub, uninvestigated		dotnet/perf-autofiling-issues#24755
dotnet/perf-autofiling-issues#39162 (Tracking)	22-Jul	Marked Look again		dotnet/perf-autofiling-issues#39081
dotnet/perf-autofiling-issues#35581 (Tracking, potentially some misattributed regs)	31-May	Closed as noise	dotnet/perf-autofiling-issues#35581	dotnet/perf-autofiling-issues#35534
dotnet/perf-autofiling-issues#33247	16-Apr	Not triaged or looked at. Likely #99736				dotnet/perf-autofiling-issues#33247
dotnet/perf-autofiling-issues#33248	17-Apr	Not triaged or looked at. Commit range: e126f8f...2fe9b3b				dotnet/perf-autofiling-issues#33248
#105250 (PR), dotnet/perf-autofiling-issues#39086	24-Jul	Marked look again and only a handful of regressions linked by autofiling (2)				dotnet/perf-autofiling-issues#39263
#100506 (PR), dotnet/perf-autofiling-issues#36173	4-Jun	Closed without followup. Found in ASP Lab (#103256 - Open issue)				dotnet/perf-autofiling-issues#36173
dotnet/perf-autofiling-issues#35725	31-May	Untriaged for Lin Arm64 CoreCLR, though seems to have been caught on Mono (dotnet/runtime/issues/#103460). Potentially from PR dotnet/runtime/pull/#102874.				dotnet/perf-autofiling-issues#35725
dotnet/perf-autofiling-issues#27496, #96555 (PR)	6-Jan					dotnet/perf-autofiling-issues#27496
dotnet/perf-autofiling-issues#30003	20-Feb	Huge issue, with gaps and some tests were not followed up on						dotnet/perf-autofiling-issues#30003
dotnet/perf-autofiling-issues#38961	19-Jul	Untriaged - WASM (11 cases)					dotnet/perf-autofiling-issues#38961
dotnet/perf-autofiling-issues#36219	5-Jun	Untriaged - WASM (12 cases)					dotnet/perf-autofiling-issues#36219
dotnet/perf-autofiling-issues#38094	10-Jul	Untriaged - WASM (8 cases)					dotnet/perf-autofiling-issues#38094
dotnet/perf-autofiling-issues#32885	14-Apr	Untriaged - WASM (2 cases)					dotnet/perf-autofiling-issues#32885
dotnet/perf-autofiling-issues#33255	16-Apr	Untriaged - WASM (1 case)					dotnet/perf-autofiling-issues#33255
dotnet/perf-autofiling-issues#32591	8-Apr	Untriaged - WASM (1 case)					dotnet/perf-autofiling-issues#32591
dotnet/perf-autofiling-issues#40277	18-Aug	Untriaged - WASM (10 cases)					dotnet/perf-autofiling-issues#40277
dotnet/perf-autofiling-issues#39259	30-Jul	Untriaged - WASM (2 cases)					dotnet/perf-autofiling-issues#39259
dotnet/perf-autofiling-issues#31645	15-Mar	Untriaged - WASM (1 case)					dotnet/perf-autofiling-issues#31645
dotnet/perf-autofiling-issues#35704	31-May	Untriaged - WASM (1 case)					dotnet/perf-autofiling-issues#35704
dotnet/perf-autofiling-issues#27906	17-Jan	Untriaged - WASM (2 cases)					dotnet/perf-autofiling-issues#27906
dotnet/perf-autofiling-issues#29881	23-Feb	Reports missing from Feb 9 - 21, large range of possible causes (13 cases)					dotnet/perf-autofiling-issues#29881
dotnet/perf-autofiling-issues#36219	5-Jun	Untriaged - WASM (12 cases)					dotnet/perf-autofiling-issues#36219

Missed by auto-file

Top level issue	Date	Notes	x64 CoreCLR Win	x64 CoreCLR Lin	x64 MonoJit Lin	Arm64 CoreCLR Lin
#97202 (PR)	26-Jan	Was not auto-filed	N/A
f36aa7e...ee501fb
15-Mar	No autofile	N/A
#100160 (PR)	26-Mar	Missed by auto-filer	N/A
6931b3b...815650e	5-Aug	No autofile		N/A
16cb414...9a3cacd	26-Jan	No autofile		N/A
#97488 (PR)	29-Jan	No regressions linked but seems to be likely cause of at least 1 Linux Coreclr x64 regression		#97488 (PR)
c7d36b4...d5026a1	28-Aug	No autofile, unclear cause		N/A
#102261 (PR)	30-May	No linked regressions, may be cause of at least one on x64LinCore		#102261 (PR)
6927fea...d7ae8c6	13-Jun	No autofile, one test x64LinCore		N/A
#99541 (PR)	12-Mar	No regressions linked, but seemingly 1 x64LinCore regression missed		#99541 (Missed, Likely PR)
#106578 (PR)	18-Aug	No linked regressions and recent so unclear if potential cause		#106578 (PR)
#98610 (PR)	19-Feb	PR never linked to any regressions, though seems most likely to be cause of an Arm64 Lin CoreCLR regression				#98610 (Missed, PR)
e1c6717...0fb0188	7-May					N/A (1 Issue, Brotli.Compress_WithoutState: 7.85% reg)
9bb2b5b...60be20c	29-Nov					N/A (1 Issue, Perf_Console.ResetConsole: 6.12%)
c422fca...8c02b08	9-Nov					N/A (1 Issue, LinqBenchmarks.Order00ManualX: 5.73%)
954ead3...fae6720	13-Feb					N/A (1 Issue, Perf_Uri.EscapeDataString: 14.2%)
74f7f91...1bf325c	19-Jun					N/A (2 Issues, IPAdressPerformanceTests: Noisy NS)
#100899 (PR)	11-Apr	No regressions linked but likely cause of Arm64 lin Core ref handle stabalization to high bi-modal behavior				#100899 (PR)
ddf9fee...0d641e5	26-Jan					N/A (1 Issue, CtorFromCollectionNonGeneric: 47.94%+)
d752f9a...b29efed	26-Feb	No issue - Likely chrome bump (3 cases)

• x64 Windows CoreClr
• x64 Linux Mono JIT
• x64 Linux CoreClr
• x64 WasmLinux
• Arm64 Linux CoreClr
• Arm64 Linux MonoAOT

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-meta
See info in area-owners.md if you want to be subscribed.