Investigate Throughtput regression in Microsoft.CodeAnalysis

[michellemcdaniel]

Benchview link: https://benchview/compare?jobid=48095&resulttypeid=956&archid=4&testid=63143&configid=2706&machinepoolid=1292&comparejobids=[48074]&

b5c3e56 appears to have caused a 4% regression in throughput of crossgening Microsoft.CodeAnalysis. We see this regression in both x64 and x86 Windows.

[briansull]

With this change we create a better more accurate crossgen native image (when IBC data is present)
If no IBC data is present then this changeset is a no-opt.

I tried crossgen /verbose both without (baseline) and with this change and found that 58 additional methods are compiling into the native image. That represents 1.6% of the total methods compiled.
Since we are compiling 1.7% more methods it is expected that the total time taken would increase.

D:\fxkit\JitBench\src\MusicStore\pubdir.opt>findstr /c:"Compiling method" crossgen_opt.verbose.baseline.CodeAnalysis | wc
   3673   11019  303426

D:\fxkit\JitBench\src\MusicStore\pubdir.opt>findstr /c:"Compiling method" crossgen_opt.verbose.withchange.CodeAnalysis | wc
   3734   11202  307984

[briansull]

The (small) increase is expected as this change fixes an issue where we were failing to compile some methods that the IBC data specified should be compiled into the native image.

[briansull]

Additional methods that I see getting compiled are:

Compiling method Microsoft.CodeAnalysis.Syntax.InternalSyntax.GreenNodeExtensions.ToGreenList
Compiling method Microsoft.CodeAnalysis.Syntax.SyntaxListBuilderExtensions.ToSeparatedList
Compiling method Microsoft.CodeAnalysis.Diagnostics.DiagnosticAnalysisContextHelpers.VerifyAction
Compiling method Microsoft.CodeAnalysis.Diagnostics.DiagnosticAnalysisContextHelpers.VerifyAction
Compiling method PerAnalyzerState.FreeState_NoLock
Compiling method PerAnalyzerState.TryStartProcessingEntity
Compiling method PerAnalyzerState.IsEntityFullyProcessed
Compiling method Microsoft.CodeAnalysis.Diagnostics.DiagnosticAnalysisContextHelpers.VerifyAction
Compiling method Microsoft.CodeAnalysis.Diagnostics.DiagnosticAnalysisContextHelpers.VerifyAction
Compiling method Microsoft.CodeAnalysis.Diagnostics.DiagnosticAnalysisContextHelpers.VerifySyntaxKinds
Compiling method Roslyn.Utilities.EnumerableExtensions.SequenceEqual
Compiling method Microsoft.CodeAnalysis.Diagnostics.AsyncQueue`1[__Canon][System.__Canon].WithCancellation
Compiling method PerAnalyzerState.MarkEntityProcessed
Compiling method Microsoft.CodeAnalysis.GreenNodeExtensions.WithAdditionalAnnotationsGreen
Compiling method Microsoft.CodeAnalysis.Semantics.OperationWalker.VisitArray
Compiling method Microsoft.CodeAnalysis.Semantics.OperationWalker.VisitArray
Compiling method Microsoft.CodeAnalysis.ImmutableArrayExtensions.AsImmutableOrEmpty
Compiling method Extensions`1[__Canon][System.__Canon].CreateLanguageSpecificExtensions
Compiling method Roslyn.Utilities.Hash.CombineValues
Compiling method Roslyn.Utilities.EnumerableExtensions.IsEmpty
Compiling method Roslyn.Utilities.EnumerableExtensions.IsEmpty
Compiling method Microsoft.CodeAnalysis.GreenNodeExtensions.WithoutAnnotationsGreen
Compiling method Parser.CopyTo
Compiling method Microsoft.CodeAnalysis.Emit.DeltaMetadataWriter.AddDefIfNecessary
Compiling method Microsoft.CodeAnalysis.Emit.DeltaMetadataWriter.PopulateEncLogTableRows
Compiling method Microsoft.CodeAnalysis.Emit.DeltaMetadataWriter.PopulateEncLogTableEventsOrProperties
Compiling method Microsoft.CodeAnalysis.Emit.DeltaMetadataWriter.PopulateEncLogTableFieldsOrMethods
Compiling method Microsoft.CodeAnalysis.Emit.DeltaMetadataWriter.AddDefinitionTokens

etc...

[AndyAyersMS]

@briansull is there any way to verify that the set of methods that are now prejitted is the set we expect?

Assuming it is, there is no bug here -- the Roslyn assemblies we use in these tests have embedded IBC data, so we're seeing the impact of what is presumably a desirable IBC only fix.

[briansull]

Yes, There is no bug here, we now generate extra methods with IBC data.

With R2R It is not easy to verify that the exact set of methods we expect to get prejjited are in fact prejitted. Before R2R we were able to check that the JIT wasn't needed and and that every method needed was prejitted. We can't do that for R2R native images.

There are quite a few random reasons where we simply omit a method from a R2R native image and fall back to using the Jit.