Add ability for crossgen2 to synthesize PGO histograms #77683
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ghost
assigned 
jakobbotsch
force-pushed
the
crossgen2-synthesized-pgo
branch
from
09a826a to
518d473
Compare
Add an option --synthesize-random-mibc. When passed, crossgen2 will use metadata from the compilation group to synthesize PGO histograms for methods that did not have any input profile data. Mainly for testing purposes. Currently only works in R2R mode, but hopefully will also work to embed random static PGO data in the future.
jakobbotsch
force-pushed
the
crossgen2-synthesized-pgo
branch
from
518d473 to
255a128
Compare
Split up "we have a profile" and "we have profile weights" questions.
Avoid ordering issues. Also revert now unrelated PhasedVar changes.
jakobbotsch
commented
Nov 1, 2022
Various places were setting this to "true" and various other places to "1". Always use "1" in the environment variable and "true" in the msbuild property. This fixes local runs of crossgen2 tests (using src/tests/run.cmd runcrossgen2tests) -- these were not skipping FileCheck tests before as the test wrapper script only checks for "1".
This was referenced Nov 4, 2022
This reverts commit a8ff57f.
jakobbotsch
commented
Nov 7, 2022
Comment on lines
-64
to
-66
|
|
||
| // R2R field layout needs compilation group information | ||
| ((ReadyToRunCompilerContext)context).SetCompilationGroup(group); |
There was a problem hiding this comment.
I've moved this so it happens right after we create the compilation group so it is early enough for the ProfileDataManager that also needs this now (for some EnsureLoadable calls).
jakobbotsch
commented
Nov 7, 2022
Comment on lines
+4
to
+5
| <!-- Synthesized PGO may cause the expected TypeLoadExceptions in this test to appear at unexpected places --> | ||
| <SynthesizedPgoIncompatible>true</SynthesizedPgoIncompatible> |
There was a problem hiding this comment.
This is probably an actual bug -- loading the type for the GDV guess can have observable behavior due to load failure, and does so in this test. Ideally we would have a "TypeHandle or null if not loadable" fixup we could use for GDV guesses, but for now I have just disabled this single test.
There was a problem hiding this comment.
Please file this issue as a bug and push for it to get fixed.
|
cc @dotnet/jit-contrib PTAL @davidwrighton |
...r/tools/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRun/MethodWithGCInfo.cs
Outdated
Show resolved
Hide resolved
|
/azp run runtime-coreclr outerloop |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add an option --synthesize-random-mibc. When passed, crossgen2 will use
metadata from the compilation group to synthesize PGO histograms for
methods that did not have any input profile data. Also supported for --embed-pgo-data,
in which case the synthesized PGO data gets embedded.
Adds a
synthesizepgoargument to thesrc/tests/run.cmd/src/tests/run.shscripts which enable this mode for compatible tests. So testing can be done locally via:This passes in pri1 on win-x64 for me currently. I have not yet hooked it up in CI. I want to do that in a separate change later, for now this is enough to allow me to do some manual testing on #77267 that has at least some level of coverage.
Some stats:
Ideally we can get the libraries tests running with crossgen'd libraries using synthesized PGO, but it will depend on #75230, so I will leave it to future work.