Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PGO profile invalidation in -Cprofile-use #100397

Open
Kobzol opened this issue Aug 11, 2022 · 3 comments · May be fixed by #100413
Open

PGO profile invalidation in -Cprofile-use #100397

Kobzol opened this issue Aug 11, 2022 · 3 comments · May be fixed by #100413
Labels
A-incr-comp Area: Incremental compilation C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@Kobzol
Copy link
Contributor

Kobzol commented Aug 11, 2022

Hi! While working on https://github.com/Kobzol/cargo-pgo, I noticed a peculiar thing about the invalidation of the PGO profile path.
When PGO is used for compiling, we specify the path to the PGO profile using -Cprofile-use=<path>. When <path> changes, it invalidates the compilation session, so the code is recompiled again (as it should be, if the profile changes).

However, if the path stays the same, but the contents of the profile file change, the code will not be recompiled. That can be a footgun, since users might try to gather profiles incrementally and then try to compile/benchmark using the new profiles, but if the profile path stays the same (if it's just overwritten each time), they might not notice that the code is not recompiled and they thus might get stale results.

Would it be feasible to check the hash/modification time of the profile file instead of just the file path, w.r.t. invalidating recompilation? Is there any prior art to this in some other Rust flags?

I think that hashing the file wouldn't be such a bottleneck (I think that it was done for sccache here), unless it would have to be done by every rustc invocation instead of just once per Cargo run? 🤔 I don't think that -Cprofile-use is that common that this would cause perf. problems, but it could resolve a potential footgun.

CC @michaelwoerister

@michaelwoerister
Copy link
Member

@Kobzol, I agree with your assessment of the problem. I don't think we have prior art here. Source files, upstream crate, and external libraries are all handled in a different way.

The most correct way to do this would be to generate and compare a hash in each rustc invocation. Comparing time stamps would be acceptable too, I think. The hash/timestamp could then be fed into the hash we already generate for commandline arguments.

@michaelwoerister michaelwoerister added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. A-incr-comp Area: Incremental compilation C-bug Category: This is a bug. labels Aug 11, 2022
@Kobzol
Copy link
Contributor Author

Kobzol commented Aug 11, 2022

Ok, I will try to send a PR to fix this. rustc is invoked by cargo once per crate, not once per source file, right? Otherwise the bottleneck might be a bit too much.

@michaelwoerister
Copy link
Member

Yes, once per crate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-incr-comp Area: Incremental compilation C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants