Replace the link step of codegen-units with ThinLTO

[pcwalton]

See: http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html

This is supposedly able to provide the optimization benefits of a normal release build while still enabling parallel codegen.

[nrc]

cc @michaelwoerister are you interested in looking into this?

[michaelwoerister]

Yes, I'm definitely interested in this.

[michaelwoerister]

And also in looking into this :)

[michaelwoerister]

Here's a little update on my findings so far:

• As the blog post describes, ThinLTO allows for cross-compilation unit inlining at a rather low cost.
• This would help us in 3 cases:
  • Regular compilation with multiple codegen units (leading to shorter compile-times with a supposedly small impact on runtime performance)
  • Incremental compilation: Here too we have multiple compilation units (many more than in the first case). If ThinLTO is fast enough we might be able to up codegen unit granularity for incr. comp. without the terrific impact on runtime performance. This might make incr.comp. a lot more efficient (although I can imagine that there's some negative interaction with debuginfo which might need to be duplicated a lot more with higher codegen unit granularity).
  • As an alternative to regular LTO in the cross-crate case. Here it would be just a cheaper way of getting 90% of what LTO does now. Maybe so cheap that it could be enabled by default. There might be some minor things that make this not trivial (like bitcode being compressed in our rlibs).

I haven't looked at the C/C++ API of this in detail yet but I imagine that we would use it somewhat like this:

1. Optimize each LLVM module that comes out of trans like we do now
2. Instruct LLVM to generate the ThinLTO metadata for each module.
3. Let LLVM build its global ThinLTO index from the modules we have generated so far and the modules from rlibs.
4. Run an additional optimization pass on each module that uses the ThinLTO index to do cross module inlining.
5. Run codegen to produce object files
6. Link

This is pretty much what we do now, just with steps (2), (3), and (4) inserted. I'm sure there are complications that will crop up but I don't see any conceptual obstacles.

cc @alexcrichton @brson @dotdash @Aatch (who might be interested in this LLVM stuff)

[alexcrichton]

Awesome, thanks for the update @michaelwoerister! That sounds like basically exactly what I would expect. I agree that if we get some good benchmarks we can probably turn parallel codegen on by default as well!

[alexcrichton]

This has been implemented, and I've opened up #45320 to enable it by default.