Consider using incremental linking with msvc #37543
Labels
A-linkage
Area: linking into static, shared libraries and binaries
C-enhancement
Category: An issue proposing an enhancement or a PR with one.
I-compiletime
Issue: Problems and improvements with respect to compile times.
O-windows-msvc
Toolchain: MSVC, Operating system: Windows
T-compiler
Relevant to the compiler team, which will review and decide on the PR/issue.
Comments
GuillaumeGomez
added
O-windows-msvc
|
This issue has been opened because the building times with rust-qt are quite high. Here's a simple example: Specs Cargo.toml [dependencies]
qt_core = "*"
qt_gui = "*"
qt_widgets = "*"Debug mode
The backlog is: Release mode
The backlog is: |
|
But the main problem is not the crazy times when doing a cold build on the qt crates but the crazy times it takes to build the simplest qt example, for instance, let's consider this dummy example: If i build and then If i do the same process for release And that's the main problem, you can't iterate with these crazy times for a trivial example. Hopefully all this info will help to improve rust. Amazing package manager, cool language syntax and features... but... if building times are that crazy I don't see which benefits would bring me to stay away from c++/qt or python/pyqt. Regards. |
|
I always envisioned that we would use any incremental linking functionality available when also compiling incrementally. That is certainly a good idea. Always using it also for regular builds is probably not a good idea because of binary sizes. Show we expose incremental linking as a separate The Gold linker also allows for incremental linking. cc @rust-lang/tools @rust-lang/compiler |
|
Seems plausible to me, I'd be fine just defaulting to using this with |
Mark-Simulacrum
added
the
C-enhancement
|
hey @alexcrichton Whats the update on this? |
|
Not much has happened on this AFAIK, but we can still implement it at any time! |
|
FWIW, here is the log of one of my normal Rust projects that's not using Qt or any huge lib, also after just cargo rustc -- -Z time-passesWin 8.1, rustc 1.26.0-nightly (2789b06 2018-03-06) The build before that one (also after just So, please at least allow running |
That sounds unrelated to this issue, consider opening a new one (in the cargo repository). |
|
@Boscop, are these timings with incremental compilation enabled? |
@retep998 @alexcrichton are these all generated in the target folder? |
|
@Jayflux I'm not sure, but the incremental folder can be assumed to be preserved and so intermediate artifacts can be stored somewhere in there. |
Unless ive got this wrong, this bit of work has already been done |
|
The incremental folder stores object files, but before linking they are copied to where the compiler would otherwise generate them. The final executable/dll is not touched by incremental compilation. I don't know if we have an explicit step in the compiler that would delete it before invoking the linker. |
|
@michaelwoerister Incremental is enabled by default, right? So then it must be enabled. |
|
@Boscop It's only the default for debug builds and If this is with incremental compilation enabled then this is some kind of bug or at least a severe corner case that we should investigate. Just |
|
I have this in my Cargo.toml: [profile.dev]
incremental = true
opt-level = 1Because otherwise it was too slow (for real-time audio) in debug mode and using too much cpu.. Is there a way I can keep the opt-level but make the compilation faster? |
|
@Boscop This should not be re-compiling so much with incremental compilation after just touching a file. Could you open an issue about this, if possible with a way to reproduce? I don't know of a way to make this faster. As long as incremental compilation doesn't work for your project, you should turn it off and use |
|
What is there to do in this issue? |
|
The long compilation times are still a problem |
|
On Servo my build is much faster after i set my local cargo to use lld-link.exe from LLVM. |
|
@jasonwilliams Did you download the win32 or win64 version? |
|
@Boscop i used the win32 version, any particular reason you ask? |
@michaelwoerister how can i reproduce this isssue? |
pnkfelix
added
the
A-linkage
jonas-schievink
added
I-compiletime
|
Any plans to still consider supporting this? Linking is still a pain point on incremental builds (often takes almost all the time), & it would be great if we could take advantage of this on windows. Trying to proxy project deps through a dylib is fairly cumbersome, as it quickly falls into you needing to pass -Cprefer-dynamic & then patch deps to other dylib proxies. I tried passing -Clink-args=/INCREMENTAL, but I'm not seeing any .ilk files in the build directory. (-Clink-args=/DEBUG:NONE does however help quite a bit by disabling the .pbds) |
|
Is lld-link.exe fast enough? My understanding is that modern (almost) fully parallel linkers like lld are often just as fast as incremental linking with old school (mostly) non-parallel linkers like link.exe. |
|
lld definitely seems faster than link. Though maybe the gap between them has been brought down [2019, 2021]? Here's some very rough numbers using a pretty contrived example that pulls in a bunch of deps. In my project I'm building a dylib that I then hot reload in my main program, so I'm seeing now that linking the dylib (along with -Zshare-generics=n -Cprefer-dynamic), appears to increase link times quite a bit vs a fully static lib: Machine: CPU=AMD 7700X, RAM=32GB 4400Mhz DDR5, SSD=RefS-formatted Samsung 990 Pro
With static lib:
Machine: CPU=i7-7700HQ, RAM=16GB 2400MHz DDR4, SSD=Samsung MZVLV256HCHP
I'm not sure how parallel lld is, eyeballing the task manager it only looked like 3 threads were getting used (vs what looked like 2 for link?), admittedly not very scientific on my end 😆. Curious on how It's a good point that it warrants seeing if link.exe + /INCREMENTAL is actually faster than lld. Looking online, I'm not really even seeing much info concerning link.exe full vs incremental other than https://devtalk.blender.org/t/speed-up-c-compilation/30508/14
If we also see a 3x improvement then that would be pretty great for incremental builds. On the other hand, it looks like incremental linking can also run into the problems reported in godotengine/godot#77968 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Incremental linking will allow for significantly faster link times, at the cost of binary size.
.ilkand.exeor.dllmust remain from the previous run./OPTto the linker or any other flag that is mutually exclusive with incremental linking./INCREMENTALto the linker.https://msdn.microsoft.com/en-us/library/4khtbfyf.aspx
Alternatively, provide LLD as a stable alternative to link.exe so users can benefit from faster links when they don't need any special features of link.exe.
The text was updated successfully, but these errors were encountered: