Repeated #[link] attributes for the same library cause it to be passed as many times to the linker.

[vadimcn]

If I specify #[link(name="pythonXY")] on 30 extern C blocks, and then compile with rustc -l dylib=pythonXY:python27, then python27.lib is passed to the linker 30 times.

In extreme cases this may cause trouble on Windows, where the maximum length of the command line is 32000 bytes. Also, may slow down linking.

[vadimcn]

As I mentioned on IRC, we could probably safely collapse repetitions of the same library. Beyond that, there's a risk of breaking someone.
Also, we can de-dupe libs passed to msvc linker, as it doesn't do left-to-right scanning and considers all libraries at once.

cc @retep998, @dgrunwald, @alexcrichton.

[retep998]

While we can de-dupe libraries passed to the msvc linker, we should still be careful to preserve the order of the first mention of any given library, because if a symbol can come from several libraries, link.exe will pick whichever library has it first.

[Ericson2314]

Item order is generally not something important in Rust. I hope with lld we can both deduplicate and ignore source order so only the set of names, and association between extern blocks and names, matter.

Can lld be used for windows too any time soon, because simply picking the first provider of a symbol would seem to to violate the mapping of extern blocks and lib names, unless I am missing something.

[retep998]

@Ericson2314 LLD has the same behavior as link.exe, so switching to LLD won't provide any benefits over just using link.exe for pc-windows-msvc. They both can resolve references in either direction, unlike ld which can only resolve references to symbols provided later. They both care about the order of two libraries foo and bar when they provide the same symbol. For both LLD and link.exe we can safely deduplicate input as long as we preserve the order of the first mention of each input, and they will behave identically.

[Ericson2314]

@retep998 So currently in all cases we cannot enforce that the symbol is linked from the requested block (except maybe macos where I hear unresolved symbols can mention a library name)? Bummer. IIRC the plan is to link lld into rustc. Perhaps that would make adding new functionality to lld to fix this issue more worthwhile. If what I said about macos is true, then perhaps lld already has some functionality for this under the hood as IIRC lld has mainly been used for that platform.

[retep998]

@Ericson2314 At the moment there is no way to choose which library a symbol gets resolved from except to make sure that library comes first. The only way you'd be able to have full explicit control is if LLD one day adds that feature.

[retep998]

@Ericson2314 When #58713 is implemented you will be able to guarantee a symbol comes from a certain library for a specific subset of cases.

[retep998]

A more extreme case of this issue found in the wild: MSxDOS/ntapi#2

[retep998]

Information on how linker order matters for msvc: https://docs.microsoft.com/en-us/cpp/build/reference/link-input-files?view=vs-2019

Object files on the command line are processed in the order they appear on the command line. Libraries are searched in command line order as well, with the following caveat: Symbols that are unresolved when bringing in an object file from a library are searched for in that library first, and then the following libraries from the command line and /DEFAULTLIB (Specify Default Library) directives, and then to any libraries at the beginning of the command line.

[nagisa]

This can cause linkage to fail entirely as described in #65847