Support static linking crates

[espindola]

Currently a crate is always compiled to a shared library. We should support both regular static linking and LTO. I think we should rename --shared to --lib. This is what it would look at:

*) rustc --lib --shared foo.rc -o foo.so
*) rustc --lib --static foo.rc -o foo.lib (really just a .o, but has a different ending)
*) rustc --lib --static --emit-llvm foo.rc -o foo.bclib (really just a .bc, but has a different ending)

rustc then looks for .lib and .bclib when resolving use.

[espindola]

A small change of plans:
Every regular .so and and .rlib has a llvm section that is loaded by rustc. This is nice for implementing inline functions. With this in place, lto is just a degenerated case where all of the crate is put as IL in a .rlib file.

[espindola]

just noticed that we don't need --shard and --static. --shared can just be the default.:

*) rustc --lib foo.rc -o foo.so
*) rustc --lib --static foo.rc -o foo.rlib (really just a .o, but has a different ending)
*) rustc --lib --static --emit-llvm foo.rc -o foo.rlib (same as above, but all the code is actually in the "inline" section)

[espindola]

Part of this is implemented now. Some open questions:

*) should we build a static rt library
*) should we build a static rustllvm library (or stop building it completely)
*) should -static refuse to use a shared library crate? (gcc -static does this)
*) should we pass -static to gcc if rustc is given -static?

[graydon]

I believe this has bitrotted some but it's important to support. Needs a new champion.

[graydon]

nominating for backwards compatible. it'll change enough about the command line and workspace that I think it's a BC risk.

[graydon]

accepted for feature-complete milestone

[catamorphism]

Visiting for triage. Still important; would be a great sorta-self-contained project for someone who's ready to jump in and sink their teeth into the compiler!

[catamorphism]

High, but not 1.0

[Aatch]

Probably related #9910

[Aatch]

I've started to fix up the current static linking code here: https://github.com/Aatch/rust/tree/static-link

That gets it back into a state where it will output a static crate. The code for loading a static crate was already there, so that works the same as before.

I'd be interested to know if what the requirements are, if any, for this before I go much further. I think that the original idea of steering away from the standard extension (e.g. .a) is a good one, since it is only a single object file, not a static archive of many objects.

One thing I ran into was handling executables, since they can link to both static and dynamic libraries. This currently means that an executable will only work if built with -Z static. I'm curious as to what a good way to deal with this might be.

[alexcrichton]

I was thinking about this the other day, and here were some thoughts that I had about this:

1. Why do we want to move away from the standard .a (.lib on windows) suffix? I would expect for embeddability that I could build a rust library and just link it into a C++ or C application (or any language really). I wouldn't relaly know what to do with this rlib file. I'm not entirely sure of the original motivations though, so perhaps there's a good reason for not doing this.
2. Right now link arguments are a mess across crates. If a dynamically linked library links to a static crate, then an executable linking to the dynamically linked library should not link to the static crate.
3. In theory a statically linked crate should pull in statically linked libraries. I figured that we would distribute lib{std, extra, rustc, syntax}.a, and nothing else. When compiling libstd, I don't think that we shouldn't need to distribute ibrustrt as a library, it should be bundled by libstd by default. I'm not sure if this is possible, but this would make distribution of static libraries a lot easier.
4. We should no longer be linking to librustrt dynamically, that should be statically linked to libstd. Furthermore, no one should be able to use the symbols from librustrt except for libstd.
5. All of the default linker arguments added in by back/link.rs should be moved into libstd because they're all implicit dependencies of the runtime, not of the crate itself.

I guess not all of these are entirely related to static linking, but it seems like we shouldn't get static linking just to get static linking working, but rather to maximize interoperability where static linking would be used. Right now I imagine that the use cases are

• Distributing static versions of the standard libraries (only these libraries, none of their dependencies).
• Linking with existing applications. You can't really expect other applications to start typing -lstd-hash-vers -Lpath/to/rustc/$target/lib/rustc/$target.

If we just get generation of libraries working, it's a good first step, but I think it's also important to not ignore these use cases.

[Aatch]

@alexcrichton For point 1 I do understand wanting to have a normal static archive, but we'd have to roll our own system for reading them on Linux anyway, since LLVM doesn't handle regular ar archives. It can probably handle windows static libraries though, I don't know enough about the windows linking model to say though.

That's the motivation for using rlib at any rate. Using a .a file would either mean implementing our own format handling for it or only using the .a extension for show and have it not actually work as a static library anyway.

I don't see the use case of people linking to a rust crate from a non-rust system being particularly common. While I agree we shouldn't exclude it, if it's slightly tricky to set up, I can live with that.

As for the rest of your points, they're kinda outside the scope of what I'm looking at right now, while I'm not just getting static linking to work for the sake of it, I'm not really thinking about the distribution of the core libraries at the moment.

[catamorphism]

@Aatch I noticed your recent blog post about linking, as well as this comment, and I wanted to mention that I'm planning on working on the rustc linker model in the immediate future. It sounds like my plans are orthogonal to yours (I'm going to be cleaning up link args, focusing on linking with native libraries), so there's still plenty of space for you to contribute. Just wanted to let you know I'm going to be working in that area, though :-)

[ludamad]

"I don't see the use case of people linking to a rust crate from a non-rust system being particularly common."

It's hard to say how common this would be given no major modern languages support this well. I have a hunch that there is a latent demand for libraries that are linkable to existing, large, C/C++ programs.

[omalley]

This would be a very important feature. One of the reasons people don't implement libraries in Rust is precisely because you can't create a static (or dynamic) library that can easily be linked into a C program.

[thehydroimpulse]

@omalley You should have said "One of the reasons people don't implement libraries in Rust that are to be used within C and C++ programs ..." Otherwise you're stating an absolute that is false, even though I know what you mean.

[steveklabnik]

@omalley can you expand more about the difficulty? This shouldn't actually be hard to do.

[alexcrichton]

One of the reasons people don't implement libraries in Rust is precisely because you can't create a static (or dynamic) library that can easily be linked into a C program.

This issue is quite old and this statement is no longer true. You can create a static rust library today to link to a C program (we have many tests exercising this feature!).

If you would like to create a static rust library, you should use the staticlib output format. That will give you an archive which you can then link into another C or C++ (or whatever language you want) application.