Is there a way for libraries to provide profiles to binaries that depend on that library?

[kitlith]

Became aware of this tool due to someone evangalizing the use of PGO to various crates. One conversation that caught my eye was with a library author: The evangalist demonstrated 40% improvements in some cases, and the library author was interested, but the only known ways of applying the optimization were shipping prebuilt binaries, or modifying the source code to try and reproduce what PGO did. (conversation I'm referencing: tokio-rs/prost#1041 )

So, going back to the title of this discussion: is there a (possibility of a) mechanism that allows libraries to float profiles up to the binaries that use them?

[Kobzol]

That someone was probably https://github.com/zamazan4ik 😆

Well, in theory, it is possible. You can PGO instrument a library and then run a binary using that library on some workload (well, you would probably just PGO instrument the whole thing, as Rust libs and binaries are usually compiled together, see below). Then when you get the set of profiles, it should be possible to compile just the library using the profiles.

However, there are two issues with this approach. The first one is relatively straightforward and might not be such a big issue; libraries are by their definitions general, and they might be used in many different situations and use-cases. It might be thus a bit challenging to find good workloads that will consistently provide a speedup for a majority of users of said library (remember, when held wrong, PGO can also hamper performance, although it's quite rare in my experience).

The second issue is more problematic. Rust libraries are usually compiled together with the resulting binary, in the form of .rlibs. Therefore, there is no "precompiled library" that you could ship. I can imagine that this could work e.g. with C or C++, where shipping a .a or a .so file isn't unheard of, but in the Rust world, you're usually not shipping dynamic nor static libraries directly to consumers, as Rust (in)famously doesn't yet provide a stable ABI. In theory, you could do that; define e.g. a C API interface, PGO optimize the library and ship .so to your users. But it's probably too much of a hassle to be worth it.

What we could do instead is to design some general way of sharing profiles; if e.g. Cargo could download a PGO profile from crates.io, it could leverage it during the compilation of downstream binaries. But this is easier said than done; primarily because PGO profiles are usually anything but portable. Different OSes, different LLVM versions, pretty much different anything on your system might cause them to stop working. So distributing them to the world might require the design of some more general stable format, which is probably a lot of work.

[kitlith]

Yep, it was, as you can see by clicking on the discussion link.

design some general way of sharing profiles

Yeah, that's what I was trying to ask about.

Sounds like sharing profiles directly is currently a non-starter, so what about including metadata avout how to generate those profiles? i.e. the library crate would come with a piece of metadata that said "generate a profile from these specific binaries/tests/benchmarks" (whatever makes the most sense), and then the library profile could be generated on the end-users machine by cargo-pgo (by running a specific command? automatically?)

[Kobzol]

I guess it's not impossible, we could design some metadata in Cargo.toml and cargo-pgo could read it. Library crate authors would have to opt into it though, of course. One problem I see with this is that for PGO, you need to typically run your code on some workload, and workloads typically mean that you have some input (data). Getting this data to downstream users might be challenging.

Btw, could you tell me more about the use-case, like, why do you want to PGO optimize a library? Why can't you just compile your downstream binary with that library, then run your binary on some workload and let PGO do the rest? If your workload doesn't exercise the code of the library, then you don't need PGO for it. If it does, it will PGO optimize the library, and possibly even in a better way than if you tried to PGO optimize the library on some generic workloads.

[kitlith]

I don't have a usecase, myself, though I can invent a hypothetical. It'd probably be better to ask zamazan4ik. I was just curious after reading the conversation I linked.

The hypothetical here is that they can continue doing their thing, users of libraries see a "by the way, you can use cargo-pgo for a free speedup, and even more if you write something based on your own workload" somewhere in documentation, and that gets them started on using PGO at all. i.e. enable low-effort usage of generic PGO that other people have already put together, while retaining the ability to customize to their specific workload later.

This is all predicated on the idea that customized PGO > well-written generic PGO > no PGO, which might be a misconception. If it's actually rather difficult to generate a good generic PGO profile, then this isn't helpful.

[Kobzol]

I suppose that for some use-cases it might be fine to have a generic profile, while for some it can be difficult. I would personally say that for Rust, doing PGO for libraries pre-emptively is less useful than e.g. for C++, because there you often ship precompiled artifacts. While as we have already discussed, in Rust, you usually recompile everything from scratch, so if you want to use PGO, you can just do PGO on your whole binary.

In theory, if Cargo somehow did it fully automatically, then it could useful to optimize certain libraries even if the user doesn't even know what is PGO. But this would be a lot of complexity and would have to be opt-in because of the increased build time costs anyway (people might not want to automatically download PGO profiles from the internet without it being opt-in), so I think that it's just easier to spread awareness of PGO in general and let people use it on their binaries directly.