-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cargo profile dependencies #2282
Conversation
55bc7db
to
81f3470
Compare
Hm, I would think Cargo should just rebuild everything in this case? And I even think this will happen naturally with the current implementation: IIRC, compiler flags are hashed into the fingerprint for determining if the crate should be rebuild. What is the purpose of profiles |
I think tracking changes to config files might be out of scope for cargo
If this is true then that works great!
Well, two purposes. Lets people specify global useful profiles like Also, it fixes the big build system case -- Having a custom profile for each combination of flags quickly leads to blowup. Firefox lets you individually flip debug symbols, debug-assertions, and optimization levels. There are more involved, though. (There are use cases; folks hacking on Rust code may want different settings from folks hacking on C++ code) Profiles aren't part of the package anyway, they're build info, and that really should go in |
Is this compatible with the profiles 2.0 proposal by @matklad ? IIRC there was an embargo for new features for profiles until they get revamped? Generally I like the idea of this RFC though. |
text/0000-custom-cargo-profiles.md
Outdated
It is not possible to have the same crate compiled in different modes as a build dependency and a regular dependency within the same profile. | ||
|
||
|
||
`build_override` is itself |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this an unfinished statement?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My bad, that line should be removed.
81f3470
to
ffb1862
Compare
@est31 I think this addresses the "some kind of DSL to map crates to flags" from profiles 2.0 This RFC does not attempt to handle the issue with test/bench/doc but I think that can be addressed independently. One solution for that would be closer to the "includeable profiles" proposal in the pre-RFC, and have something like |
@est31 I think of this RFC as basically profiles 2.0, and I really love it :) It doesn't solve the "which build in profile is run when" problem, but I think it can be addressed separately. Although I would prefer that instead of |
text/0000-custom-cargo-profiles.md
Outdated
|
||
We amend this to add that you can define custom profiles with the `profile.foo` key syntax. These can be invoked via | ||
`cargo build --profile foo`. The `dev`/`doc`/`bench`/etc profiles remain special. Each custom profile, aside from the | ||
"special" ones, gets a folder in `target/`, named after the profile. "dev" and "debug" are considered to be aliases |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Naming folders after profiles is no good idea because of profiles named del
or similar. Better is to name them as profile.foo
or sth.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Profiles affect only the leaf crate, so I think it's ok not to guard against bad names here. That is, even if one uploads crate with a forbidden profile name to crates.io, no bad things should happen.
Although, it might be a good idea to give a warning/error like "Hey, you named your profile NUL, and this will probably break your crate on windows".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, I agree with @matklad here. We can add warnings for badly-named profiles (doesn't need to be in the rfc)
Hash the profile data into the output path and you don't need to worry about undefined behavior on profile changes, I guess. |
One question I have with this is how this applies transitively. For example, consider crate |
@jonhoo currently, only the leaf crate's profiles are accounted for (it can even be argued on this basis that profiles should not live in Cargo.toml and should go into I can imagine an orthogonal RFC though, which allows upstream crate to advice some profile settings to the downstream crates. For example, the image crate could advise "compile me with -O 3 in the dev profile", and this setting would be used for crates which depend on image, unless they explicitly override it. The possible interaction between this RFC and hypothetical advise RFC is that it's not clear how upstream crates can advice on custom profiles (they don't even know their names!). I think it's reasonable to assume though that this is not a super important problem which we need to solve now: either of "advises do not work with custom profiles" or "custom profile can opt into advises from a particular build-in profile" seems like an ok solution. |
Also, cc @wycats for the whole discussion :) |
@matklad you're right that this could totally be addressed in a separate RFC. In fact, I wonder whether, if we had that RFC, that would alleviate many of the pains that this RFC tries to counter. The reason I bring this up as a concern here is that I worry that we'll see these extra profiles added all over the place as a result of this particular solution to the problem. Basically every crate that depends on On a somewhat separate tangent: how often do we expect users to actually add separate profiles, rather than just adding overrides for the |
@jonhoo FWIW this RFC does propose allowing profiles in .cargo/config, but it also allows them in Cargo.toml for consistency |
@jonhoo note that the RFC supports overrides for build-in profiles, so I don't believe that "every crate that depends on image would add this extra profile". Rather, every crate which depends on In general, I think the need to customize profiles should be rare, so I'd go for the increased flexibility rather than for increased convenience (but the syntax proposed in this RFC looks pretty and convenient).
Hm, interesting! I believe that it is indeed possible to split this RFC further into "overrides" and "custom profiles" parts, and I agree that "overrides" part has potentially many more users, because fully custom profiles a required mainly by large users like Servo. My personal preference would be to nevertheless pursue the custom profiles part. I really like how it creates a very simple mental model for profiles: "if you pass |
@rfcbot fcp merge |
Team member @matklad has proposed to merge this. The next step is review by the rest of the tagged teams: Concerns:
Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! See this document for info about what commands tagged team members can give me. |
@rust-lang/cargo discussed this RFC at today's meeting, and it looks really great! There are couple of implementation-level concerns/questions though. |
@rfcbot concern workspaces Cargo supports building several packages together as a part of the workspace. In workspaces, only the profile section from the root of the workspace (virtual or not) is taking an effect. Currently this works as expected, because profiles apply to all crates in the DAG of dependencies equally. However with this RFC there's an observable difference between the "current" crate and all other crates. How this interacts with multi-package workspaces. For example, if workspace consists of
what rule applies to |
@rfcbot concern build_override
I personally also think that maybe we should just always compile It would be great if someone could measure the impact of always-release |
uh oh it's the robot rebellion |
text/0000-profile-dependencies.md
Outdated
# Summary | ||
[summary]: #summary | ||
|
||
Allow overriding profile keys for certain dependencies, as well as providing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This sentence is :)
b3dcffd
to
e3f0e86
Compare
The final comment period is now complete. |
We've discussed the broader issue of workflows at today's meeting, here's the summary. Today's profiles represent two different things. The first one is the workflow, or operation, which Cargo currently is doing, like testing, documenting or benchmarking. The other, more narrow, is the set of "optimization flags" applied when building the code. It makes sense to split the current profile concept into two separate ones, by introducing workflow as a first-class thing for describing a particular set of operations cargo does, and scaling down profile concept to just mean the set of optimization flags. One can imagine that there will be roughly one workflow for each major Cargo command (testing workflow, documenting workflow, benchmarking workflow). Ideally, it should be possible to add custom workflows like a fuzzing workflow. In contrast, there should be relatively fewer profiles: Each workflow should be linked to some default profile, but the profile could be overridden via command line flag. For example, testing workflow would use Now, the interesting question is, which flags belong to profiles, and which flags belong to workflow? The interesting dimension for comparison here is "can I apply this flag to only part of the packages?". It is interesting because workflows are global for the whole project, while profiles, with this RFC, can apply only to the specific packages. However, certain optimization flags, like
Sorting the current flags gives this picture:
|
Why is codegen-units profile global? I think LTO, rpath, panic are all global, of which rpath/LTO are workflow-related ones, and panic is weird. We can amend this RFC so that LTO, rpath, panic can only be specified at the top level (we already do for panics) |
@matklad I'm confused as to what the status of this RFC is. Was your comment an indicator of the overall plans for profiles, or a request for amendments to this one (seems like they would be minor). The FCP has finished already. |
@Manishearth this was a comment about the general future direction of profiles, the RFC is great as it stands! I think it should be merged now, but I am not sure about the precise process to do this :) |
Cool. I think we're just supposed to rename the file and fill in details and then merge it. I'll let @nrc merge |
Updated |
Stabilize profile-overrides. This stabilizes the profile-overrides feature. This was proposed in [RFC 2282](rust-lang/rfcs#2282) and implemented in #5384. Tracking issue is rust-lang/rust#48683. This is intended to land in 1.41 which will reach the stable channel on Jan 30th. This includes a new documentation chapter on profiles. See the ["Overrides" section](https://github.com/rust-lang/cargo/blob/9c993a92ce33f219aaaed963bef51fc0f6a7677a/src/doc/src/reference/profiles.md#overrides) in `profiles.md` file for details on what is being stabilized. Note: The `config-profile` and `named-profiles` features are still unstable. Closes #6214 **Concerns** - There is some risk that `build-override` may be confusing with the [proposed future dedicated `build` profile](#6577). There is some more discussion about the differences at rust-lang/rust#48683 (comment). I don't expect it to be a significant drawback. If we proceed later with a dedicated `build` profile, I think we can handle explaining the differences in the documentation. (The linked PR is designed to work with profile-overrides.) - I don't anticipate any unexpected interactions with `config-profiles` or `named-profiles`. - Some of the syntax like `"*"` or `build-override` may not be immediately obvious what it means without reading the documentation. Nobody suggested any alternatives, though. - Configuring overrides for multiple packages is currently a pain, as you have to repeat the settings separately for each package. I propose that we can extend the syntax in the future to allow a comma-separated list of package names to alleviate this concern if it is deemed worthwhile. - The user may not know which packages to override, particularly for some dependencies that are split across multiple crates. I think, since this is an advanced feature, the user will likely be comfortable with using things like `cargo tree` to understand what needs to be overridden. There is [some discussion](rust-lang/rust#48683 (comment)) in the tracking issue about automatically including a package's dependencies, but this is somewhat complex. - There is some possibly confusing interaction with the test/bench profile. Because dependencies are always built with the dev/release profiles, overridding test/bench usually does not have an effect (unless specifying a workspace member that is being tested/benched). Overriding test/bench was previously prohibited, but was relaxed when named profiles were added. - We may want to allow overriding the `panic`, `lto`, and `rpath` settings in the future. I can imagine a case where someone has a large workspace, and wants to override those settings for just one package in the workspace. They are currently not allowed because it doesn't make sense to change those settings for rlibs, and `panic` usually needs to be in sync for the whole profile. - There are some strange interactions with `dylib` crates detailed in rust-lang/rust#64319. A fix was attempted, but later reverted. Since `dylib` crates are rare (this mostly applied to `libstd`), and a workaround was implemented for `build-std` (it no longer builds a dylib), I'm not too worried about this. - The interaction with `share-generics` can be quite confusing (see rust-lang/rust#63484). I added a section in the docs that tries to address this concern. It's also possible future versions of `rustc` may handle this better. - The new documentation duplicates some of the information in the rustc book. I think it's fine, as there are subtle differences, and it avoids needing to flip back and forth between the two books to learn what the settings do.
Stabilize config-profile. This is a proposal to stabilize config-profiles. This feature was proposed in [RFC 2282](rust-lang/rfcs#2282) and implemented in #5506. Tracking issue is rust-lang/rust#48683. This is intended to land in 1.43 which will reach the stable channel on April 23rd. This is a fairly straightforward extension of profiles where the exact same syntax from `Cargo.toml` can be specified in a config file. Environment variables are supported for everything except the `package` override table, where we do not support the ability to read arbitrary keys in the environment name.
Tracking issue: rust-lang/rust#48683
Rendered