From ffb1862b3f0d22b3665b16a1cd3ff12dfb373792 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Mon, 8 Jan 2018 17:03:50 +0530 Subject: [PATCH 01/10] Initial RFC for custom cargo profiles --- text/0000-custom-cargo-profiles.md | 141 +++++++++++++++++++++++++++++ 1 file changed, 141 insertions(+) create mode 100644 text/0000-custom-cargo-profiles.md diff --git a/text/0000-custom-cargo-profiles.md b/text/0000-custom-cargo-profiles.md new file mode 100644 index 00000000000..b386de05428 --- /dev/null +++ b/text/0000-custom-cargo-profiles.md @@ -0,0 +1,141 @@ +- Feature Name: custom_cargo_profiles +- Start Date: 2018-01-08 +- RFC PR: (leave this empty) +- Rust Issue: (leave this empty) + + +# Summary +[summary]: #summary + +Add the ability to create custom profiles in Cargo.toml, to provide further control over how the project is built. Allow overriding profile keys for certain dependency trees. + +# Motivation +[motivation]: #motivation + +Currently the "stable" way to tweak build parameters like "debug symbols", "debug assertions", and "optimization level" is to edit Cargo.toml. + +This file is typically checked in tree, so for many projects overriding things involves making +temporary changes to this, which feels hacky. On top of this, if Cargo is being called by an +encompassing build system as what happens in Firefox, these changes can seem surprising. There are +currently two main profiles in Cargo ("dev" and "release"), and we're forced to fit everything we +need into these two categories. This isn't really enough. + +Furthermore, this doesn't allow for much customization. For example, when trying to optimize for +compilation speed by building in debug mode, build scripts will get built in debug mode as well. In +case of complex build-time dependencies like bindgen, this can end up significantly slowing down +compilation. It would be nice to be able to say "build in debug mode, but build build dependencies +in release". Also, your program may have large dependencies that it doesn't use in critical paths, +being able to ask for just these dependencies to be run in debug mode would be nice. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + + +Currently, the [Cargo guide has a section on this](http://doc.crates.io/manifest.html#the-profile-sections). + +We amend this to add that you can define custom profiles with the `profile.foo` key syntax. These can be invoked via +`cargo build --profile foo`. The `dev`/`doc`/`bench`/etc profiles remain special. Each custom profile, aside from the +"special" ones, gets a folder in `target/`, named after the profile. "dev" and "debug" are considered to be aliases + +Profile keys can be "overridden": + +```toml +[profile.dev] +opt-level = 0 +debug = true + +# the `image` crate will be compiled with -Copt-level=3 +[profile.dev.overrides.image] +opt-level = 3 + +# Dependencies semver-matching any entry in the space separated list +# will be compiled without debuginfo +[profile.dev.overrides."image=0.2 piston>5.0"] +debug=false + +# All dependencies (but not this crate itself) will be compiled +# with -Copt-level=2 . This includes build dependencies. +[profile.dev.overrides."*"] +opt-level = 2 + +# Build scripts and their dependencies will be compiled with -Copt-level=3 +# By default, build scripts use the same rules as the rest of the profile +[profile.dev.build_override] +opt-level = 3 +``` + +Custom profiles _can_ be listed in a `.cargo/config`, however the user is responsible for +clearing up build directories if the profile changes. That is, it is undefined behavior +to run `cargo build --profile foo` if `foo` has been defined in `.cargo/config` and the +profile has been edited since the last time you ran `cargo build --profile foo`. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +In case of overlapping rules, the last mentioned rule will be applied. This applies to build scripts +as well; if, for example, you have the following profile: + +```toml +[profile.dev] +opt-level = 0 +[profile.dev.build_override] +opt-level = 3 +``` + +and the `image` crate is _both_ a build dependency and a regular dependency; it will be compiled +as per the `build_override` rule. If you wish it to be compiled as per the original rule, +use a normal override rule: + +```toml +[profile.dev] +opt-level = 0 +[profile.dev.build_override] +opt-level = 3 +[profile.dev.overrides.image] +opt-level = 0 +``` + +It is not possible to have the same crate compiled in different modes as a build dependency and a regular dependency within the same profile. + + +`cargo build --target foo` will fail to run if `foo` clashes with the name of a profile; so avoid +giving profiles the same name as possible build targets. + + +# Drawbacks +[drawbacks]: #drawbacks + +This complicates cargo. + +# Rationale and alternatives +[alternatives]: #alternatives + +There are really two or three concerns here: + + - A stable interface for setting various profile keys (`cargo rustc -- -Clto` is not good, for example, and doesn't integrate into Cargo's target directories) + - The ability to use a different profile for build scripts (usually, the ability to flip optimization modes; I don't think folks care as much about `-g` in build scripts) + - The ability to use a different profile for specific dependencies + +The first one can be resolved partially by stabilizing `cargo` arguments for overriding these. It +doesn't fix the target directory issue, but that might not be a major concern. Allowing profiles to +come from `.cargo/config` is another minimal solution to this for use cases like Firefox, which +wraps Cargo in another build system. + +The second one can be fixed with a specific `build-scripts = release` key for profiles. + +The third can't be as easily fixed, however it's not clear if that's a major need. + +The nice thing about this proposal is that it is able to handle all three of these concerns. However, separate RFCs for separate features could be introduced as well. + +In general there are plans for Cargo to support other build systems by making it more modular (so +that you can ask it for a build plan and then execute it yourself). Such build systems would be able to +provide the ability to override profiles themselves instead. It's unclear if the general Rust +community needs the ability to override profiles. + +# Unresolved questions +[unresolved]: #unresolved-questions + +- Bikeshedding the naming of the keys +- The priority order when doing resolution +- Should `build_override` itself take an `overrides.foo` key? +- The current proposal provides a way to say "special-case all build dependencies, even if they are regular dependencies as well", but not "special-case all build-only dependencies" (which can be solved with a `!build_override` thing, but that's weird and unweildy) \ No newline at end of file From de19a1b8326493869fecd8d331e50146020411e9 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Fri, 2 Feb 2018 13:40:00 +0530 Subject: [PATCH 02/10] Make cargo clobber builds for .cargo/config profile changes --- text/0000-custom-cargo-profiles.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/text/0000-custom-cargo-profiles.md b/text/0000-custom-cargo-profiles.md index b386de05428..2e7807abd0f 100644 --- a/text/0000-custom-cargo-profiles.md +++ b/text/0000-custom-cargo-profiles.md @@ -64,10 +64,8 @@ opt-level = 2 opt-level = 3 ``` -Custom profiles _can_ be listed in a `.cargo/config`, however the user is responsible for -clearing up build directories if the profile changes. That is, it is undefined behavior -to run `cargo build --profile foo` if `foo` has been defined in `.cargo/config` and the -profile has been edited since the last time you ran `cargo build --profile foo`. +Custom profiles _can_ be listed in a `.cargo/config`. When building cargo will calculate +the current profile, and if it has changed, it will do a fresh/clean build. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation From 7271006bf64c228f8751252d0707a850248fa322 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Fri, 2 Feb 2018 15:04:39 +0530 Subject: [PATCH 03/10] Specify interaction with workspaces --- text/0000-custom-cargo-profiles.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/0000-custom-cargo-profiles.md b/text/0000-custom-cargo-profiles.md index 2e7807abd0f..86c2d9fed76 100644 --- a/text/0000-custom-cargo-profiles.md +++ b/text/0000-custom-cargo-profiles.md @@ -100,6 +100,9 @@ It is not possible to have the same crate compiled in different modes as a build giving profiles the same name as possible build targets. +When in a workspace, `"*"` will apply to all dependencies that are _not_ workspace members, you can explicitly +apply things to workspace members with `[profile.dev.overrides.membername]`. + # Drawbacks [drawbacks]: #drawbacks From 9aae9d5c5dd6bb6f9894944dbbc288e79ebc8b2f Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Fri, 2 Feb 2018 17:46:12 +0530 Subject: [PATCH 04/10] Change priority ordering --- text/0000-custom-cargo-profiles.md | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/text/0000-custom-cargo-profiles.md b/text/0000-custom-cargo-profiles.md index 86c2d9fed76..bf51cd4acd3 100644 --- a/text/0000-custom-cargo-profiles.md +++ b/text/0000-custom-cargo-profiles.md @@ -70,8 +70,12 @@ the current profile, and if it has changed, it will do a fresh/clean build. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation -In case of overlapping rules, the last mentioned rule will be applied. This applies to build scripts -as well; if, for example, you have the following profile: +In case of overlapping rules, the precedence order is that `overrides.foo` +will win over `overrides."*"` and both will win over `build_override`. + +So if you specify `build_override` +it will not affect the compilation of any dependencies which are both +build-dependencies and regular dependencies. If you have ```toml [profile.dev] @@ -81,7 +85,7 @@ opt-level = 3 ``` and the `image` crate is _both_ a build dependency and a regular dependency; it will be compiled -as per the `build_override` rule. If you wish it to be compiled as per the original rule, +as per the top level `opt-level=0` rule. If you wish it to be compiled as per the build_override rule, use a normal override rule: ```toml @@ -90,16 +94,21 @@ opt-level = 0 [profile.dev.build_override] opt-level = 3 [profile.dev.overrides.image] -opt-level = 0 +opt-level = 3 ``` -It is not possible to have the same crate compiled in different modes as a build dependency and a regular dependency within the same profile. +This clash may not occur whilst cross compiling since two separate versions of the crate will be compiled. +(This RFC leaves the decision of whether or not to handle this up to the implementors) +It is not possible to have the same crate compiled in different modes as a build dependency and a +regular dependency within the same profile when not cross compiling. (This is a current limitation +in Cargo, but it would be nice if we could fix this) + +Put succinctly, `build_override` is not able to affect anything compiled into the final binary. `cargo build --target foo` will fail to run if `foo` clashes with the name of a profile; so avoid giving profiles the same name as possible build targets. - When in a workspace, `"*"` will apply to all dependencies that are _not_ workspace members, you can explicitly apply things to workspace members with `[profile.dev.overrides.membername]`. @@ -138,5 +147,4 @@ community needs the ability to override profiles. - Bikeshedding the naming of the keys - The priority order when doing resolution -- Should `build_override` itself take an `overrides.foo` key? - The current proposal provides a way to say "special-case all build dependencies, even if they are regular dependencies as well", but not "special-case all build-only dependencies" (which can be solved with a `!build_override` thing, but that's weird and unweildy) \ No newline at end of file From 7f8c00d9786dd5526856119990ab11ec38cd38c4 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Fri, 2 Feb 2018 18:57:42 +0530 Subject: [PATCH 05/10] Disallow panic=foo in overrides --- text/0000-custom-cargo-profiles.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/0000-custom-cargo-profiles.md b/text/0000-custom-cargo-profiles.md index bf51cd4acd3..7541c60035f 100644 --- a/text/0000-custom-cargo-profiles.md +++ b/text/0000-custom-cargo-profiles.md @@ -112,6 +112,9 @@ giving profiles the same name as possible build targets. When in a workspace, `"*"` will apply to all dependencies that are _not_ workspace members, you can explicitly apply things to workspace members with `[profile.dev.overrides.membername]`. +The `panic` key cannot be specified in an override; only in the top level of a profile. Rust does not allow +the linking together of crates with different `panic` settings. + # Drawbacks [drawbacks]: #drawbacks From e3b9483bf4afa8d7e0eba4b1477d1922cb420cfc Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Fri, 2 Feb 2018 23:53:20 +0530 Subject: [PATCH 06/10] Mention panic modes --- text/0000-custom-cargo-profiles.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/text/0000-custom-cargo-profiles.md b/text/0000-custom-cargo-profiles.md index 7541c60035f..dac16d7d97b 100644 --- a/text/0000-custom-cargo-profiles.md +++ b/text/0000-custom-cargo-profiles.md @@ -150,4 +150,8 @@ community needs the ability to override profiles. - Bikeshedding the naming of the keys - The priority order when doing resolution -- The current proposal provides a way to say "special-case all build dependencies, even if they are regular dependencies as well", but not "special-case all build-only dependencies" (which can be solved with a `!build_override` thing, but that's weird and unweildy) \ No newline at end of file +- The current proposal provides a way to say "special-case all build dependencies, even if they are regular dependencies as well", + but not "special-case all build-only dependencies" (which can be solved with a `!build_override` thing, but that's weird and unweildy) +- It would be nice to have a way for crates to _declare_ that they use a particular + panic mode (something like `allow-panic=all` vs `allow-panic=abort`/`allow_panic=unwind`, with `all` as default) + so that they can assume a panic mode and cargo will refuse to compile them with anything else From 492e63b5bfd7ebd8b83aae102ce9641e32b21a3e Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Fri, 16 Feb 2018 11:03:06 -0800 Subject: [PATCH 07/10] Remove semver selection --- text/0000-custom-cargo-profiles.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/text/0000-custom-cargo-profiles.md b/text/0000-custom-cargo-profiles.md index dac16d7d97b..333fba12782 100644 --- a/text/0000-custom-cargo-profiles.md +++ b/text/0000-custom-cargo-profiles.md @@ -48,11 +48,6 @@ debug = true [profile.dev.overrides.image] opt-level = 3 -# Dependencies semver-matching any entry in the space separated list -# will be compiled without debuginfo -[profile.dev.overrides."image=0.2 piston>5.0"] -debug=false - # All dependencies (but not this crate itself) will be compiled # with -Copt-level=2 . This includes build dependencies. [profile.dev.overrides."*"] From ce1568437c7eae540b2b17a830b2885bb29e2ef5 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Fri, 16 Feb 2018 11:07:26 -0800 Subject: [PATCH 08/10] remove custom profiles --- ...ofiles.md => 0000-profile-dependencies.md} | 21 +++++++------------ 1 file changed, 8 insertions(+), 13 deletions(-) rename text/{0000-custom-cargo-profiles.md => 0000-profile-dependencies.md} (84%) diff --git a/text/0000-custom-cargo-profiles.md b/text/0000-profile-dependencies.md similarity index 84% rename from text/0000-custom-cargo-profiles.md rename to text/0000-profile-dependencies.md index 333fba12782..1462af40d4c 100644 --- a/text/0000-custom-cargo-profiles.md +++ b/text/0000-profile-dependencies.md @@ -1,4 +1,4 @@ -- Feature Name: custom_cargo_profiles +- Feature Name: profile_dependencies - Start Date: 2018-01-08 - RFC PR: (leave this empty) - Rust Issue: (leave this empty) @@ -7,7 +7,7 @@ # Summary [summary]: #summary -Add the ability to create custom profiles in Cargo.toml, to provide further control over how the project is built. Allow overriding profile keys for certain dependency trees. +Allow overriding profile keys for certain dependencies, as well as providing # Motivation [motivation]: #motivation @@ -16,28 +16,23 @@ Currently the "stable" way to tweak build parameters like "debug symbols", "debu This file is typically checked in tree, so for many projects overriding things involves making temporary changes to this, which feels hacky. On top of this, if Cargo is being called by an -encompassing build system as what happens in Firefox, these changes can seem surprising. There are -currently two main profiles in Cargo ("dev" and "release"), and we're forced to fit everything we -need into these two categories. This isn't really enough. +encompassing build system as what happens in Firefox, these changes can seem surprising. -Furthermore, this doesn't allow for much customization. For example, when trying to optimize for +This also doesn't allow for much customization. For example, when trying to optimize for compilation speed by building in debug mode, build scripts will get built in debug mode as well. In case of complex build-time dependencies like bindgen, this can end up significantly slowing down compilation. It would be nice to be able to say "build in debug mode, but build build dependencies in release". Also, your program may have large dependencies that it doesn't use in critical paths, being able to ask for just these dependencies to be run in debug mode would be nice. + # Guide-level explanation [guide-level-explanation]: #guide-level-explanation Currently, the [Cargo guide has a section on this](http://doc.crates.io/manifest.html#the-profile-sections). -We amend this to add that you can define custom profiles with the `profile.foo` key syntax. These can be invoked via -`cargo build --profile foo`. The `dev`/`doc`/`bench`/etc profiles remain special. Each custom profile, aside from the -"special" ones, gets a folder in `target/`, named after the profile. "dev" and "debug" are considered to be aliases - -Profile keys can be "overridden": +We amend this to add that you can override dependency configurations via `profile.foo.overrides`: ```toml [profile.dev] @@ -59,8 +54,8 @@ opt-level = 2 opt-level = 3 ``` -Custom profiles _can_ be listed in a `.cargo/config`. When building cargo will calculate -the current profile, and if it has changed, it will do a fresh/clean build. +Additionally, profiles may be listed in `.cargo/config`. When building, cargo will calculate the +current profile, and if it has changed, it will do a fresh/clean build. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation From e3f0e86196b0f9d344c7686904cae48919c57099 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Wed, 21 Feb 2018 07:51:18 -0800 Subject: [PATCH 09/10] .cargo/config --- text/0000-profile-dependencies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-profile-dependencies.md b/text/0000-profile-dependencies.md index 1462af40d4c..4db9ff0902f 100644 --- a/text/0000-profile-dependencies.md +++ b/text/0000-profile-dependencies.md @@ -7,7 +7,7 @@ # Summary [summary]: #summary -Allow overriding profile keys for certain dependencies, as well as providing +Allow overriding profile keys for certain dependencies, as well as providing a way to set profiles in `.cargo/config` # Motivation [motivation]: #motivation From a52f15bbabc668ecb8915130f38d9046fa585617 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Fri, 2 Mar 2018 12:25:01 -0800 Subject: [PATCH 10/10] RFC 2282 is Cargo Profile Dependencies --- ...-profile-dependencies.md => 2282-profile-dependencies.md} | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) rename text/{0000-profile-dependencies.md => 2282-profile-dependencies.md} (97%) diff --git a/text/0000-profile-dependencies.md b/text/2282-profile-dependencies.md similarity index 97% rename from text/0000-profile-dependencies.md rename to text/2282-profile-dependencies.md index 4db9ff0902f..7c2dcc0f7ea 100644 --- a/text/0000-profile-dependencies.md +++ b/text/2282-profile-dependencies.md @@ -1,7 +1,7 @@ - Feature Name: profile_dependencies - Start Date: 2018-01-08 -- RFC PR: (leave this empty) -- Rust Issue: (leave this empty) +- RFC PR: [rust-lang/rfcs#2282](https://github.com/rust-lang/rfcs/pull/2282) +- Rust Issue: [rust-lang/rust#48683](https://github.com/rust-lang/rust/issues/48683) # Summary @@ -139,7 +139,6 @@ community needs the ability to override profiles. [unresolved]: #unresolved-questions - Bikeshedding the naming of the keys -- The priority order when doing resolution - The current proposal provides a way to say "special-case all build dependencies, even if they are regular dependencies as well", but not "special-case all build-only dependencies" (which can be solved with a `!build_override` thing, but that's weird and unweildy) - It would be nice to have a way for crates to _declare_ that they use a particular