Refactor `RustTarget` #2993

pvdrz · 2024-11-27T19:19:23Z

This PR refactors the RustTarget type to be more flexible.

The motivation behind this change comes from the current definition of RustTarget as an enum. Which forces users to pick a specific version of Rust that might not match the msrv of their projects exactly.

As an example, let's say that a project using bindgen has an msrv of 1.41.1 and they decide to use the --rust-target flag to guarantee that bindgen won't produce code that breaks their msrv.

Given that bindgen doesn't have a RustTarget variant that specifically matches 1.41.1, they have to use the closest RustTarget that is still compatible. In this case that would be 1.40. If they decide to change their msrv in the future, they have to repeat this search process with their new msrv.

With the changes done in this PR, it would be possible to pass any* Rust version to --rust-target and bindgen will automatically enable the Rust features that are compatible with that version. In our example, that means that it would be possible to use --rust-version=1.41.1 explicitly, or even better, read the value from rust-version so they don't have to manually change it in the future.

The specific changes done in this PR are:

Replace the RustTarget enum for an opaque struct that can represent any* Rust version.
Update the FromStr implementation for RustTarget so it can parse any* Rust version.
Introduce several deprecated constants under RustTarget so people can still use those as if they were still using the variants of the removed enum type.
Introduce the builders RustTarget::nightly and RustTarget::stable.
Remove all the deprecated RustTargets.
Update the expectation tests to accommodate the removal of the deprecated targets.

(*): Unless it's not supported by bindgen anymore. To the date, anything earlier than 1.33 (inclusive)

cc @ojeda

ojeda · 2024-11-28T10:26:55Z

Thanks for this, Christian!

Just to confirm: for projects that need to support bindgen releases with and without this PR, it is still possible to pass their current (old) --rust-target, i.e. all previous valid inputs will still be valid ones with the PR and do the same thing, right? (I see the parsing allows minor only, i.e. 1.x, so I think that is the intention, which is great.)

pvdrz · 2024-11-28T14:16:11Z

Thanks for this, Christian!

Just to confirm: for projects that need to support bindgen releases with and without this PR, it is still possible to pass their current (old) --rust-target, i.e. all previous valid inputs will still be valid ones with the PR and do the same thing, right? (I see the parsing allows minor only, i.e. 1.x, so I think that is the intention, which is great.)

Yes all the previous valid inputs are still valid, e.g. 1.33 gets parsed as 1.33.0 and so on. The only breaking change I expect to see here (other than the removal of deprecated targets) is if anyone was trying to match the RustTarget enum, as their matches would no longer be exhaustive

ojeda · 2024-11-28T17:23:48Z

Sounds great, thanks for confirming!

The RustTarget part wouldn't affect the kernel since we use the CLI so far. Regarding the match, it is true that they wouldn't be able to simply match all without a wildcard; however, in that case, they needed to add new variants anyway when new releases added a new target version, no?

emilio

Would've been nice to send the removal of deprecated targets separately, but looks good! Some minor bits only.

bindgen/features.rs

bindgen-tests/tests/headers/layout.h

This is done because bindgen is producing bogus code where a single struct has both `packed` and `align` attributes.

pvdrz · 2024-11-28T19:23:36Z

Sounds great, thanks for confirming!

The RustTarget part wouldn't affect the kernel since we use the CLI so far. Regarding the match, it is true that they wouldn't be able to simply match all without a wildcard; however, in that case, they needed to add new variants anyway when new releases added a new target version, no?

Yes, they'd have to add extra match arms everytime we added more variants to RustTarget

Each `bindgen` release may upgrade the list of Rust targets. For instance, currently, in their master branch [1], the latest ones are: Nightly => { vectorcall_abi: #124485, ptr_metadata: #81513, layout_for_ptr: #69835, }, Stable_1_77(77) => { offset_of: #106655 }, Stable_1_73(73) => { thiscall_abi: #42202 }, Stable_1_71(71) => { c_unwind_abi: #106075 }, Stable_1_68(68) => { abi_efiapi: #105795 }, By default, the highest stable release in their list is used, and users are expected to set one if they need to support older Rust versions (e.g. see [2]). Thus, over time, new Rust features are used by default, and at some point, it is likely that `bindgen` will emit Rust code that requires a Rust version higher than our minimum (or perhaps enabling an unstable feature). Currently, there is no problem because the maximum they have, as seen above, is Rust 1.77.0, and our current minimum is Rust 1.78.0. Therefore, set a Rust target explicitly now to prevent going forward in time too much and thus getting potential build failures at some point. Since we also support a minimum `bindgen` version, and since `bindgen` does not support passing unknown Rust target versions, we need to use the list of our minimum `bindgen` version, rather than the latest. So, since `bindgen` 0.65.1 had this list [3], we need to use Rust 1.68.0: /// Rust stable 1.64 /// * `core_ffi_c` ([Tracking issue](rust-lang/rust#94501)) => Stable_1_64 => 1.64; /// Rust stable 1.68 /// * `abi_efiapi` calling convention ([Tracking issue](rust-lang/rust#65815)) => Stable_1_68 => 1.68; /// Nightly rust /// * `thiscall` calling convention ([Tracking issue](rust-lang/rust#42202)) /// * `vectorcall` calling convention (no tracking issue) /// * `c_unwind` calling convention ([Tracking issue](rust-lang/rust#74990)) => Nightly => nightly; ... /// Latest stable release of Rust pub const LATEST_STABLE_RUST: RustTarget = RustTarget::Stable_1_68; Thus add the `--rust-target 1.68` parameter. Add a comment as well explaining this. An alternative would be to use the currently running (i.e. actual) `rustc` and `bindgen` versions to pick a "better" Rust target version. However, that would introduce more moving parts depending on the user setup and is also more complex to implement. Starting with `bindgen` 0.71.0 [4], we will be able to set any future Rust version instead, i.e. we will be able to set here our minimum supported Rust version. Christian implemented it [5] after seeing this patch. Thanks! Cc: stable@vger.kernel.org # 6.12.y only (since older LTSs only support a single `bindgen` version) Cc: Christian Poveda <git@pvdrz.com> Cc: Emilio Cobos Álvarez <emilio@crisal.io> Link: https://github.com/rust-lang/rust-bindgen/blob/21c60f473f4e824d4aa9b2b508056320d474b110/bindgen/features.rs#L97-L105 [1] Link: rust-lang/rust-bindgen#2960 [2] Link: https://github.com/rust-lang/rust-bindgen/blob/7d243056d335fdc4537f7bca73c06d01aae24ddc/bindgen/features.rs#L131-L150 [3] Link: https://github.com/rust-lang/rust-bindgen/blob/main/CHANGELOG.md#0710-2024-12-06 [4] Link: rust-lang/rust-bindgen#2993 [5] Reviewed-by: Alice Ryhl <aliceryhl@google.com> Link: https://lore.kernel.org/r/20241123180323.255997-1-ojeda@kernel.org Signed-off-by: Miguel Ojeda <ojeda@kernel.org>

Each `bindgen` release may upgrade the list of Rust targets. For instance, currently, in their master branch [1], the latest ones are: Nightly => { vectorcall_abi: #124485, ptr_metadata: #81513, layout_for_ptr: #69835, }, Stable_1_77(77) => { offset_of: #106655 }, Stable_1_73(73) => { thiscall_abi: #42202 }, Stable_1_71(71) => { c_unwind_abi: #106075 }, Stable_1_68(68) => { abi_efiapi: #105795 }, By default, the highest stable release in their list is used, and users are expected to set one if they need to support older Rust versions (e.g. see [2]). Thus, over time, new Rust features are used by default, and at some point, it is likely that `bindgen` will emit Rust code that requires a Rust version higher than our minimum (or perhaps enabling an unstable feature). Currently, there is no problem because the maximum they have, as seen above, is Rust 1.77.0, and our current minimum is Rust 1.78.0. Therefore, set a Rust target explicitly now to prevent going forward in time too much and thus getting potential build failures at some point. Since we also support a minimum `bindgen` version, and since `bindgen` does not support passing unknown Rust target versions, we need to use the list of our minimum `bindgen` version, rather than the latest. So, since `bindgen` 0.65.1 had this list [3], we need to use Rust 1.68.0: /// Rust stable 1.64 /// * `core_ffi_c` ([Tracking issue](rust-lang/rust#94501)) => Stable_1_64 => 1.64; /// Rust stable 1.68 /// * `abi_efiapi` calling convention ([Tracking issue](rust-lang/rust#65815)) => Stable_1_68 => 1.68; /// Nightly rust /// * `thiscall` calling convention ([Tracking issue](rust-lang/rust#42202)) /// * `vectorcall` calling convention (no tracking issue) /// * `c_unwind` calling convention ([Tracking issue](rust-lang/rust#74990)) => Nightly => nightly; ... /// Latest stable release of Rust pub const LATEST_STABLE_RUST: RustTarget = RustTarget::Stable_1_68; Thus add the `--rust-target 1.68` parameter. Add a comment as well explaining this. An alternative would be to use the currently running (i.e. actual) `rustc` and `bindgen` versions to pick a "better" Rust target version. However, that would introduce more moving parts depending on the user setup and is also more complex to implement. Starting with `bindgen` 0.71.0 [4], we will be able to set any future Rust version instead, i.e. we will be able to set here our minimum supported Rust version. Christian implemented it [5] after seeing this patch. Thanks! Cc: Christian Poveda <git@pvdrz.com> Cc: Emilio Cobos Álvarez <emilio@crisal.io> Cc: stable@vger.kernel.org # needed for 6.12.y; unneeded for 6.6.y; do not apply to 6.1.y Fixes: c844fa6 ("rust: start supporting several `bindgen` versions") Link: https://github.com/rust-lang/rust-bindgen/blob/21c60f473f4e824d4aa9b2b508056320d474b110/bindgen/features.rs#L97-L105 [1] Link: rust-lang/rust-bindgen#2960 [2] Link: https://github.com/rust-lang/rust-bindgen/blob/7d243056d335fdc4537f7bca73c06d01aae24ddc/bindgen/features.rs#L131-L150 [3] Link: https://github.com/rust-lang/rust-bindgen/blob/main/CHANGELOG.md#0710-2024-12-06 [4] Link: rust-lang/rust-bindgen#2993 [5] Reviewed-by: Alice Ryhl <aliceryhl@google.com> Link: https://lore.kernel.org/r/20241123180323.255997-1-ojeda@kernel.org Signed-off-by: Miguel Ojeda <ojeda@kernel.org>

commit 7a5f93e upstream. Each `bindgen` release may upgrade the list of Rust targets. For instance, currently, in their master branch [1], the latest ones are: Nightly => { vectorcall_abi: #124485, ptr_metadata: #81513, layout_for_ptr: #69835, }, Stable_1_77(77) => { offset_of: #106655 }, Stable_1_73(73) => { thiscall_abi: #42202 }, Stable_1_71(71) => { c_unwind_abi: #106075 }, Stable_1_68(68) => { abi_efiapi: #105795 }, By default, the highest stable release in their list is used, and users are expected to set one if they need to support older Rust versions (e.g. see [2]). Thus, over time, new Rust features are used by default, and at some point, it is likely that `bindgen` will emit Rust code that requires a Rust version higher than our minimum (or perhaps enabling an unstable feature). Currently, there is no problem because the maximum they have, as seen above, is Rust 1.77.0, and our current minimum is Rust 1.78.0. Therefore, set a Rust target explicitly now to prevent going forward in time too much and thus getting potential build failures at some point. Since we also support a minimum `bindgen` version, and since `bindgen` does not support passing unknown Rust target versions, we need to use the list of our minimum `bindgen` version, rather than the latest. So, since `bindgen` 0.65.1 had this list [3], we need to use Rust 1.68.0: /// Rust stable 1.64 /// * `core_ffi_c` ([Tracking issue](rust-lang/rust#94501)) => Stable_1_64 => 1.64; /// Rust stable 1.68 /// * `abi_efiapi` calling convention ([Tracking issue](rust-lang/rust#65815)) => Stable_1_68 => 1.68; /// Nightly rust /// * `thiscall` calling convention ([Tracking issue](rust-lang/rust#42202)) /// * `vectorcall` calling convention (no tracking issue) /// * `c_unwind` calling convention ([Tracking issue](rust-lang/rust#74990)) => Nightly => nightly; ... /// Latest stable release of Rust pub const LATEST_STABLE_RUST: RustTarget = RustTarget::Stable_1_68; Thus add the `--rust-target 1.68` parameter. Add a comment as well explaining this. An alternative would be to use the currently running (i.e. actual) `rustc` and `bindgen` versions to pick a "better" Rust target version. However, that would introduce more moving parts depending on the user setup and is also more complex to implement. Starting with `bindgen` 0.71.0 [4], we will be able to set any future Rust version instead, i.e. we will be able to set here our minimum supported Rust version. Christian implemented it [5] after seeing this patch. Thanks! Cc: Christian Poveda <git@pvdrz.com> Cc: Emilio Cobos Álvarez <emilio@crisal.io> Cc: stable@vger.kernel.org # needed for 6.12.y; unneeded for 6.6.y; do not apply to 6.1.y Fixes: c844fa6 ("rust: start supporting several `bindgen` versions") Link: https://github.com/rust-lang/rust-bindgen/blob/21c60f473f4e824d4aa9b2b508056320d474b110/bindgen/features.rs#L97-L105 [1] Link: rust-lang/rust-bindgen#2960 [2] Link: https://github.com/rust-lang/rust-bindgen/blob/7d243056d335fdc4537f7bca73c06d01aae24ddc/bindgen/features.rs#L131-L150 [3] Link: https://github.com/rust-lang/rust-bindgen/blob/main/CHANGELOG.md#0710-2024-12-06 [4] Link: rust-lang/rust-bindgen#2993 [5] Reviewed-by: Alice Ryhl <aliceryhl@google.com> Link: https://lore.kernel.org/r/20241123180323.255997-1-ojeda@kernel.org Signed-off-by: Miguel Ojeda <ojeda@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

commit 7a5f93ea5862da91488975acaa0c7abd508f192b upstream. Each `bindgen` release may upgrade the list of Rust targets. For instance, currently, in their master branch [1], the latest ones are: Nightly => { vectorcall_abi: #124485, ptr_metadata: #81513, layout_for_ptr: #69835, }, Stable_1_77(77) => { offset_of: #106655 }, Stable_1_73(73) => { thiscall_abi: #42202 }, Stable_1_71(71) => { c_unwind_abi: #106075 }, Stable_1_68(68) => { abi_efiapi: #105795 }, By default, the highest stable release in their list is used, and users are expected to set one if they need to support older Rust versions (e.g. see [2]). Thus, over time, new Rust features are used by default, and at some point, it is likely that `bindgen` will emit Rust code that requires a Rust version higher than our minimum (or perhaps enabling an unstable feature). Currently, there is no problem because the maximum they have, as seen above, is Rust 1.77.0, and our current minimum is Rust 1.78.0. Therefore, set a Rust target explicitly now to prevent going forward in time too much and thus getting potential build failures at some point. Since we also support a minimum `bindgen` version, and since `bindgen` does not support passing unknown Rust target versions, we need to use the list of our minimum `bindgen` version, rather than the latest. So, since `bindgen` 0.65.1 had this list [3], we need to use Rust 1.68.0: /// Rust stable 1.64 /// * `core_ffi_c` ([Tracking issue](rust-lang/rust#94501)) => Stable_1_64 => 1.64; /// Rust stable 1.68 /// * `abi_efiapi` calling convention ([Tracking issue](rust-lang/rust#65815)) => Stable_1_68 => 1.68; /// Nightly rust /// * `thiscall` calling convention ([Tracking issue](rust-lang/rust#42202)) /// * `vectorcall` calling convention (no tracking issue) /// * `c_unwind` calling convention ([Tracking issue](rust-lang/rust#74990)) => Nightly => nightly; ... /// Latest stable release of Rust pub const LATEST_STABLE_RUST: RustTarget = RustTarget::Stable_1_68; Thus add the `--rust-target 1.68` parameter. Add a comment as well explaining this. An alternative would be to use the currently running (i.e. actual) `rustc` and `bindgen` versions to pick a "better" Rust target version. However, that would introduce more moving parts depending on the user setup and is also more complex to implement. Starting with `bindgen` 0.71.0 [4], we will be able to set any future Rust version instead, i.e. we will be able to set here our minimum supported Rust version. Christian implemented it [5] after seeing this patch. Thanks! Cc: Christian Poveda <git@pvdrz.com> Cc: Emilio Cobos Álvarez <emilio@crisal.io> Cc: stable@vger.kernel.org # needed for 6.12.y; unneeded for 6.6.y; do not apply to 6.1.y Fixes: c844fa6 ("rust: start supporting several `bindgen` versions") Link: https://github.com/rust-lang/rust-bindgen/blob/21c60f473f4e824d4aa9b2b508056320d474b110/bindgen/features.rs#L97-L105 [1] Link: rust-lang/rust-bindgen#2960 [2] Link: https://github.com/rust-lang/rust-bindgen/blob/7d243056d335fdc4537f7bca73c06d01aae24ddc/bindgen/features.rs#L131-L150 [3] Link: https://github.com/rust-lang/rust-bindgen/blob/main/CHANGELOG.md#0710-2024-12-06 [4] Link: rust-lang/rust-bindgen#2993 [5] Reviewed-by: Alice Ryhl <aliceryhl@google.com> Link: https://lore.kernel.org/r/20241123180323.255997-1-ojeda@kernel.org Signed-off-by: Miguel Ojeda <ojeda@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

pvdrz added 2 commits November 26, 2024 22:31

Make RustTarget parsing more permissive

c7361b1

Represent Rust versions with integers

9665f63

pvdrz force-pushed the pvdrz/rust-target-refactor branch from aa7ba97 to 43d5d32 Compare November 27, 2024 19:26

pvdrz added 3 commits November 27, 2024 14:35

Add constructors to RustTarget

f6a0914

Remove deprecated targets

7ce65e7

Make clippy happy :)

923fed2

pvdrz force-pushed the pvdrz/rust-target-refactor branch from 43d5d32 to 9117083 Compare November 27, 2024 19:36

pvdrz requested a review from emilio November 27, 2024 20:17

pvdrz force-pushed the pvdrz/rust-target-refactor branch from 9117083 to 5f6c6cd Compare November 28, 2024 04:20

ojeda mentioned this pull request Nov 28, 2024

bindgen wanted features & bugfixes Rust-for-Linux/linux#353

Open

27 tasks

emilio approved these changes Nov 28, 2024

View reviewed changes

bindgen/features.rs Show resolved Hide resolved

bindgen-tests/tests/headers/layout.h Show resolved Hide resolved

pvdrz added 2 commits November 28, 2024 14:21

Ignore the layout.h test

3ac14ed

This is done because bindgen is producing bogus code where a single struct has both `packed` and `align` attributes.

Update the help message for --rust-target

2d68245

pvdrz force-pushed the pvdrz/rust-target-refactor branch from 5f7c9d5 to 2d68245 Compare November 28, 2024 19:22

pvdrz enabled auto-merge November 28, 2024 19:23

pvdrz added this pull request to the merge queue Nov 28, 2024

Merged via the queue into main with commit 887dc73 Nov 28, 2024
28 checks passed

pvdrz deleted the pvdrz/rust-target-refactor branch December 2, 2024 02:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor `RustTarget` #2993

Refactor `RustTarget` #2993

pvdrz commented Nov 27, 2024 •

edited

Loading

ojeda commented Nov 28, 2024

pvdrz commented Nov 28, 2024 •

edited

Loading

ojeda commented Nov 28, 2024

emilio left a comment

pvdrz commented Nov 28, 2024

Refactor RustTarget #2993

Refactor RustTarget #2993

Conversation

pvdrz commented Nov 27, 2024 • edited Loading

ojeda commented Nov 28, 2024

pvdrz commented Nov 28, 2024 • edited Loading

ojeda commented Nov 28, 2024

emilio left a comment

Choose a reason for hiding this comment

pvdrz commented Nov 28, 2024

Refactor `RustTarget` #2993

Refactor `RustTarget` #2993

pvdrz commented Nov 27, 2024 •

edited

Loading

pvdrz commented Nov 28, 2024 •

edited

Loading