-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add float semantics RFC #3514
add float semantics RFC #3514
Conversation
3239d32
to
e983331
Compare
e983331
to
b9ff02a
Compare
Co-authored-by: Ruby Lazuli <general@patchmixolydic.com>
text/0000-float-semantics.md
Outdated
|
||
GCC [says](https://gcc.gnu.org/wiki/FloatingPointMath) "Without any explicit options, GCC assumes round to nearest or even and does not care about signalling NaNs". It is unclear whether "does not care" also means "guarantees to never produce by itself", i.e. whether `0.0 / 0.0` is ever allowed to evaluate to a signaling NaN or not. If it *is* allowed to evaluate to a signaling NaN, that is probably a violation of the C standard, which guarantees that `pow(1, 0.0/0.0)` returns `1` -- but in practice, `pow(1, sNaN)` returns a NaN. | ||
|
||
LLVM [recently adopted](https://github.com/llvm/llvm-project/pull/66579) new NaN rules that this RFC copies exactly into Rust. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since MSVC is one of the tier-one targets, does it have anything to say about these things?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The MSVC ABI is one of our tier 1 targets, but that's orthogonal to what the MSVC compiler does for float operations in C functions it compiles.
I don't know what MSVC does wrt float guarantees since I'm completely disconnected from the Windows ecosystem; if someone has that information and can fill it in, please let me know. :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You'd probably want to see the docs for guarantees. /fp:precise
is the default.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the link! It raises about as many questions as it answers. :)
- They talk about "source precision" and "machine precision". So the result of
a + b + c
might be different from what IEEE says since intermediate results might be stored at a higher (or lower?) precision than that of the source type? - It says that rounding to source precision is done at function calls and returns. So their inliner must preserve these function boundaries then to ensure the value doesn't stay at machine precision? Also, does "rounding to source precision" imply "NaN bits can change"?
- They say "The compiler doesn't perform algebraic transformations on floating-point expressions, such as reassociation or distribution, unless it can guarantee the transformation produces a bitwise identical result". Does this mean they do not even apply commutativity? Commutativity does not produce a bitwise identical result on actual hardware if you take NaNs into account, but usually people don't take NaNs into account when they say things like this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They talk about "source precision" and "machine precision". So the result of
a + b + c
might be different from what IEEE says since intermediate results might be stored at a higher (or lower?) precision than that of the source type?
This is essentially reference to C's FLT_EVAL_METHOD
(which is C's solution to the x87 problem). C§5.2.4.2 says:
The values of floating type yielded by operators subject to the usual arithmetic conversions, including the values yielded by the implicit conversion of operands, and the values of floating constants are evaluated to a format whose range and precision may be greater than required by the type. Such a format is called an evaluation format. In all cases, assignment and cast operators yield values in the format of the type.
It says that rounding to source precision is done at function calls and returns. So their inliner must preserve these function boundaries then to ensure the value doesn't stay at machine precision?
This is adding function parameters and returns to the list of expressions that force to source precision. It requires the same logic in the optimizer as representing an assignment or a cast expression, so it's not much extra burden to support at all.
Does this mean they do not even apply commutativity? Commutativity does not produce a bitwise identical result on actual hardware if you take NaNs into account, but usually people don't take NaNs into account when they say things like this.
IEEE 754 has a concept of "value-preserving transformations" and changing the NaN bit pattern is not a "value-preserving transformation." I don't know specifically what MSVC guarantees, but in general, I would not expect NaN payloads to be preserved by optimization.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changing the NaN bit pattern is not a "value-preserving transformation."
I would not expect NaN payloads to be preserved by optimization.
Those two statements seem to contradiction each other, is there a negation missing / too much somewhere?
But I think we can conclude that MSVC does not document what the permissible bit patterns in NaNs are. So it's probably at least "any qNaN". It's also unclear if they perform any transformations that would make arithmetic return an sNaN, like x * 1.0 -> x
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is x * 1.0 -> x
considered a "value-preserving transformation" by IEEE 754? If so, that seems self-contradicting with also saying that arithmetic operations never return a signaling NaN.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is
x * 1.0 -> x
considered a "value-preserving transformation" by IEEE 754? If so, that seems self-contradicting with also saying that arithmetic operations never return a signaling NaN.
It's explicitly called out as something that is not a value-preserving transformation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interestingly I just noticed that C18 explicitly calls it out as an allowed transformation. So C seems to be fine with operations returning signaling NaNs, but also signaling NaNs lead to unspecified behavior elsewhere? I guess the idea is that when I pass a signaling NaN to a multiplication, that's already unspecified behavior, and that's why it is okay for the output to violate the usual float semantics rules.
text/0000-float-semantics.md
Outdated
@@ -0,0 +1,255 @@ | |||
- Feature Name: `float_semantics` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe it should be float_nan_semantics
instead? There is more to float semantics than just NaNs, like for example there could be RFCs about changing Rust from IEEE 754-2008 to IEE 754-2019 (see the maximum vs maxnum thread in rust-lang/rust#83984 for some discussion on this for example).
Otherwise this reminds me of #1857 which if you read the RFC only stabilized a little part of Rust's drop order logic, but got me confused at the start, making me think that say &&
chain drop order was stable (it was not and I changed it).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We also say that outside NaNs we exactly match IEEE. In particular, we will use exactly the precision declared in the source. For instance, this RFC explicitly says we do not do what #2686 proposed.
So I think the name is adequate.
text/0000-float-semantics.md
Outdated
Rust's floating point operations follow IEEE 754-2008 -- with some caveats around operations producing NaNs: IEEE makes almost no guarantees about the sign and payload bits of the NaN; however, actual hardware does not pick those bits completely arbitrarily, and Rust will expose some of those hardware-provided guarantees to programmers. | ||
On the flip side, NaN generation is non-deterministic: running the same operation on the same inputs several times can produce different results. | ||
And there is a caveat: while IEEE specifies that float operations can never output a signaling NaN, Rust float operations *can* produce signaling NaNs, *but only if* an input is signaling. | ||
That means the only way to ever see a signaling NaN in a program is to create one with `from_bits` (or equivalent unsafe operations). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From a reader's perspective, this part doesn't differentiate 'current behavior' and 'proposed behavior' well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The RFC describes the proposed spec.
Current situation is that we have to spec so there's not really anything to compare with.
Current behavior as implemented by rustc already satisfies the proposed spec.
Just as a note, the LLVM LangRef does not currently actually specify that floating-point operations obey IEEE 754 (llvm/llvm-project#60942). Should this be mentioned in the RFC? |
Good point, I added a note. |
text/0000-float-semantics.md
Outdated
However, when mixing Rust with inline assembly, those details *do* become observable. | ||
To ensure that Rust can provide the above guarantees to user code, it is UB for inline assembly to alter the behavior of floating-point operations in any way: when leaving the inline assembly block, the floating-point environment must be in exactly the same state as when the inline assembly block was entered. | ||
This is just an instance of the general principle that it is UB for inline assembly to violate any of the invariants that the Rust compiler relies on when implementing Rust semantics on the target hardware. | ||
Furthermore, observing the floating-point exception state yields entirely unspecified results: Rust floating-point operations may or may not be executed at the place in the code where they were originally written, and the exception state can change even if no floating-point operation exists in the source code. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Specifically, this section implies that rust-lang/rust#72252 is not-a-bug.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rust-lang/unsafe-code-guidelines#471 is an objection to this part of the RFC. However it's also not clear whether there are any viable alternatives.
4363023
to
c64f06d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is a good RFC. However, having written some numerics code I'm also rather sympathetic to the need for signaling NaNs (and possibly for other fpenv changes). I would like to make sure this RFC doesn't close doors to better supporting them one day.
EDIT: After rereading the end of the RFC I think this is basically resolved.
text/0000-float-semantics.md
Outdated
This RFC is primarily concerned with the guarantee Rust provides to its users. | ||
How exactly those guarantees are achieved is an implementation detail, and not observable when writing pure Rust code. | ||
However, when mixing Rust with inline assembly, those details *do* become observable. | ||
To ensure that Rust can provide the above guarantees to user code, it is UB for inline assembly to alter the behavior of floating-point operations in any way: when leaving the inline assembly block, the floating-point environment must be in exactly the same state as when the inline assembly block was entered. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we do better than blanket "UB"? What about saying that if you do this, your code will still run, but the above guarantees no longer hold?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Possibly, but it would probably have to be fairly specific and would rely on LLVM promising more things. We presently have an example causing a segfault where the only unsafe thing is setting the rounding mode: rust-lang/unsafe-code-guidelines#471 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we can do better than "UB" here. When an operation does not produce the result that the spec says it produces, there's no limit to what happens. People can write if 1+1 != 2 { unreachable_unchecked() }
, and they can do the same with float operations, so changing the rounding mode can lead to arbitrary misbehavior including UB.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But this is the spec we're writing now, isn't it, so if the spec was weaker they couldn't rely on it? Can we weaken it to say something like "in the absence of code which modifies these flags, floats behave according to IEEE 754-2008" and "if you change any of these flags, optimizations will cause your code to behave nondeterministically"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the difference between "behave nondeterministically" and UB?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, what I meant was that the result of floating point operations would be nondeterministic. But we can still put bounds around what they would do; e.g. they would always return a value if you haven't set any exception flags.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's strengthening the spec, not weakening it. In any case, we cannot, as the aforementioned example shows. LLVM will happily optimize away (integer!) array index bounds checks if it can prove that they are never needed under default rounding.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we can still put bounds around what they would do
That's exactly the point, we cannot. (a) there is the example, and (b) the compiler is allowed to move floating-point operations around, so even if you wrote the FP operation outside the block where the flags are different, it might end up executing inside the block.
I don't know any alternative to declaring this UB.
🔔 This is now entering its final comment period, as per the review above. 🔔 |
text/0000-float-semantics.md
Outdated
So the current de-facto semantics of at least some platform intrinsics is that they do *not* match what the platform does. | ||
|
||
# Future possibilities | ||
[future-possibilities]: #future-possibilities |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could have a lint that triggers when a const op generates a NaN.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
text/0000-float-semantics.md
Outdated
|
||
This RFC specifies the behavior of `+`, `-` (unary and binary), `*`, `/`, `%`, `abs`, `copysign`, `mul_add`, `sqrt`, `as`-casts that involve floating-point types, and all comparison operations on floating-point types. | ||
Here, "floating-point types" are `f32` and `f64` and all similar types that might be added in the future such as `f16`, `f128`. | ||
Except for the cases handled below, these operations produce results that exactly match IEEE 754-2008 (with roundTiesToEven [except for float-to-int casts, which round towards zero] and default exception handling without traps, without abruptUnderflow/flush-to-zero). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Except for the cases handled below, these operations produce results that exactly match IEEE 754-2008 (with roundTiesToEven [except for float-to-int casts, which round towards zero] and default exception handling without traps, without abruptUnderflow/flush-to-zero). | |
Except for the cases handled below, these operations produce results that exactly match IEEE 754-2008 (with roundTiesToEven [except for float-to-int casts, which round towards zero](...) and default exception handling without traps, without abruptUnderflow/flush-to-zero). |
The link seems to miss a URL.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, these are just nested parentheses.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I see. Thanks for the clarification.
The final comment period, with a disposition to merge, as per the review above, is now complete. As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed. This will be merged soon. |
The lang team has accepted this RFC, and we've now merged it. Thanks to @RalfJung for pushing forward this important work, and thanks to all those who reviewed this and provided useful feedback. For further updates, follow the tracking issue: |
…jubilee float to/from bits and classify: update for float semantics RFC With rust-lang/rfcs#3514 having been accepted, it is clear that hardware which e.g. flushes subnormal to zero is just non-conformant from a Rust perspective -- this is a hardware bug, or maybe an LLVM backend bug (where LLVM doesn't lower floating-point ops in a way that they have the standardized behavior). So update the comments here to make it clear that we don't have to do any of this, we're just being nice. Also remove the subnormal/NaN checks from the (unstable) const-version of to/from-bits; they are not needed since we decided with the aforementioned RFC that it is okay to get a different result at const-time and at run-time. r? `@workingjubilee` since I think you wrote many of the comments I am editing here.
…bilee float to/from bits and classify: update for float semantics RFC With rust-lang/rfcs#3514 having been accepted, it is clear that hardware which e.g. flushes subnormal to zero is just non-conformant from a Rust perspective -- this is a hardware bug, or maybe an LLVM backend bug (where LLVM doesn't lower floating-point ops in a way that they have the standardized behavior). So update the comments here to make it clear that we don't have to do any of this, we're just being nice. Also remove the subnormal/NaN checks from the (unstable) const-version of to/from-bits; they are not needed since we decided with the aforementioned RFC that it is okay to get a different result at const-time and at run-time. r? `@workingjubilee` since I think you wrote many of the comments I am editing here.
float to/from bits and classify: update for float semantics RFC With rust-lang/rfcs#3514 having been accepted, it is clear that hardware which e.g. flushes subnormal to zero is just non-conformant from a Rust perspective -- this is a hardware bug, or maybe an LLVM backend bug (where LLVM doesn't lower floating-point ops in a way that they have the standardized behavior). So update the comments here to make it clear that we don't have to do any of this, we're just being nice. Also remove the subnormal/NaN checks from the (unstable) const-version of to/from-bits; they are not needed since we decided with the aforementioned RFC that it is okay to get a different result at const-time and at run-time. r? `@workingjubilee` since I think you wrote many of the comments I am editing here.
…rithmetic, r=nnethercote stabilize const_fn_floating_point_arithmetic Part of rust-lang#128288 Fixes rust-lang#57241 The existing test `tests/ui/consts/const_let_eq_float.rs` ([link](https://github.com/RalfJung/rust/blob/const_fn_floating_point_arithmetic/tests/ui/consts/const_let_eq_float.rs)) covers the basics, and also Miri has extensive tests covering the interpreter's float machinery. Also, that machinery can already be used on stable inside `const`/`static` initializers, just not inside `const fn`. This was explicitly called out in rust-lang/rfcs#3514 so in a sense t-lang just recently already FCP'd this, but let's hear from them whether they want another FCP for the stabilization here or whether that was covered by the FCP for the RFC. Cc `@rust-lang/lang` ### Open items - [x] Update the Reference: rust-lang/reference#1566
…rithmetic, r=nnethercote stabilize const_fn_floating_point_arithmetic Part of rust-lang#128288 Fixes rust-lang#57241 The existing test `tests/ui/consts/const_let_eq_float.rs` ([link](https://github.com/RalfJung/rust/blob/const_fn_floating_point_arithmetic/tests/ui/consts/const_let_eq_float.rs)) covers the basics, and also Miri has extensive tests covering the interpreter's float machinery. Also, that machinery can already be used on stable inside `const`/`static` initializers, just not inside `const fn`. This was explicitly called out in rust-lang/rfcs#3514 so in a sense t-lang just recently already FCP'd this, but let's hear from them whether they want another FCP for the stabilization here or whether that was covered by the FCP for the RFC. Cc ``@rust-lang/lang`` ### Open items - [x] Update the Reference: rust-lang/reference#1566
Rollup merge of rust-lang#128596 - RalfJung:const_fn_floating_point_arithmetic, r=nnethercote stabilize const_fn_floating_point_arithmetic Part of rust-lang#128288 Fixes rust-lang#57241 The existing test `tests/ui/consts/const_let_eq_float.rs` ([link](https://github.com/RalfJung/rust/blob/const_fn_floating_point_arithmetic/tests/ui/consts/const_let_eq_float.rs)) covers the basics, and also Miri has extensive tests covering the interpreter's float machinery. Also, that machinery can already be used on stable inside `const`/`static` initializers, just not inside `const fn`. This was explicitly called out in rust-lang/rfcs#3514 so in a sense t-lang just recently already FCP'd this, but let's hear from them whether they want another FCP for the stabilization here or whether that was covered by the FCP for the RFC. Cc ``@rust-lang/lang`` ### Open items - [x] Update the Reference: rust-lang/reference#1566
…, r=nnethercote stabilize const_fn_floating_point_arithmetic Part of rust-lang/rust#128288 Fixes rust-lang/rust#57241 The existing test `tests/ui/consts/const_let_eq_float.rs` ([link](https://github.com/RalfJung/rust/blob/const_fn_floating_point_arithmetic/tests/ui/consts/const_let_eq_float.rs)) covers the basics, and also Miri has extensive tests covering the interpreter's float machinery. Also, that machinery can already be used on stable inside `const`/`static` initializers, just not inside `const fn`. This was explicitly called out in rust-lang/rfcs#3514 so in a sense t-lang just recently already FCP'd this, but let's hear from them whether they want another FCP for the stabilization here or whether that was covered by the FCP for the RFC. Cc ``@rust-lang/lang`` ### Open items - [x] Update the Reference: rust-lang/reference#1566
float to/from bits and classify: update for float semantics RFC With rust-lang/rfcs#3514 having been accepted, it is clear that hardware which e.g. flushes subnormal to zero is just non-conformant from a Rust perspective -- this is a hardware bug, or maybe an LLVM backend bug (where LLVM doesn't lower floating-point ops in a way that they have the standardized behavior). So update the comments here to make it clear that we don't have to do any of this, we're just being nice. Also remove the subnormal/NaN checks from the (unstable) const-version of to/from-bits; they are not needed since we decided with the aforementioned RFC that it is okay to get a different result at const-time and at run-time. r? `@workingjubilee` since I think you wrote many of the comments I am editing here.
Rust's floating point operations follow IEEE 754-2008 -- with some caveats around operations producing NaNs: IEEE makes almost no guarantees about the sign and payload bits of the NaN; however, actual hardware does not pick those bits completely arbitrarily, and Rust will expose some of those hardware-provided guarantees to programmers.
On the flip side, NaN generation is non-deterministic: running the same operation on the same inputs several times can produce different results.
And there is a caveat: while IEEE specifies that float operations can never output a signaling NaN, Rust float operations can produce signaling NaNs, but only if an input is signaling.
That means the only way to ever see a signaling NaN in a program is to create one with
from_bits
(or equivalent unsafe operations).Floating-point operations at compile-time follow the same specification. In particular, since operations are non-deterministic, the same operation can lead to different bit-patterns when executed at compile-time (in a
const
context) vs at run-time. This is the first case of allowing a non-deterministic operation insideconst
. Of course, the compile-time interpreter is still deterministic. It is entirely possible to implement a non-deterministic language on a deterministic machine, by simply making some fixed choices. However, we will not specify a particular choice, and we will not guarantee it to remain the same in the future.(The first paragraph is basically just documenting existing behavior. The second paragraph aims at stabilizing floating-point operations in
const fn
, where they are currently unstable.)@rust-lang/lang, probably the most controversial part is the section on
const
.Rendered
FCP comment
Tracking:
float_semantics
RFC 3514 rust#128288