From b5e67a00d90fb7e77471f8bbc9a5469ee611929f Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Tue, 5 Sep 2023 19:50:48 +0200 Subject: [PATCH 1/9] document when atomic loads are guaranteed read-only --- library/core/src/sync/atomic.rs | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/library/core/src/sync/atomic.rs b/library/core/src/sync/atomic.rs index de41bd1a1167a..1443b9d594c6e 100644 --- a/library/core/src/sync/atomic.rs +++ b/library/core/src/sync/atomic.rs @@ -79,6 +79,26 @@ //! //! [lock-free]: https://en.wikipedia.org/wiki/Non-blocking_algorithm //! +//! # Atomic accesses to read-only memory +//! +//! In general, atomic accesses on read-only memory are Undefined Behavior. For instance, attempting +//! to do a `compare_exchange` that will definitely fail (making it conceptually a read-only +//! operation) can still cause a page fault if the underlying memory page is mapped read-only. Since +//! atomic `load`s might be implemented using compare-exchange operations, even a `load` can fault +//! on read-only memory. +//! +//! However, as an exception from this general rule, Rust guarantees that "sufficiently small" +//! atomic loads are implemented in a way that works on read-only memory. This threshold of +//! "sufficiently small" depends on the architecture: +//! +//! | Target architecture | Maximal atomic `load` size that is guaranteed read-only | +//! |----------|---------| +//! | `x86` | 4 bytes | +//! | `x86_64` | 8 bytes | +//! +//! Any atomic `load` on read-only memory larger than the given size are Undefined Behavior. For +//! architectures not listed above, all atomic `load` on read-only memory are Undefined Behavior. +//! //! # Examples //! //! A simple spinlock: From 7453235febe1230fd475edb8a9ac9d446c9d3008 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Tue, 5 Sep 2023 21:55:42 +0200 Subject: [PATCH 2/9] add ARM and RISC-V values --- library/core/src/sync/atomic.rs | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/library/core/src/sync/atomic.rs b/library/core/src/sync/atomic.rs index 1443b9d594c6e..118b90b5d64e4 100644 --- a/library/core/src/sync/atomic.rs +++ b/library/core/src/sync/atomic.rs @@ -92,9 +92,13 @@ //! "sufficiently small" depends on the architecture: //! //! | Target architecture | Maximal atomic `load` size that is guaranteed read-only | -//! |----------|---------| -//! | `x86` | 4 bytes | -//! | `x86_64` | 8 bytes | +//! |-----------|---------| +//! | `x86` | 4 bytes | +//! | `x86_64` | 8 bytes | +//! | `arm` | 4 bytes | +//! | `aarch64` | 8 bytes | +//! | `riscv32` | 4 bytes | +//! | `riscv64` | 8 bytes | //! //! Any atomic `load` on read-only memory larger than the given size are Undefined Behavior. For //! architectures not listed above, all atomic `load` on read-only memory are Undefined Behavior. From 07b8c10ed89c64d7cb03e16cf9f61509100a3a2b Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Fri, 8 Sep 2023 09:26:06 +0200 Subject: [PATCH 3/9] add general powerpc64le bound (some powerpc64le targets can guarantee more, but for now it doesn't seem worth separating by OS/vendor) --- library/core/src/sync/atomic.rs | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/library/core/src/sync/atomic.rs b/library/core/src/sync/atomic.rs index 118b90b5d64e4..b3f277654a733 100644 --- a/library/core/src/sync/atomic.rs +++ b/library/core/src/sync/atomic.rs @@ -92,13 +92,14 @@ //! "sufficiently small" depends on the architecture: //! //! | Target architecture | Maximal atomic `load` size that is guaranteed read-only | -//! |-----------|---------| -//! | `x86` | 4 bytes | -//! | `x86_64` | 8 bytes | -//! | `arm` | 4 bytes | -//! | `aarch64` | 8 bytes | -//! | `riscv32` | 4 bytes | -//! | `riscv64` | 8 bytes | +//! |---------------|---------| +//! | `x86` | 4 bytes | +//! | `x86_64` | 8 bytes | +//! | `arm` | 4 bytes | +//! | `aarch64` | 8 bytes | +//! | `riscv32` | 4 bytes | +//! | `riscv64` | 8 bytes | +//! | `powerpc64le` | 8 bytes | //! //! Any atomic `load` on read-only memory larger than the given size are Undefined Behavior. For //! architectures not listed above, all atomic `load` on read-only memory are Undefined Behavior. From 69b62ecc69ce42967eefb08c40d59c6b6f4a6d5a Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Fri, 8 Sep 2023 10:35:09 +0200 Subject: [PATCH 4/9] define 'read-only memory' --- library/core/src/sync/atomic.rs | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/library/core/src/sync/atomic.rs b/library/core/src/sync/atomic.rs index b3f277654a733..6adf199a7631a 100644 --- a/library/core/src/sync/atomic.rs +++ b/library/core/src/sync/atomic.rs @@ -81,12 +81,19 @@ //! //! # Atomic accesses to read-only memory //! -//! In general, atomic accesses on read-only memory are Undefined Behavior. For instance, attempting +//! In general, *all* atomic accesses on read-only memory are Undefined Behavior. For instance, attempting //! to do a `compare_exchange` that will definitely fail (making it conceptually a read-only //! operation) can still cause a page fault if the underlying memory page is mapped read-only. Since //! atomic `load`s might be implemented using compare-exchange operations, even a `load` can fault //! on read-only memory. //! +//! For the purpose of this section, "read-only memory" is defined as memory that is read-only in +//! the underlying target, i.e., the pages are mapped with a read-only flag and any attempt to write +//! will cause a page fault. In particular, an `&u128` reference that points to memory that is +//! read-write mapped is *not* considered to point to "read-only memory". In Rust, almost all memory +//! is read-write; the only exceptions are memory created by `const` items or `static` items without +//! interior mutability. +//! //! However, as an exception from this general rule, Rust guarantees that "sufficiently small" //! atomic loads are implemented in a way that works on read-only memory. This threshold of //! "sufficiently small" depends on the architecture: From 275d5c82518897d465ae1c9d1d4cc0e6d426a4d1 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Sat, 9 Sep 2023 17:06:10 +0200 Subject: [PATCH 5/9] wording --- library/core/src/sync/atomic.rs | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/library/core/src/sync/atomic.rs b/library/core/src/sync/atomic.rs index 6adf199a7631a..71ee578f8b460 100644 --- a/library/core/src/sync/atomic.rs +++ b/library/core/src/sync/atomic.rs @@ -87,18 +87,21 @@ //! atomic `load`s might be implemented using compare-exchange operations, even a `load` can fault //! on read-only memory. //! -//! For the purpose of this section, "read-only memory" is defined as memory that is read-only in +//! (For the purpose of this section, "read-only memory" is defined as memory that is read-only in //! the underlying target, i.e., the pages are mapped with a read-only flag and any attempt to write //! will cause a page fault. In particular, an `&u128` reference that points to memory that is //! read-write mapped is *not* considered to point to "read-only memory". In Rust, almost all memory //! is read-write; the only exceptions are memory created by `const` items or `static` items without -//! interior mutability. +//! interior mutability, and memory that was specifically marked as read-only by the operating +//! system via platform-specific APIs.) //! -//! However, as an exception from this general rule, Rust guarantees that "sufficiently small" -//! atomic loads are implemented in a way that works on read-only memory. This threshold of -//! "sufficiently small" depends on the architecture: +//! However, as an exception from this general rule, "sufficiently small" atomic loads are +//! implemented in a way that works on read-only memory. The exact threshold for what makes a load +//! "sufficiently small" varies depending on the architecture and feature flags, but Rust guarantees +//! that atomic loads that do not exceed the size documented in the following table are guaranteed +//! to be read-only: //! -//! | Target architecture | Maximal atomic `load` size that is guaranteed read-only | +//! | Target architecture | Atomic loads no larger than this are guaranteed read-only | //! |---------------|---------| //! | `x86` | 4 bytes | //! | `x86_64` | 8 bytes | @@ -106,10 +109,11 @@ //! | `aarch64` | 8 bytes | //! | `riscv32` | 4 bytes | //! | `riscv64` | 8 bytes | -//! | `powerpc64le` | 8 bytes | +//! | `powerpc64` | 8 bytes | //! -//! Any atomic `load` on read-only memory larger than the given size are Undefined Behavior. For -//! architectures not listed above, all atomic `load` on read-only memory are Undefined Behavior. +//! Atomics loads that are larger than this threshold (and *all* atomic loads on targets not listed +//! in the table) might still be read-only under certain conditions, but that is not a stable +//! guarantee and should not be relied upon. //! //! # Examples //! From 9b8686d832d1122f3ac25a558d393366082e9e38 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Sat, 14 Oct 2023 11:13:07 +0200 Subject: [PATCH 6/9] only guarantee for Relaxed; add ptr-size fallback --- library/core/src/sync/atomic.rs | 33 +++++++++++++++------------------ 1 file changed, 15 insertions(+), 18 deletions(-) diff --git a/library/core/src/sync/atomic.rs b/library/core/src/sync/atomic.rs index 71ee578f8b460..53a57412f8265 100644 --- a/library/core/src/sync/atomic.rs +++ b/library/core/src/sync/atomic.rs @@ -87,33 +87,30 @@ //! atomic `load`s might be implemented using compare-exchange operations, even a `load` can fault //! on read-only memory. //! -//! (For the purpose of this section, "read-only memory" is defined as memory that is read-only in +//! For the purpose of this section, "read-only memory" is defined as memory that is read-only in //! the underlying target, i.e., the pages are mapped with a read-only flag and any attempt to write //! will cause a page fault. In particular, an `&u128` reference that points to memory that is //! read-write mapped is *not* considered to point to "read-only memory". In Rust, almost all memory //! is read-write; the only exceptions are memory created by `const` items or `static` items without //! interior mutability, and memory that was specifically marked as read-only by the operating -//! system via platform-specific APIs.) +//! system via platform-specific APIs. //! -//! However, as an exception from this general rule, "sufficiently small" atomic loads are -//! implemented in a way that works on read-only memory. The exact threshold for what makes a load -//! "sufficiently small" varies depending on the architecture and feature flags, but Rust guarantees -//! that atomic loads that do not exceed the size documented in the following table are guaranteed -//! to be read-only: +//! As an exception from the general rule stated above, "sufficiently small" atomic loads with +//! `Ordering::Relaxed` are implemented in a way that works on read-only memory, and are hence not +//! Undefined Behavior. The exact size limit for what makes a load "sufficiently small" varies +//! depending on the target: //! -//! | Target architecture | Atomic loads no larger than this are guaranteed read-only | +//! | Target triple prefix (regular expression) | Size limit | //! |---------------|---------| -//! | `x86` | 4 bytes | -//! | `x86_64` | 8 bytes | -//! | `arm` | 4 bytes | -//! | `aarch64` | 8 bytes | -//! | `riscv32` | 4 bytes | -//! | `riscv64` | 8 bytes | -//! | `powerpc64` | 8 bytes | +//! | `i(3|5|6)86-`, `arm`, `thumb`, `mips(|el)-`, `powerpc-`, `riscv32`, `sparc-` | 4 bytes | +//! | `x86_64-`, `aarch64-`, `loongarch64-`, `mips64(|el)-`, `powerpc64-`, `riscv64` | 8 bytes | +//! | `powerpc64le-` | 16 bytes | +//! | `s390x-` | 16 bytes | //! -//! Atomics loads that are larger than this threshold (and *all* atomic loads on targets not listed -//! in the table) might still be read-only under certain conditions, but that is not a stable -//! guarantee and should not be relied upon. +//! Atomics loads that are larger than this limit as well as atomic loads with ordering other +//! than `Relaxed`, as well as *all* atomic loads on targets not listed in the table, might still be +//! read-only under certain conditions, but that is not a stable guarantee and should not be relied +//! upon. //! //! # Examples //! From 9d8506d27faf8b396dd54c361182269d265d5d98 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Sun, 15 Oct 2023 17:41:38 +0200 Subject: [PATCH 7/9] acquire loads can be done as relaxed load; acquire fence --- library/core/src/sync/atomic.rs | 3 +++ 1 file changed, 3 insertions(+) diff --git a/library/core/src/sync/atomic.rs b/library/core/src/sync/atomic.rs index 53a57412f8265..aa8ca1e025f3a 100644 --- a/library/core/src/sync/atomic.rs +++ b/library/core/src/sync/atomic.rs @@ -112,6 +112,9 @@ //! read-only under certain conditions, but that is not a stable guarantee and should not be relied //! upon. //! +//! If you need to do an acquire load on read-only memory, you can do a relaxed load followed by an +//! acquire fence instead. +//! //! # Examples //! //! A simple spinlock: From 6605116463a4c1173db7026ee21a22c963569b44 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Mon, 16 Oct 2023 19:29:16 +0200 Subject: [PATCH 8/9] use target-arch based table --- library/core/src/sync/atomic.rs | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/library/core/src/sync/atomic.rs b/library/core/src/sync/atomic.rs index aa8ca1e025f3a..5a429826c0d4a 100644 --- a/library/core/src/sync/atomic.rs +++ b/library/core/src/sync/atomic.rs @@ -100,12 +100,11 @@ //! Undefined Behavior. The exact size limit for what makes a load "sufficiently small" varies //! depending on the target: //! -//! | Target triple prefix (regular expression) | Size limit | +//! | `target_arch` | Size limit | //! |---------------|---------| -//! | `i(3|5|6)86-`, `arm`, `thumb`, `mips(|el)-`, `powerpc-`, `riscv32`, `sparc-` | 4 bytes | -//! | `x86_64-`, `aarch64-`, `loongarch64-`, `mips64(|el)-`, `powerpc64-`, `riscv64` | 8 bytes | -//! | `powerpc64le-` | 16 bytes | -//! | `s390x-` | 16 bytes | +//! | `x86`, `arm`, `mips`, `mips32r6, `powerpc`, `riscv32`, `sparc`, `hexagon` | 4 bytes | +//! | `x86_64`, `aarch64`, `loongarch64`, `mips64`, `mips64r6`, `powerpc64`, `riscv64`, `sparc64` | 8 bytes | +//! | `s390x`, `powerpc64` with `target_feature = "quadword-atomics"` | 16 bytes | //! //! Atomics loads that are larger than this limit as well as atomic loads with ordering other //! than `Relaxed`, as well as *all* atomic loads on targets not listed in the table, might still be From e494df436df105f8a3d767dfcd8e51c597c0fca5 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Tue, 17 Oct 2023 07:56:49 +0200 Subject: [PATCH 9/9] remove 128bit atomics, they are anyway not exposed on those targets --- library/core/src/sync/atomic.rs | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/library/core/src/sync/atomic.rs b/library/core/src/sync/atomic.rs index 5a429826c0d4a..073488817c483 100644 --- a/library/core/src/sync/atomic.rs +++ b/library/core/src/sync/atomic.rs @@ -103,8 +103,7 @@ //! | `target_arch` | Size limit | //! |---------------|---------| //! | `x86`, `arm`, `mips`, `mips32r6, `powerpc`, `riscv32`, `sparc`, `hexagon` | 4 bytes | -//! | `x86_64`, `aarch64`, `loongarch64`, `mips64`, `mips64r6`, `powerpc64`, `riscv64`, `sparc64` | 8 bytes | -//! | `s390x`, `powerpc64` with `target_feature = "quadword-atomics"` | 16 bytes | +//! | `x86_64`, `aarch64`, `loongarch64`, `mips64`, `mips64r6`, `powerpc64`, `riscv64`, `sparc64`, `s390x` | 8 bytes | //! //! Atomics loads that are larger than this limit as well as atomic loads with ordering other //! than `Relaxed`, as well as *all* atomic loads on targets not listed in the table, might still be