diff --git a/doc/src/SUMMARY.md b/doc/src/SUMMARY.md index 9d445fc7dc10c..ca25c36bf449a 100644 --- a/doc/src/SUMMARY.md +++ b/doc/src/SUMMARY.md @@ -19,6 +19,7 @@ - [4: Memory safety of BTreeMap's `btree::node` module](./challenges/0004-btree-node.md) - [5: Verify functions iterating over inductive data type: `linked_list`](./challenges/0005-linked-list.md) - [6: Safety of `NonNull`](./challenges/0006-nonnull.md) + - [7: Safety of Methods for Atomic Types and `ReentrantLock`](./challenges/0007-atomic-types.md) - [8: Contracts for SmallSort](./challenges/0008-smallsort.md) - [9: Safe abstractions for `core::time::Duration`](./challenges/0009-duration.md) - [10: Memory safety of String](./challenges/0010-string.md) diff --git a/doc/src/challenges/0007-atomic-types.md b/doc/src/challenges/0007-atomic-types.md new file mode 100644 index 0000000000000..69bff582f7751 --- /dev/null +++ b/doc/src/challenges/0007-atomic-types.md @@ -0,0 +1,128 @@ +# Challenge 7: Safety of Methods for Atomic Types & Atomic Intrinsics + +- **Status:** Open +- **Tracking Issue:** [#83](https://github.com/model-checking/verify-rust-std/issues/83) +- **Start date:** *2024-10-30* +- **End date:** *2024-12-10* + +------------------- + +## Goal + +[`core::sync::atomic`](https://doc.rust-lang.org/std/sync/atomic/index.html) provides methods that operate on atomic types. +For example, `AtomicBool::store(&self, val: bool, order: Ordering)` stores `val` in the atomic boolean referenced by `self` according to the specified [atomic memory ordering](https://doc.rust-lang.org/std/sync/atomic/enum.Ordering.html). + +The goal of this challenge is to verify that these methods are safe.[^1] + +### Success Criteria + +#### Part 1: Unsafe Methods + +First, verify that the unsafe `from_ptr` methods are safe, given that their safety preconditions are met. + +Write safety contracts for each of the `from_ptr` methods: + +- `AtomicBool::from_ptr` +- `AtomicPtr::from_ptr` +- `AtomicI8::from_ptr` +- `AtomicU8::from_ptr` +- `AtomicI16::from_ptr` +- `AtomicU16::from_ptr` +- `AtomicI32::from_ptr` +- `AtomicU32::from_ptr` +- `AtomicI64::from_ptr` +- `AtomicU64::from_ptr` +- `AtomicI128::from_ptr` +- `AtomicU128::from_ptr` + +Specifically, encode the conditions about `ptr`'s alignment and validity (marked `#Safety` in the methods' documentation) as preconditions. +Then, verify that the methods are safe for all possible values for the type that `ptr` points to, given that `ptr` satisfies those preconditions. + +For example, `AtomicI8::from_ptr` is defined as: +```rust +/// `ptr` must be [valid] ... (comment continues; omitted for brevity) +pub const unsafe fn from_ptr<'a>(ptr: *mut i8) -> &'a AtomicI8 { + // SAFETY: guaranteed by the caller + unsafe { &*ptr.cast() } +} +``` + +To verify this method, first encode the safety comments (e.g., about pointer validity) as preconditions, then verify the absence of undefined behavior for all possible `i8` values. + +For the `AtomicPtr` case only, we do not require that you verify safety for all possible types. +Concretely, below is the type signature for `AtomicPtr::from_ptr`: + +```rust +pub const unsafe fn from_ptr<'a>(ptr: *mut *mut T) -> &'a AtomicPtr +``` + +The type pointed to is a `*mut T`. +Verify that for any arbitrary value for this `*mut T` (i.e., any arbitrary memory address), the method is safe. +However, you need not verify the method for all possible instantiations of `T`. +It suffices to verify this method for `T` of byte sizes 0, 1, 2, 4, and at least one non-power of two. + +#### Part 2: Safe Abstractions + +The atomic types expose safe abstractions for atomic operations. +In this part, you will work toward verifying that these abstractions are indeed safe by writing and verifying safety contracts for the unsafe code in their bodies. + +For example, `AtomicBool::store` is the (public) safe method that atomically updates the boolean's value. +This method invokes the unsafe function `atomic_store`, which in turn calls `intrinsics::atomic_store_relaxed`, `intrinsics::atomic_store_release`, or `intrinsics::atomic_store_seqcst`, depending on the provided ordering. + +Write and verify safety contracts for the unsafe functions: + +- `atomic_store` +- `atomic_load` +- `atomic_swap` +- `atomic_add` +- `atomic_sub` +- `atomic_compare_exchange` +- `atomic_compare_exchange_weak` +- `atomic_and` +- `atomic_nand` +- `atomic_or` +- `atomic_xor` +- `atomic_max` +- `atomic_umax` +- `atomic_umin` + +Then, for each of the safe abstractions that invoke the unsafe functions listed above, write contracts that ensure that they are not invoked with `order`s that would cause panics. + +For example, `atomic_store` panics if invoked with `Acquire` or `AcqRel` ordering. +In this case, you would write contracts on the safe `store` methods that enforce that they are not called with either of those `order`s. + +#### Part 3: Atomic Intrinsics + +Write and verify safety contracts for the intrinsics invoked by the unsafe functions from Part 2 (in `core::intrinsics`): + +| Operations | Functions | +|-----------------------|-------------| +| Store | `atomic_store_relaxed`, `atomic_store_release`, `atomic_store_seqcst` | +| Load | `atomic_load_relaxed`, `atomic_load_acquire`, `atomic_load_seqcst` | +| Swap | `atomic_xchg_relaxed`, `atomic_xchg_acquire`, `atomic_xchg_release`, `atomic_xchg_acqrel`, `atomic_xchg_seqcst` | +| Add | `atomic_xadd_relaxed`, `atomic_xadd_acquire`, `atomic_xadd_release`, `atomic_xadd_acqrel`, `atomic_xadd_seqcst` | +| Sub | `atomic_xsub_relaxed`, `atomic_xsub_acquire`, `atomic_xsub_release`, `atomic_xsub_acqrel`, `atomic_xsub_seqcst` | +| Compare Exchange | `atomic_cxchg_relaxed_relaxed`, `atomic_cxchg_relaxed_acquire`, `atomic_cxchg_relaxed_seqcst`, `atomic_cxchg_acquire_relaxed`, `atomic_cxchg_acquire_acquire`, `atomic_cxchg_acquire_seqcst`, `atomic_cxchg_release_relaxed`, `atomic_cxchg_release_acquire`, `atomic_cxchg_release_seqcst`, `atomic_cxchg_acqrel_relaxed`, `atomic_cxchg_acqrel_acquire`, `atomic_cxchg_acqrel_seqcst`, `atomic_cxchg_seqcst_relaxed`, `atomic_cxchg_seqcst_acquire`, `atomic_cxchg_seqcst_seqcst` | +| Compare Exchange Weak | `atomic_cxchgweak_relaxed_relaxed`, `atomic_cxchgweak_relaxed_acquire`, `atomic_cxchgweak_relaxed_seqcst`, `atomic_cxchgweak_acquire_relaxed`, `atomic_cxchgweak_acquire_acquire`, `atomic_cxchgweak_acquire_seqcst`, `atomic_cxchgweak_release_relaxed`, `atomic_cxchgweak_release_acquire`, `atomic_cxchgweak_release_seqcst`, `atomic_cxchgweak_acqrel_relaxed`, `atomic_cxchgweak_acqrel_acquire`, `atomic_cxchgweak_acqrel_seqcst`, `atomic_cxchgweak_seqcst_relaxed`, `atomic_cxchgweak_seqcst_acquire`, `atomic_cxchgweak_seqcst_seqcst` | +| And | `atomic_and_relaxed`, `atomic_and_acquire`, `atomic_and_release`, `atomic_and_acqrel`, `atomic_and_seqcst` | +| Nand | `atomic_nand_relaxed`, `atomic_nand_acquire`, `atomic_nand_release`, `atomic_nand_acqrel`, `atomic_nand_seqcst` | +| Or | `atomic_or_seqcst`, `atomic_or_acquire`, `atomic_or_release`, `atomic_or_acqrel`, `atomic_or_relaxed` | +| Xor | `atomic_xor_seqcst`, `atomic_xor_acquire`, `atomic_xor_release`, `atomic_xor_acqrel`, `atomic_xor_relaxed` | +| Max | `atomic_max_relaxed`, `atomic_max_acquire`, `atomic_max_release`, `atomic_max_acqrel`, `atomic_max_seqcst` | +| Min | `atomic_min_relaxed`, `atomic_min_acquire`, `atomic_min_release`, `atomic_min_acqrel`, `atomic_min_seqcst` | +| Umax | `atomic_umax_relaxed`, `atomic_umax_acquire`, `atomic_umax_release`, `atomic_umax_acqrel`, `atomic_umax_seqcst` | +| Umin | `atomic_umin_relaxed`, `atomic_umin_acquire`, `atomic_umin_release`, `atomic_umin_acqrel`, `atomic_umin_seqcst` | + +## List of UBs + +In addition to any safety properties mentioned in the API documentation, all proofs must automatically ensure the absence of the following [undefined behaviors](https://github.com/rust-lang/reference/blob/142b2ed77d33f37a9973772bd95e6144ed9dce43/src/behavior-considered-undefined.md): + +* Data races. +* Accessing (loading from or storing to) a place that is dangling or based on a misaligned pointer. +* Reading from uninitialized memory. +* Invoking undefined behavior via compiler intrinsics. +* Producing an invalid value. + +Note: All solutions to verification challenges need to satisfy the criteria established in the [challenge book](../general-rules.md) in addition to the ones listed above. + +[^1]: Throughout this challenge, when we say "safe", it is identical to saying "does not exhibit undefined behavior". \ No newline at end of file diff --git a/doc/src/general-rules.md b/doc/src/general-rules.md index 4f419decbdf6b..79c940230dea0 100644 --- a/doc/src/general-rules.md +++ b/doc/src/general-rules.md @@ -5,7 +5,9 @@ **Verification Target:** [Our repository](https://github.com/model-checking/verify-rust-std) is a fork of the original Rust repository, and we kept a copy of the Rust standard library inside the `library/` folder that shall be used as the verification target for all our challenges. We will periodically update the `library/` folder to track newer versions of the [official Rust standard library](https://github.com/rust-lang/rust/). -NOTE: This work is not officially affiliated, or endorsed by the Rust project or Rust Foundation. + +**NOTE:** This work is not officially affiliated, or endorsed by the Rust project or Rust Foundation. + **Challenges:** Each individual verification effort will have a tracking issue where contributors can add comments and ask clarification questions. You can find the list of [open challenges here](https://github.com/model-checking/verify-rust-std/labels/Challenge). diff --git a/doc/src/tools/kani.md b/doc/src/tools/kani.md index 9cfd198e9bc8e..d1066b8d9076d 100644 --- a/doc/src/tools/kani.md +++ b/doc/src/tools/kani.md @@ -59,7 +59,8 @@ Create a local copy of the [model-checking fork](https://github.com/model-checki `assert`, `assume`, `proof` and [function-contracts](https://github.com/model-checking/kani/blob/main/rfc/src/rfcs/0009-function-contracts.md) such as `modifies`, `requires` and `ensures`) directly. -For example, insert this module into an existing file in the core library, like `library/core/src/hint.rs` or `library/core/src/error.rs` in your copy of the library. This is just for the purpose of getting started, so you can insert in any existing file in the core library if you have other preferences. +For example, insert this module into an existing file in the core library, like `library/core/src/hint.rs` or `library/core/src/error.rs` in your copy of the library. +This is just for the purpose of getting started, so you can insert it in a different (existing) file in the core library instead. ``` rust #[cfg(kani)] @@ -84,22 +85,24 @@ pub mod verify { } ``` -### Step 2 - Run the Kani verify-std subcommand +### Step 2 - Run the Kani script on the std library -To aid the Rust Standard Library verification effort, Kani provides a sub-command out of the box to help you get started. -Run the following command in your local terminal (Replace "/path/to/library" and "/path/to/target" with your local paths) from the verify repository root: +To aid the Rust Standard Library verification effort, Kani provides a script out of the box to help you get started. +Run the following command in your local terminal from the verify repository root: ``` -kani verify-std -Z unstable-options "/path/to/library" --target-dir "/path/to/target" -Z function-contracts -Z mem-predicates +./scripts/run-kani.sh --path . ``` -The command `kani verify-std` is a sub-command of the `kani`. This specific sub-command is used to verify the Rust Standard Library with the following arguments. +To pass kani arguments such as `--harness`, you can run the script with `--kani-args` and continue passing in all the necessary arguments: -- `"path/to/library"`: This argument specifies the path to the modified Rust Standard Library that was prepared earlier in the script. For example, `./library` or `/home/ubuntu/verify-rust-std/library` -- `--target-dir "path/to/target"`: This optional argument sets the target directory where Kani will store its output and intermediate files. For example, `/tmp` or `/tmp/verify-std` +``` +./scripts/run-kani.sh --path . --kani-args --harness alloc::layout::verify::check_array_i32 --output-format=terse +``` + +The script `run-kani` installs the right version of Kani for you, builds it and then finally runs the verify-std sub-command of the `kani` with some default flags. -Apart from these, you can use your regular `kani-args` such as `-Z function-contracts`, `-Z stubbing` and `-Z mem-predicates` depending on your verification needs. If you run into a Kani error that says `Use of unstable feature`, add the corresponding feature with `-Z` to the command line. -For more details on Kani's features, refer to [the features section in the Kani Book](https://model-checking.github.io/kani/reference/attributes.html) +**NOTE:** This script may crash due to linking issues. If the script fails with an error message related to linking, link the new CBMC version, delete the `./kani_build` directory and re-run. ### Step 3 - Check verification result @@ -122,7 +125,7 @@ You can specify a specific harness to be verified using the `--harness` flag. For example, in your local copy of the verify repo, run the following command. ``` -kani verify-std --harness harness_introduction -Z unstable-options "./library" --target-dir "/tmp" -Z function-contracts -Z mem-predicates +./scripts/run-kani.sh --kani-args --harness harness_introduction ``` This gives you the verification result for just `harness_introduction` from the aforementioned blob. @@ -144,13 +147,16 @@ Verification Time: 0.01885804s Complete - 1 successfully verified harnesses, 0 failures, 1 total. ``` -Now you can write proof harnesses to verify specific functions in the library. -The current convention is to keep proofs in the same module file of the verification target. -To run Kani for an individual proof, use `--harness [harness_function_name]`. -Note that Kani will batch run all proofs in the library folder if you do not supply the `--harness` flag. -If Kani returns the error `no harnesses matched the harness filter`, you can give the full name of the harness. -For example, to run the proof harness named `check_new` in `library/core/src/ptr/unique.rs`, use -`--harness ptr::unique::verify::check_new`. To run all proofs in `unique.rs`, use `--harness ptr::unique::verify`. +Now you can write proof harnesses to verify specific functions in the library. +The current convention is to keep proofs in the same module file of the verification target. + +To run Kani for an individual proof, use `--harness [harness_function_name]`. +Note that Kani will batch run all proofs in the library folder if you do not supply the `--harness` flag. + +If Kani returns the error `no harnesses matched the harness filter`, you can give the full name of the harness. +For example, to run the proof harness named `check_new` in `library/core/src/ptr/unique.rs`, use +`--harness ptr::unique::verify::check_new`. To run all proofs in `unique.rs`, use `--harness ptr::unique::verify`. + To find the full name of a harness, check the Kani output and find the line starting with `Checking harness [harness full name]`. ## More details diff --git a/library/Cargo.lock b/library/Cargo.lock index 54ad052c52322..5defd2950e8ea 100644 --- a/library/Cargo.lock +++ b/library/Cargo.lock @@ -1,6 +1,6 @@ # This file is automatically @generated by Cargo. # It is not intended for manual editing. -version = 3 +version = 4 [[package]] name = "addr2line" @@ -42,9 +42,12 @@ checksum = "5c6cb57a04249c6480766f7f7cef5467412af1490f8d1e243141daddada3264f" [[package]] name = "cc" -version = "1.0.99" +version = "1.1.22" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "96c51067fd44124faa7f870b4b1c969379ad32b2ba805aa959430ceaa384f695" +checksum = "9540e661f81799159abee814118cc139a2004b3a3aa3ea37724a1b66530b90e0" +dependencies = [ + "shlex", +] [[package]] name = "cfg-if" @@ -58,9 +61,9 @@ dependencies = [ [[package]] name = "compiler_builtins" -version = "0.1.123" +version = "0.1.136" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b47fcbecb558bdad78c7d3a998523c60a50dd6cd046d5fe74163e309e878fff7" +checksum = "33ccee9dd499d7ada4c81533382ce87e88c52b0676c7320b2e617d29e1bb3a3f" dependencies = [ "cc", "rustc-std-workspace-core", @@ -110,9 +113,9 @@ dependencies = [ [[package]] name = "gimli" -version = "0.28.1" +version = "0.29.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4271d37baee1b8c7e4b708028c57d816cf9d2434acb33a549475f78c181f6253" +checksum = "40ecd4077b5ae9fd2e9e169b102c6c330d0605168eb0e8bf79952b256dbefffd" dependencies = [ "compiler_builtins", "rustc-std-workspace-alloc", @@ -121,9 +124,9 @@ dependencies = [ [[package]] name = "gimli" -version = "0.29.0" +version = "0.31.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "40ecd4077b5ae9fd2e9e169b102c6c330d0605168eb0e8bf79952b256dbefffd" +checksum = "07e28edb80900c19c28f1072f2e8aeca7fa06b23cd4169cefe1af5aa3260783f" dependencies = [ "compiler_builtins", "rustc-std-workspace-alloc", @@ -132,9 +135,9 @@ dependencies = [ [[package]] name = "hashbrown" -version = "0.14.5" +version = "0.15.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1" +checksum = "1e087f84d4f86bf4b218b927129862374b72199ae7d8657835f1e89000eea4fb" dependencies = [ "allocator-api2", "compiler_builtins", @@ -155,9 +158,9 @@ dependencies = [ [[package]] name = "libc" -version = "0.2.158" +version = "0.2.161" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439" +checksum = "8e9489c2807c139ffd9c1794f4af0ebe86a828db53ecdc7fea2111d0fed085d1" dependencies = [ "rustc-std-workspace-core", ] @@ -186,9 +189,9 @@ dependencies = [ [[package]] name = "object" -version = "0.36.2" +version = "0.36.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3f203fa8daa7bb185f760ae12bd8e097f63d17041dcdcaf675ac54cdf863170e" +checksum = "084f1a5821ac4c651660a94a7153d27ac9d8a53736203f58b31945ded098070a" dependencies = [ "compiler_builtins", "memchr", @@ -312,6 +315,12 @@ dependencies = [ "std", ] +[[package]] +name = "shlex" +version = "1.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64" + [[package]] name = "std" version = "0.0.0" @@ -326,11 +335,11 @@ dependencies = [ "hashbrown", "hermit-abi", "libc", + "memchr", "miniz_oxide", "object", "panic_abort", "panic_unwind", - "profiler_builtins", "r-efi", "r-efi-alloc", "rand", @@ -358,6 +367,7 @@ name = "sysroot" version = "0.0.0" dependencies = [ "proc_macro", + "profiler_builtins", "std", "test", ] @@ -374,9 +384,9 @@ dependencies = [ [[package]] name = "unicode-width" -version = "0.1.13" +version = "0.1.14" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0336d538f7abc86d282a4189614dfaa90810dfc2c6f6427eaf88e16311dd225d" +checksum = "7dd6e30e90baa6f72411720665d41d89b9a3d039dc45b8faea1ddd07f617f6af" dependencies = [ "compiler_builtins", "rustc-std-workspace-core", @@ -396,12 +406,12 @@ dependencies = [ [[package]] name = "unwinding" -version = "0.2.1" +version = "0.2.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "37a19a21a537f635c16c7576f22d0f2f7d63353c1337ad4ce0d8001c7952a25b" +checksum = "637d511437df708cee34bdec7ba2f1548d256b7acf3ff20e0a1c559f9bf3a987" dependencies = [ "compiler_builtins", - "gimli 0.28.1", + "gimli 0.31.1", "rustc-std-workspace-core", ] @@ -422,7 +432,7 @@ version = "0.52.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d" dependencies = [ - "windows-targets 0.52.5", + "windows-targets 0.52.6", ] [[package]] @@ -431,9 +441,9 @@ version = "0.0.0" [[package]] name = "windows-targets" -version = "0.52.5" +version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6f0713a46559409d202e70e28227288446bf7841d3211583a4b53e3f6d96e7eb" +checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973" dependencies = [ "windows_aarch64_gnullvm", "windows_aarch64_msvc", @@ -447,48 +457,48 @@ dependencies = [ [[package]] name = "windows_aarch64_gnullvm" -version = "0.52.5" +version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7088eed71e8b8dda258ecc8bac5fb1153c5cffaf2578fc8ff5d61e23578d3263" +checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3" [[package]] name = "windows_aarch64_msvc" -version = "0.52.5" +version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9985fd1504e250c615ca5f281c3f7a6da76213ebd5ccc9561496568a2752afb6" +checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469" [[package]] name = "windows_i686_gnu" -version = "0.52.5" +version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "88ba073cf16d5372720ec942a8ccbf61626074c6d4dd2e745299726ce8b89670" +checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b" [[package]] name = "windows_i686_gnullvm" -version = "0.52.5" +version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "87f4261229030a858f36b459e748ae97545d6f1ec60e5e0d6a3d32e0dc232ee9" +checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66" [[package]] name = "windows_i686_msvc" -version = "0.52.5" +version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "db3c2bf3d13d5b658be73463284eaf12830ac9a26a90c717b7f771dfe97487bf" +checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66" [[package]] name = "windows_x86_64_gnu" -version = "0.52.5" +version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4e4246f76bdeff09eb48875a0fd3e2af6aada79d409d33011886d3e1581517d9" +checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78" [[package]] name = "windows_x86_64_gnullvm" -version = "0.52.5" +version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "852298e482cd67c356ddd9570386e2862b5673c85bd5f88df9ab6802b334c596" +checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d" [[package]] name = "windows_x86_64_msvc" -version = "0.52.5" +version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bec47e5bfd1bff0eeaf6d8b485cc1074891a197ab4225d504cb7a1ab88b02bf0" +checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec" diff --git a/library/alloc/benches/binary_heap.rs b/library/alloc/benches/binary_heap.rs index 917e71f250ee8..1b8f7f1c24278 100644 --- a/library/alloc/benches/binary_heap.rs +++ b/library/alloc/benches/binary_heap.rs @@ -1,7 +1,7 @@ use std::collections::BinaryHeap; use rand::seq::SliceRandom; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; #[bench] fn bench_find_smallest_1000(b: &mut Bencher) { diff --git a/library/alloc/benches/btree/map.rs b/library/alloc/benches/btree/map.rs index 3bddef5045a00..b8119c9f0ebf4 100644 --- a/library/alloc/benches/btree/map.rs +++ b/library/alloc/benches/btree/map.rs @@ -1,9 +1,9 @@ use std::collections::BTreeMap; use std::ops::RangeBounds; -use rand::seq::SliceRandom; use rand::Rng; -use test::{black_box, Bencher}; +use rand::seq::SliceRandom; +use test::{Bencher, black_box}; macro_rules! map_insert_rand_bench { ($name: ident, $n: expr, $map: ident) => { diff --git a/library/alloc/benches/lib.rs b/library/alloc/benches/lib.rs index ae9608ec7bd5c..c1907361f93e1 100644 --- a/library/alloc/benches/lib.rs +++ b/library/alloc/benches/lib.rs @@ -4,7 +4,8 @@ #![feature(iter_next_chunk)] #![feature(repr_simd)] #![feature(slice_partition_dedup)] -#![feature(strict_provenance)] +#![cfg_attr(bootstrap, feature(strict_provenance))] +#![cfg_attr(not(bootstrap), feature(strict_provenance_lints))] #![feature(test)] #![deny(fuzzy_provenance_casts)] diff --git a/library/alloc/benches/slice.rs b/library/alloc/benches/slice.rs index b62be9d39a1cd..48c74c4491dc8 100644 --- a/library/alloc/benches/slice.rs +++ b/library/alloc/benches/slice.rs @@ -1,8 +1,8 @@ use std::{mem, ptr}; -use rand::distributions::{Alphanumeric, DistString, Standard}; use rand::Rng; -use test::{black_box, Bencher}; +use rand::distributions::{Alphanumeric, DistString, Standard}; +use test::{Bencher, black_box}; #[bench] fn iterator(b: &mut Bencher) { @@ -336,10 +336,10 @@ reverse!(reverse_u32, u32, |x| x as u32); reverse!(reverse_u64, u64, |x| x as u64); reverse!(reverse_u128, u128, |x| x as u128); #[repr(simd)] -struct F64x4(f64, f64, f64, f64); +struct F64x4([f64; 4]); reverse!(reverse_simd_f64x4, F64x4, |x| { let x = x as f64; - F64x4(x, x, x, x) + F64x4([x, x, x, x]) }); macro_rules! rotate { diff --git a/library/alloc/benches/str.rs b/library/alloc/benches/str.rs index c148ab6b220a5..98c7c5413caef 100644 --- a/library/alloc/benches/str.rs +++ b/library/alloc/benches/str.rs @@ -1,4 +1,4 @@ -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; #[bench] fn char_iterator(b: &mut Bencher) { @@ -347,3 +347,5 @@ make_test!(rsplitn_space_char, s, s.rsplitn(10, ' ').count()); make_test!(split_space_str, s, s.split(" ").count()); make_test!(split_ad_str, s, s.split("ad").count()); + +make_test!(to_lowercase, s, s.to_lowercase()); diff --git a/library/alloc/benches/string.rs b/library/alloc/benches/string.rs index e0dbe80d28896..3d79ab78c6950 100644 --- a/library/alloc/benches/string.rs +++ b/library/alloc/benches/string.rs @@ -1,6 +1,6 @@ use std::iter::repeat; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; #[bench] fn bench_with_capacity(b: &mut Bencher) { diff --git a/library/alloc/benches/vec.rs b/library/alloc/benches/vec.rs index 13d784d3fd40e..d29ffae9d70b1 100644 --- a/library/alloc/benches/vec.rs +++ b/library/alloc/benches/vec.rs @@ -1,7 +1,7 @@ use std::iter::repeat; use rand::RngCore; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; #[bench] fn bench_new(b: &mut Bencher) { diff --git a/library/alloc/benches/vec_deque.rs b/library/alloc/benches/vec_deque.rs index fb1e2685cc333..a56f8496963bc 100644 --- a/library/alloc/benches/vec_deque.rs +++ b/library/alloc/benches/vec_deque.rs @@ -1,7 +1,7 @@ -use std::collections::{vec_deque, VecDeque}; +use std::collections::{VecDeque, vec_deque}; use std::mem; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; #[bench] fn bench_new(b: &mut Bencher) { diff --git a/library/alloc/src/alloc.rs b/library/alloc/src/alloc.rs index cddf4f6f39963..a91659b6de5ad 100644 --- a/library/alloc/src/alloc.rs +++ b/library/alloc/src/alloc.rs @@ -89,6 +89,7 @@ pub use std::alloc::Global; #[stable(feature = "global_alloc", since = "1.28.0")] #[must_use = "losing the pointer will leak memory"] #[inline] +#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub unsafe fn alloc(layout: Layout) -> *mut u8 { unsafe { // Make sure we don't accidentally allow omitting the allocator shim in @@ -113,6 +114,7 @@ pub unsafe fn alloc(layout: Layout) -> *mut u8 { /// See [`GlobalAlloc::dealloc`]. #[stable(feature = "global_alloc", since = "1.28.0")] #[inline] +#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub unsafe fn dealloc(ptr: *mut u8, layout: Layout) { unsafe { __rust_dealloc(ptr, layout.size(), layout.align()) } } @@ -132,6 +134,7 @@ pub unsafe fn dealloc(ptr: *mut u8, layout: Layout) { #[stable(feature = "global_alloc", since = "1.28.0")] #[must_use = "losing the pointer will leak memory"] #[inline] +#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub unsafe fn realloc(ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 { unsafe { __rust_realloc(ptr, layout.size(), layout.align(), new_size) } } @@ -166,13 +169,21 @@ pub unsafe fn realloc(ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 #[stable(feature = "global_alloc", since = "1.28.0")] #[must_use = "losing the pointer will leak memory"] #[inline] +#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub unsafe fn alloc_zeroed(layout: Layout) -> *mut u8 { - unsafe { __rust_alloc_zeroed(layout.size(), layout.align()) } + unsafe { + // Make sure we don't accidentally allow omitting the allocator shim in + // stable code until it is actually stabilized. + core::ptr::read_volatile(&__rust_no_alloc_shim_is_unstable); + + __rust_alloc_zeroed(layout.size(), layout.align()) + } } #[cfg(not(test))] impl Global { #[inline] + #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces fn alloc_impl(&self, layout: Layout, zeroed: bool) -> Result, AllocError> { match layout.size() { 0 => Ok(NonNull::slice_from_raw_parts(layout.dangling(), 0)), @@ -187,6 +198,7 @@ impl Global { // SAFETY: Same as `Allocator::grow` #[inline] + #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces unsafe fn grow_impl( &self, ptr: NonNull, @@ -237,16 +249,19 @@ impl Global { #[cfg(not(test))] unsafe impl Allocator for Global { #[inline] + #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces fn allocate(&self, layout: Layout) -> Result, AllocError> { self.alloc_impl(layout, false) } #[inline] + #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces fn allocate_zeroed(&self, layout: Layout) -> Result, AllocError> { self.alloc_impl(layout, true) } #[inline] + #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces unsafe fn deallocate(&self, ptr: NonNull, layout: Layout) { if layout.size() != 0 { // SAFETY: `layout` is non-zero in size, @@ -256,6 +271,7 @@ unsafe impl Allocator for Global { } #[inline] + #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces unsafe fn grow( &self, ptr: NonNull, @@ -267,6 +283,7 @@ unsafe impl Allocator for Global { } #[inline] + #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces unsafe fn grow_zeroed( &self, ptr: NonNull, @@ -278,6 +295,7 @@ unsafe impl Allocator for Global { } #[inline] + #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces unsafe fn shrink( &self, ptr: NonNull, @@ -325,6 +343,7 @@ unsafe impl Allocator for Global { #[cfg(all(not(no_global_oom_handling), not(test)))] #[lang = "exchange_malloc"] #[inline] +#[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces unsafe fn exchange_malloc(size: usize, align: usize) -> *mut u8 { let layout = unsafe { Layout::from_size_align_unchecked(size, align) }; match Global.allocate(layout) { diff --git a/library/alloc/src/borrow.rs b/library/alloc/src/borrow.rs index f86face3f90cb..dbfd2e74abee6 100644 --- a/library/alloc/src/borrow.rs +++ b/library/alloc/src/borrow.rs @@ -225,7 +225,6 @@ impl Cow<'_, B> { /// assert!(!bull.is_borrowed()); /// ``` #[unstable(feature = "cow_is_borrowed", issue = "65143")] - #[rustc_const_unstable(feature = "const_cow_is_borrowed", issue = "65143")] pub const fn is_borrowed(&self) -> bool { match *self { Borrowed(_) => true, @@ -248,7 +247,6 @@ impl Cow<'_, B> { /// assert!(!bull.is_owned()); /// ``` #[unstable(feature = "cow_is_borrowed", issue = "65143")] - #[rustc_const_unstable(feature = "const_cow_is_borrowed", issue = "65143")] pub const fn is_owned(&self) -> bool { !self.is_borrowed() } diff --git a/library/alloc/src/boxed.rs b/library/alloc/src/boxed.rs index 6dc75478700ce..e4956c7c53c8d 100644 --- a/library/alloc/src/boxed.rs +++ b/library/alloc/src/boxed.rs @@ -183,51 +183,48 @@ #![stable(feature = "rust1", since = "1.0.0")] -use core::any::Any; -use core::async_iter::AsyncIterator; +use core::borrow::{Borrow, BorrowMut}; #[cfg(not(no_global_oom_handling))] use core::clone::CloneToUninit; use core::cmp::Ordering; -use core::error::Error; +use core::error::{self, Error}; +use core::fmt; use core::future::Future; use core::hash::{Hash, Hasher}; -use core::iter::FusedIterator; use core::marker::{Tuple, Unsize}; use core::mem::{self, SizedTypeProperties}; use core::ops::{ AsyncFn, AsyncFnMut, AsyncFnOnce, CoerceUnsized, Coroutine, CoroutineState, Deref, DerefMut, - DerefPure, DispatchFromDyn, Receiver, + DerefPure, DispatchFromDyn, LegacyReceiver, }; use core::pin::{Pin, PinCoerceUnsized}; -use core::ptr::{self, addr_of_mut, NonNull, Unique}; +use core::ptr::{self, NonNull, Unique}; use core::task::{Context, Poll}; -use core::{borrow, fmt, slice}; - -#[unstable(feature = "thin_box", issue = "92791")] -pub use thin::ThinBox; #[cfg(not(no_global_oom_handling))] use crate::alloc::handle_alloc_error; use crate::alloc::{AllocError, Allocator, Global, Layout}; -#[cfg(not(no_global_oom_handling))] -use crate::borrow::Cow; use crate::raw_vec::RawVec; #[cfg(not(no_global_oom_handling))] use crate::str::from_boxed_utf8_unchecked; -#[cfg(not(no_global_oom_handling))] -use crate::string::String; -use crate::vec; -#[cfg(not(no_global_oom_handling))] -use crate::vec::Vec; +/// Conversion related impls for `Box<_>` (`From`, `downcast`, etc) +mod convert; +/// Iterator related impls for `Box<_>`. +mod iter; +/// [`ThinBox`] implementation. mod thin; +#[unstable(feature = "thin_box", issue = "92791")] +pub use thin::ThinBox; + /// A pointer type that uniquely owns a heap allocation of type `T`. /// /// See the [module-level documentation](../../std/boxed/index.html) for more. #[lang = "owned_box"] #[fundamental] #[stable(feature = "rust1", since = "1.0.0")] +#[rustc_insignificant_dtor] // The declaration of the `Box` struct must be kept in sync with the // compiler or ICEs will happen. pub struct Box< @@ -250,6 +247,7 @@ impl Box { #[stable(feature = "rust1", since = "1.0.0")] #[must_use] #[rustc_diagnostic_item = "box_new"] + #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub fn new(x: T) -> Self { #[rustc_box] Box::new(x) @@ -1059,6 +1057,59 @@ impl Box { pub unsafe fn from_raw(raw: *mut T) -> Self { unsafe { Self::from_raw_in(raw, Global) } } + + /// Constructs a box from a `NonNull` pointer. + /// + /// After calling this function, the `NonNull` pointer is owned by + /// the resulting `Box`. Specifically, the `Box` destructor will call + /// the destructor of `T` and free the allocated memory. For this + /// to be safe, the memory must have been allocated in accordance + /// with the [memory layout] used by `Box` . + /// + /// # Safety + /// + /// This function is unsafe because improper use may lead to + /// memory problems. For example, a double-free may occur if the + /// function is called twice on the same `NonNull` pointer. + /// + /// The safety conditions are described in the [memory layout] section. + /// + /// # Examples + /// + /// Recreate a `Box` which was previously converted to a `NonNull` + /// pointer using [`Box::into_non_null`]: + /// ``` + /// #![feature(box_vec_non_null)] + /// + /// let x = Box::new(5); + /// let non_null = Box::into_non_null(x); + /// let x = unsafe { Box::from_non_null(non_null) }; + /// ``` + /// Manually create a `Box` from scratch by using the global allocator: + /// ``` + /// #![feature(box_vec_non_null)] + /// + /// use std::alloc::{alloc, Layout}; + /// use std::ptr::NonNull; + /// + /// unsafe { + /// let non_null = NonNull::new(alloc(Layout::new::()).cast::()) + /// .expect("allocation failed"); + /// // In general .write is required to avoid attempting to destruct + /// // the (uninitialized) previous contents of `non_null`. + /// non_null.write(5); + /// let x = Box::from_non_null(non_null); + /// } + /// ``` + /// + /// [memory layout]: self#memory-layout + /// [`Layout`]: crate::Layout + #[unstable(feature = "box_vec_non_null", reason = "new API", issue = "130364")] + #[inline] + #[must_use = "call `drop(Box::from_non_null(ptr))` if you intend to drop the `Box`"] + pub unsafe fn from_non_null(ptr: NonNull) -> Self { + unsafe { Self::from_raw(ptr.as_ptr()) } + } } impl Box { @@ -1116,6 +1167,61 @@ impl Box { Box(unsafe { Unique::new_unchecked(raw) }, alloc) } + /// Constructs a box from a `NonNull` pointer in the given allocator. + /// + /// After calling this function, the `NonNull` pointer is owned by + /// the resulting `Box`. Specifically, the `Box` destructor will call + /// the destructor of `T` and free the allocated memory. For this + /// to be safe, the memory must have been allocated in accordance + /// with the [memory layout] used by `Box` . + /// + /// # Safety + /// + /// This function is unsafe because improper use may lead to + /// memory problems. For example, a double-free may occur if the + /// function is called twice on the same raw pointer. + /// + /// + /// # Examples + /// + /// Recreate a `Box` which was previously converted to a `NonNull` pointer + /// using [`Box::into_non_null_with_allocator`]: + /// ``` + /// #![feature(allocator_api, box_vec_non_null)] + /// + /// use std::alloc::System; + /// + /// let x = Box::new_in(5, System); + /// let (non_null, alloc) = Box::into_non_null_with_allocator(x); + /// let x = unsafe { Box::from_non_null_in(non_null, alloc) }; + /// ``` + /// Manually create a `Box` from scratch by using the system allocator: + /// ``` + /// #![feature(allocator_api, box_vec_non_null, slice_ptr_get)] + /// + /// use std::alloc::{Allocator, Layout, System}; + /// + /// unsafe { + /// let non_null = System.allocate(Layout::new::())?.cast::(); + /// // In general .write is required to avoid attempting to destruct + /// // the (uninitialized) previous contents of `non_null`. + /// non_null.write(5); + /// let x = Box::from_non_null_in(non_null, System); + /// } + /// # Ok::<(), std::alloc::AllocError>(()) + /// ``` + /// + /// [memory layout]: self#memory-layout + /// [`Layout`]: crate::Layout + #[unstable(feature = "allocator_api", issue = "32838")] + // #[unstable(feature = "box_vec_non_null", reason = "new API", issue = "130364")] + #[rustc_const_unstable(feature = "const_box", issue = "92521")] + #[inline] + pub const unsafe fn from_non_null_in(raw: NonNull, alloc: A) -> Self { + // SAFETY: guaranteed by the caller. + unsafe { Box::from_raw_in(raw.as_ptr(), alloc) } + } + /// Consumes the `Box`, returning a wrapped raw pointer. /// /// The pointer will be properly aligned and non-null. @@ -1168,7 +1274,67 @@ impl Box { #[inline] pub fn into_raw(b: Self) -> *mut T { // Make sure Miri realizes that we transition from a noalias pointer to a raw pointer here. - unsafe { addr_of_mut!(*&mut *Self::into_raw_with_allocator(b).0) } + unsafe { &raw mut *&mut *Self::into_raw_with_allocator(b).0 } + } + + /// Consumes the `Box`, returning a wrapped `NonNull` pointer. + /// + /// The pointer will be properly aligned. + /// + /// After calling this function, the caller is responsible for the + /// memory previously managed by the `Box`. In particular, the + /// caller should properly destroy `T` and release the memory, taking + /// into account the [memory layout] used by `Box`. The easiest way to + /// do this is to convert the `NonNull` pointer back into a `Box` with the + /// [`Box::from_non_null`] function, allowing the `Box` destructor to + /// perform the cleanup. + /// + /// Note: this is an associated function, which means that you have + /// to call it as `Box::into_non_null(b)` instead of `b.into_non_null()`. + /// This is so that there is no conflict with a method on the inner type. + /// + /// # Examples + /// Converting the `NonNull` pointer back into a `Box` with [`Box::from_non_null`] + /// for automatic cleanup: + /// ``` + /// #![feature(box_vec_non_null)] + /// + /// let x = Box::new(String::from("Hello")); + /// let non_null = Box::into_non_null(x); + /// let x = unsafe { Box::from_non_null(non_null) }; + /// ``` + /// Manual cleanup by explicitly running the destructor and deallocating + /// the memory: + /// ``` + /// #![feature(box_vec_non_null)] + /// + /// use std::alloc::{dealloc, Layout}; + /// + /// let x = Box::new(String::from("Hello")); + /// let non_null = Box::into_non_null(x); + /// unsafe { + /// non_null.drop_in_place(); + /// dealloc(non_null.as_ptr().cast::(), Layout::new::()); + /// } + /// ``` + /// Note: This is equivalent to the following: + /// ``` + /// #![feature(box_vec_non_null)] + /// + /// let x = Box::new(String::from("Hello")); + /// let non_null = Box::into_non_null(x); + /// unsafe { + /// drop(Box::from_non_null(non_null)); + /// } + /// ``` + /// + /// [memory layout]: self#memory-layout + #[must_use = "losing the pointer will leak memory"] + #[unstable(feature = "box_vec_non_null", reason = "new API", issue = "130364")] + #[inline] + pub fn into_non_null(b: Self) -> NonNull { + // SAFETY: `Box` is guaranteed to be non-null. + unsafe { NonNull::new_unchecked(Self::into_raw(b)) } } /// Consumes the `Box`, returning a wrapped raw pointer and the allocator. @@ -1227,11 +1393,66 @@ impl Box { // want *no* aliasing requirements here! // In case `A` *is* `Global`, this does not quite have the right behavior; `into_raw` // works around that. - let ptr = addr_of_mut!(**b); + let ptr = &raw mut **b; let alloc = unsafe { ptr::read(&b.1) }; (ptr, alloc) } + /// Consumes the `Box`, returning a wrapped `NonNull` pointer and the allocator. + /// + /// The pointer will be properly aligned. + /// + /// After calling this function, the caller is responsible for the + /// memory previously managed by the `Box`. In particular, the + /// caller should properly destroy `T` and release the memory, taking + /// into account the [memory layout] used by `Box`. The easiest way to + /// do this is to convert the `NonNull` pointer back into a `Box` with the + /// [`Box::from_non_null_in`] function, allowing the `Box` destructor to + /// perform the cleanup. + /// + /// Note: this is an associated function, which means that you have + /// to call it as `Box::into_non_null_with_allocator(b)` instead of + /// `b.into_non_null_with_allocator()`. This is so that there is no + /// conflict with a method on the inner type. + /// + /// # Examples + /// Converting the `NonNull` pointer back into a `Box` with + /// [`Box::from_non_null_in`] for automatic cleanup: + /// ``` + /// #![feature(allocator_api, box_vec_non_null)] + /// + /// use std::alloc::System; + /// + /// let x = Box::new_in(String::from("Hello"), System); + /// let (non_null, alloc) = Box::into_non_null_with_allocator(x); + /// let x = unsafe { Box::from_non_null_in(non_null, alloc) }; + /// ``` + /// Manual cleanup by explicitly running the destructor and deallocating + /// the memory: + /// ``` + /// #![feature(allocator_api, box_vec_non_null)] + /// + /// use std::alloc::{Allocator, Layout, System}; + /// + /// let x = Box::new_in(String::from("Hello"), System); + /// let (non_null, alloc) = Box::into_non_null_with_allocator(x); + /// unsafe { + /// non_null.drop_in_place(); + /// alloc.deallocate(non_null.cast::(), Layout::new::()); + /// } + /// ``` + /// + /// [memory layout]: self#memory-layout + #[must_use = "losing the pointer will leak memory"] + #[unstable(feature = "allocator_api", issue = "32838")] + // #[unstable(feature = "box_vec_non_null", reason = "new API", issue = "130364")] + #[inline] + pub fn into_non_null_with_allocator(b: Self) -> (NonNull, A) { + let (ptr, alloc) = Box::into_raw_with_allocator(b); + // SAFETY: `Box` is guaranteed to be non-null. + unsafe { (NonNull::new_unchecked(ptr), alloc) } + } + #[unstable( feature = "ptr_internals", issue = "none", @@ -1282,7 +1503,7 @@ impl Box { pub fn as_mut_ptr(b: &mut Self) -> *mut T { // This is a primitive deref, not going through `DerefMut`, and therefore not materializing // any references. - ptr::addr_of_mut!(**b) + &raw mut **b } /// Returns a raw pointer to the `Box`'s contents. @@ -1330,7 +1551,7 @@ impl Box { pub fn as_ptr(b: &Self) -> *const T { // This is a primitive deref, not going through `DerefMut`, and therefore not materializing // any references. - ptr::addr_of!(**b) + &raw const **b } /// Returns a reference to the underlying allocator. @@ -1463,7 +1684,7 @@ impl Default for Box { /// Creates a `Box`, with the `Default` value for T. #[inline] fn default() -> Self { - Box::new(T::default()) + Box::write(Box::new_uninit(), T::default()) } } @@ -1541,6 +1762,41 @@ impl Clone for Box { } } +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "box_slice_clone", since = "1.3.0")] +impl Clone for Box<[T], A> { + fn clone(&self) -> Self { + let alloc = Box::allocator(self).clone(); + self.to_vec_in(alloc).into_boxed_slice() + } + + /// Copies `source`'s contents into `self` without creating a new allocation, + /// so long as the two are of the same length. + /// + /// # Examples + /// + /// ``` + /// let x = Box::new([5, 6, 7]); + /// let mut y = Box::new([8, 9, 10]); + /// let yp: *const [i32] = &*y; + /// + /// y.clone_from(&x); + /// + /// // The value is the same + /// assert_eq!(x, y); + /// + /// // And no allocation occurred + /// assert_eq!(yp, &*y); + /// ``` + fn clone_from(&mut self, source: &Self) { + if self.len() == source.len() { + self.clone_from_slice(&source); + } else { + *self = source.clone(); + } + } +} + #[cfg(not(no_global_oom_handling))] #[stable(feature = "box_slice_clone", since = "1.3.0")] impl Clone for Box { @@ -1562,6 +1818,7 @@ impl PartialEq for Box { PartialEq::ne(&**self, &**other) } } + #[stable(feature = "rust1", since = "1.0.0")] impl PartialOrd for Box { #[inline] @@ -1585,6 +1842,7 @@ impl PartialOrd for Box { PartialOrd::gt(&**self, &**other) } } + #[stable(feature = "rust1", since = "1.0.0")] impl Ord for Box { #[inline] @@ -1592,6 +1850,7 @@ impl Ord for Box { Ord::cmp(&**self, &**other) } } + #[stable(feature = "rust1", since = "1.0.0")] impl Eq for Box {} @@ -1654,572 +1913,51 @@ impl Hasher for Box { } } -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "from_for_ptrs", since = "1.6.0")] -impl From for Box { - /// Converts a `T` into a `Box` - /// - /// The conversion allocates on the heap and moves `t` - /// from the stack into it. - /// - /// # Examples - /// - /// ```rust - /// let x = 5; - /// let boxed = Box::new(5); - /// - /// assert_eq!(Box::from(x), boxed); - /// ``` - fn from(t: T) -> Self { - Box::new(t) +#[stable(feature = "rust1", since = "1.0.0")] +impl fmt::Display for Box { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + fmt::Display::fmt(&**self, f) } } -#[stable(feature = "pin", since = "1.33.0")] -impl From> for Pin> -where - A: 'static, -{ - /// Converts a `Box` into a `Pin>`. If `T` does not implement [`Unpin`], then - /// `*boxed` will be pinned in memory and unable to be moved. - /// - /// This conversion does not allocate on the heap and happens in place. - /// - /// This is also available via [`Box::into_pin`]. - /// - /// Constructing and pinning a `Box` with >>::from([Box::new]\(x)) - /// can also be written more concisely using [Box::pin]\(x). - /// This `From` implementation is useful if you already have a `Box`, or you are - /// constructing a (pinned) `Box` in a different way than with [`Box::new`]. - fn from(boxed: Box) -> Self { - Box::into_pin(boxed) +#[stable(feature = "rust1", since = "1.0.0")] +impl fmt::Debug for Box { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + fmt::Debug::fmt(&**self, f) } } -/// Specialization trait used for `From<&[T]>`. -#[cfg(not(no_global_oom_handling))] -trait BoxFromSlice { - fn from_slice(slice: &[T]) -> Self; -} - -#[cfg(not(no_global_oom_handling))] -impl BoxFromSlice for Box<[T]> { - #[inline] - default fn from_slice(slice: &[T]) -> Self { - slice.to_vec().into_boxed_slice() +#[stable(feature = "rust1", since = "1.0.0")] +impl fmt::Pointer for Box { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + // It's not possible to extract the inner Uniq directly from the Box, + // instead we cast it to a *const which aliases the Unique + let ptr: *const T = &**self; + fmt::Pointer::fmt(&ptr, f) } } -#[cfg(not(no_global_oom_handling))] -impl BoxFromSlice for Box<[T]> { - #[inline] - fn from_slice(slice: &[T]) -> Self { - let len = slice.len(); - let buf = RawVec::with_capacity(len); - unsafe { - ptr::copy_nonoverlapping(slice.as_ptr(), buf.ptr(), len); - buf.into_box(slice.len()).assume_init() - } - } -} +#[stable(feature = "rust1", since = "1.0.0")] +impl Deref for Box { + type Target = T; -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "box_from_slice", since = "1.17.0")] -impl From<&[T]> for Box<[T]> { - /// Converts a `&[T]` into a `Box<[T]>` - /// - /// This conversion allocates on the heap - /// and performs a copy of `slice` and its contents. - /// - /// # Examples - /// ```rust - /// // create a &[u8] which will be used to create a Box<[u8]> - /// let slice: &[u8] = &[104, 101, 108, 108, 111]; - /// let boxed_slice: Box<[u8]> = Box::from(slice); - /// - /// println!("{boxed_slice:?}"); - /// ``` - #[inline] - fn from(slice: &[T]) -> Box<[T]> { - >::from_slice(slice) + fn deref(&self) -> &T { + &**self } } -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "box_from_cow", since = "1.45.0")] -impl From> for Box<[T]> { - /// Converts a `Cow<'_, [T]>` into a `Box<[T]>` - /// - /// When `cow` is the `Cow::Borrowed` variant, this - /// conversion allocates on the heap and copies the - /// underlying slice. Otherwise, it will try to reuse the owned - /// `Vec`'s allocation. - #[inline] - fn from(cow: Cow<'_, [T]>) -> Box<[T]> { - match cow { - Cow::Borrowed(slice) => Box::from(slice), - Cow::Owned(slice) => Box::from(slice), - } - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "box_from_slice", since = "1.17.0")] -impl From<&str> for Box { - /// Converts a `&str` into a `Box` - /// - /// This conversion allocates on the heap - /// and performs a copy of `s`. - /// - /// # Examples - /// - /// ```rust - /// let boxed: Box = Box::from("hello"); - /// println!("{boxed}"); - /// ``` - #[inline] - fn from(s: &str) -> Box { - unsafe { from_boxed_utf8_unchecked(Box::from(s.as_bytes())) } - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "box_from_cow", since = "1.45.0")] -impl From> for Box { - /// Converts a `Cow<'_, str>` into a `Box` - /// - /// When `cow` is the `Cow::Borrowed` variant, this - /// conversion allocates on the heap and copies the - /// underlying `str`. Otherwise, it will try to reuse the owned - /// `String`'s allocation. - /// - /// # Examples - /// - /// ```rust - /// use std::borrow::Cow; - /// - /// let unboxed = Cow::Borrowed("hello"); - /// let boxed: Box = Box::from(unboxed); - /// println!("{boxed}"); - /// ``` - /// - /// ```rust - /// # use std::borrow::Cow; - /// let unboxed = Cow::Owned("hello".to_string()); - /// let boxed: Box = Box::from(unboxed); - /// println!("{boxed}"); - /// ``` - #[inline] - fn from(cow: Cow<'_, str>) -> Box { - match cow { - Cow::Borrowed(s) => Box::from(s), - Cow::Owned(s) => Box::from(s), - } - } -} - -#[stable(feature = "boxed_str_conv", since = "1.19.0")] -impl From> for Box<[u8], A> { - /// Converts a `Box` into a `Box<[u8]>` - /// - /// This conversion does not allocate on the heap and happens in place. - /// - /// # Examples - /// ```rust - /// // create a Box which will be used to create a Box<[u8]> - /// let boxed: Box = Box::from("hello"); - /// let boxed_str: Box<[u8]> = Box::from(boxed); - /// - /// // create a &[u8] which will be used to create a Box<[u8]> - /// let slice: &[u8] = &[104, 101, 108, 108, 111]; - /// let boxed_slice = Box::from(slice); - /// - /// assert_eq!(boxed_slice, boxed_str); - /// ``` - #[inline] - fn from(s: Box) -> Self { - let (raw, alloc) = Box::into_raw_with_allocator(s); - unsafe { Box::from_raw_in(raw as *mut [u8], alloc) } - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "box_from_array", since = "1.45.0")] -impl From<[T; N]> for Box<[T]> { - /// Converts a `[T; N]` into a `Box<[T]>` - /// - /// This conversion moves the array to newly heap-allocated memory. - /// - /// # Examples - /// - /// ```rust - /// let boxed: Box<[u8]> = Box::from([4, 2]); - /// println!("{boxed:?}"); - /// ``` - fn from(array: [T; N]) -> Box<[T]> { - Box::new(array) - } -} - -/// Casts a boxed slice to a boxed array. -/// -/// # Safety -/// -/// `boxed_slice.len()` must be exactly `N`. -unsafe fn boxed_slice_as_array_unchecked( - boxed_slice: Box<[T], A>, -) -> Box<[T; N], A> { - debug_assert_eq!(boxed_slice.len(), N); - - let (ptr, alloc) = Box::into_raw_with_allocator(boxed_slice); - // SAFETY: Pointer and allocator came from an existing box, - // and our safety condition requires that the length is exactly `N` - unsafe { Box::from_raw_in(ptr as *mut [T; N], alloc) } -} - -#[stable(feature = "boxed_slice_try_from", since = "1.43.0")] -impl TryFrom> for Box<[T; N]> { - type Error = Box<[T]>; - - /// Attempts to convert a `Box<[T]>` into a `Box<[T; N]>`. - /// - /// The conversion occurs in-place and does not require a - /// new memory allocation. - /// - /// # Errors - /// - /// Returns the old `Box<[T]>` in the `Err` variant if - /// `boxed_slice.len()` does not equal `N`. - fn try_from(boxed_slice: Box<[T]>) -> Result { - if boxed_slice.len() == N { - Ok(unsafe { boxed_slice_as_array_unchecked(boxed_slice) }) - } else { - Err(boxed_slice) - } - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "boxed_array_try_from_vec", since = "1.66.0")] -impl TryFrom> for Box<[T; N]> { - type Error = Vec; - - /// Attempts to convert a `Vec` into a `Box<[T; N]>`. - /// - /// Like [`Vec::into_boxed_slice`], this is in-place if `vec.capacity() == N`, - /// but will require a reallocation otherwise. - /// - /// # Errors - /// - /// Returns the original `Vec` in the `Err` variant if - /// `boxed_slice.len()` does not equal `N`. - /// - /// # Examples - /// - /// This can be used with [`vec!`] to create an array on the heap: - /// - /// ``` - /// let state: Box<[f32; 100]> = vec![1.0; 100].try_into().unwrap(); - /// assert_eq!(state.len(), 100); - /// ``` - fn try_from(vec: Vec) -> Result { - if vec.len() == N { - let boxed_slice = vec.into_boxed_slice(); - Ok(unsafe { boxed_slice_as_array_unchecked(boxed_slice) }) - } else { - Err(vec) - } - } -} - -impl Box { - /// Attempts to downcast the box to a concrete type. - /// - /// # Examples - /// - /// ``` - /// use std::any::Any; - /// - /// fn print_if_string(value: Box) { - /// if let Ok(string) = value.downcast::() { - /// println!("String ({}): {}", string.len(), string); - /// } - /// } - /// - /// let my_string = "Hello World".to_string(); - /// print_if_string(Box::new(my_string)); - /// print_if_string(Box::new(0i8)); - /// ``` - #[inline] - #[stable(feature = "rust1", since = "1.0.0")] - pub fn downcast(self) -> Result, Self> { - if self.is::() { unsafe { Ok(self.downcast_unchecked::()) } } else { Err(self) } - } - - /// Downcasts the box to a concrete type. - /// - /// For a safe alternative see [`downcast`]. - /// - /// # Examples - /// - /// ``` - /// #![feature(downcast_unchecked)] - /// - /// use std::any::Any; - /// - /// let x: Box = Box::new(1_usize); - /// - /// unsafe { - /// assert_eq!(*x.downcast_unchecked::(), 1); - /// } - /// ``` - /// - /// # Safety - /// - /// The contained value must be of type `T`. Calling this method - /// with the incorrect type is *undefined behavior*. - /// - /// [`downcast`]: Self::downcast - #[inline] - #[unstable(feature = "downcast_unchecked", issue = "90850")] - pub unsafe fn downcast_unchecked(self) -> Box { - debug_assert!(self.is::()); - unsafe { - let (raw, alloc): (*mut dyn Any, _) = Box::into_raw_with_allocator(self); - Box::from_raw_in(raw as *mut T, alloc) - } - } -} - -impl Box { - /// Attempts to downcast the box to a concrete type. - /// - /// # Examples - /// - /// ``` - /// use std::any::Any; - /// - /// fn print_if_string(value: Box) { - /// if let Ok(string) = value.downcast::() { - /// println!("String ({}): {}", string.len(), string); - /// } - /// } - /// - /// let my_string = "Hello World".to_string(); - /// print_if_string(Box::new(my_string)); - /// print_if_string(Box::new(0i8)); - /// ``` - #[inline] - #[stable(feature = "rust1", since = "1.0.0")] - pub fn downcast(self) -> Result, Self> { - if self.is::() { unsafe { Ok(self.downcast_unchecked::()) } } else { Err(self) } - } - - /// Downcasts the box to a concrete type. - /// - /// For a safe alternative see [`downcast`]. - /// - /// # Examples - /// - /// ``` - /// #![feature(downcast_unchecked)] - /// - /// use std::any::Any; - /// - /// let x: Box = Box::new(1_usize); - /// - /// unsafe { - /// assert_eq!(*x.downcast_unchecked::(), 1); - /// } - /// ``` - /// - /// # Safety - /// - /// The contained value must be of type `T`. Calling this method - /// with the incorrect type is *undefined behavior*. - /// - /// [`downcast`]: Self::downcast - #[inline] - #[unstable(feature = "downcast_unchecked", issue = "90850")] - pub unsafe fn downcast_unchecked(self) -> Box { - debug_assert!(self.is::()); - unsafe { - let (raw, alloc): (*mut (dyn Any + Send), _) = Box::into_raw_with_allocator(self); - Box::from_raw_in(raw as *mut T, alloc) - } - } -} - -impl Box { - /// Attempts to downcast the box to a concrete type. - /// - /// # Examples - /// - /// ``` - /// use std::any::Any; - /// - /// fn print_if_string(value: Box) { - /// if let Ok(string) = value.downcast::() { - /// println!("String ({}): {}", string.len(), string); - /// } - /// } - /// - /// let my_string = "Hello World".to_string(); - /// print_if_string(Box::new(my_string)); - /// print_if_string(Box::new(0i8)); - /// ``` - #[inline] - #[stable(feature = "box_send_sync_any_downcast", since = "1.51.0")] - pub fn downcast(self) -> Result, Self> { - if self.is::() { unsafe { Ok(self.downcast_unchecked::()) } } else { Err(self) } - } - - /// Downcasts the box to a concrete type. - /// - /// For a safe alternative see [`downcast`]. - /// - /// # Examples - /// - /// ``` - /// #![feature(downcast_unchecked)] - /// - /// use std::any::Any; - /// - /// let x: Box = Box::new(1_usize); - /// - /// unsafe { - /// assert_eq!(*x.downcast_unchecked::(), 1); - /// } - /// ``` - /// - /// # Safety - /// - /// The contained value must be of type `T`. Calling this method - /// with the incorrect type is *undefined behavior*. - /// - /// [`downcast`]: Self::downcast - #[inline] - #[unstable(feature = "downcast_unchecked", issue = "90850")] - pub unsafe fn downcast_unchecked(self) -> Box { - debug_assert!(self.is::()); - unsafe { - let (raw, alloc): (*mut (dyn Any + Send + Sync), _) = - Box::into_raw_with_allocator(self); - Box::from_raw_in(raw as *mut T, alloc) - } - } -} - -#[stable(feature = "rust1", since = "1.0.0")] -impl fmt::Display for Box { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - fmt::Display::fmt(&**self, f) - } -} - -#[stable(feature = "rust1", since = "1.0.0")] -impl fmt::Debug for Box { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - fmt::Debug::fmt(&**self, f) - } -} - -#[stable(feature = "rust1", since = "1.0.0")] -impl fmt::Pointer for Box { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - // It's not possible to extract the inner Uniq directly from the Box, - // instead we cast it to a *const which aliases the Unique - let ptr: *const T = &**self; - fmt::Pointer::fmt(&ptr, f) - } -} - -#[stable(feature = "rust1", since = "1.0.0")] -impl Deref for Box { - type Target = T; - - fn deref(&self) -> &T { - &**self - } -} - -#[stable(feature = "rust1", since = "1.0.0")] -impl DerefMut for Box { - fn deref_mut(&mut self) -> &mut T { - &mut **self +#[stable(feature = "rust1", since = "1.0.0")] +impl DerefMut for Box { + fn deref_mut(&mut self) -> &mut T { + &mut **self } } #[unstable(feature = "deref_pure_trait", issue = "87121")] unsafe impl DerefPure for Box {} -#[unstable(feature = "receiver_trait", issue = "none")] -impl Receiver for Box {} - -#[stable(feature = "rust1", since = "1.0.0")] -impl Iterator for Box { - type Item = I::Item; - fn next(&mut self) -> Option { - (**self).next() - } - fn size_hint(&self) -> (usize, Option) { - (**self).size_hint() - } - fn nth(&mut self, n: usize) -> Option { - (**self).nth(n) - } - fn last(self) -> Option { - BoxIter::last(self) - } -} - -trait BoxIter { - type Item; - fn last(self) -> Option; -} - -impl BoxIter for Box { - type Item = I::Item; - default fn last(self) -> Option { - #[inline] - fn some(_: Option, x: T) -> Option { - Some(x) - } - - self.fold(None, some) - } -} - -/// Specialization for sized `I`s that uses `I`s implementation of `last()` -/// instead of the default. -#[stable(feature = "rust1", since = "1.0.0")] -impl BoxIter for Box { - fn last(self) -> Option { - (*self).last() - } -} - -#[stable(feature = "rust1", since = "1.0.0")] -impl DoubleEndedIterator for Box { - fn next_back(&mut self) -> Option { - (**self).next_back() - } - fn nth_back(&mut self, n: usize) -> Option { - (**self).nth_back(n) - } -} -#[stable(feature = "rust1", since = "1.0.0")] -impl ExactSizeIterator for Box { - fn len(&self) -> usize { - (**self).len() - } - fn is_empty(&self) -> bool { - (**self).is_empty() - } -} - -#[stable(feature = "fused", since = "1.26.0")] -impl FusedIterator for Box {} +#[unstable(feature = "legacy_receiver_trait", issue = "none")] +impl LegacyReceiver for Box {} #[stable(feature = "boxed_closure_impls", since = "1.35.0")] impl + ?Sized, A: Allocator> FnOnce for Box { @@ -2256,7 +1994,10 @@ impl + ?Sized, A: Allocator> AsyncFnOnce #[unstable(feature = "async_fn_traits", issue = "none")] impl + ?Sized, A: Allocator> AsyncFnMut for Box { - type CallRefFuture<'a> = F::CallRefFuture<'a> where Self: 'a; + type CallRefFuture<'a> + = F::CallRefFuture<'a> + where + Self: 'a; extern "rust-call" fn async_call_mut(&mut self, args: Args) -> Self::CallRefFuture<'_> { F::async_call_mut(self, args) @@ -2273,157 +2014,24 @@ impl + ?Sized, A: Allocator> AsyncFn for Box #[unstable(feature = "coerce_unsized", issue = "18598")] impl, U: ?Sized, A: Allocator> CoerceUnsized> for Box {} +#[unstable(feature = "pin_coerce_unsized_trait", issue = "123430")] +unsafe impl PinCoerceUnsized for Box {} + // It is quite crucial that we only allow the `Global` allocator here. // Handling arbitrary custom allocators (which can affect the `Box` layout heavily!) // would need a lot of codegen and interpreter adjustments. #[unstable(feature = "dispatch_from_dyn", issue = "none")] impl, U: ?Sized> DispatchFromDyn> for Box {} -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "boxed_slice_from_iter", since = "1.32.0")] -impl FromIterator for Box<[I]> { - fn from_iter>(iter: T) -> Self { - iter.into_iter().collect::>().into_boxed_slice() - } -} - -/// This implementation is required to make sure that the `Box<[I]>: IntoIterator` -/// implementation doesn't overlap with `IntoIterator for T where T: Iterator` blanket. -#[stable(feature = "boxed_slice_into_iter", since = "1.80.0")] -impl !Iterator for Box<[I], A> {} - -/// This implementation is required to make sure that the `&Box<[I]>: IntoIterator` -/// implementation doesn't overlap with `IntoIterator for T where T: Iterator` blanket. -#[stable(feature = "boxed_slice_into_iter", since = "1.80.0")] -impl<'a, I, A: Allocator> !Iterator for &'a Box<[I], A> {} - -/// This implementation is required to make sure that the `&mut Box<[I]>: IntoIterator` -/// implementation doesn't overlap with `IntoIterator for T where T: Iterator` blanket. -#[stable(feature = "boxed_slice_into_iter", since = "1.80.0")] -impl<'a, I, A: Allocator> !Iterator for &'a mut Box<[I], A> {} - -// Note: the `#[rustc_skip_during_method_dispatch(boxed_slice)]` on `trait IntoIterator` -// hides this implementation from explicit `.into_iter()` calls on editions < 2024, -// so those calls will still resolve to the slice implementation, by reference. -#[stable(feature = "boxed_slice_into_iter", since = "1.80.0")] -impl IntoIterator for Box<[I], A> { - type IntoIter = vec::IntoIter; - type Item = I; - fn into_iter(self) -> vec::IntoIter { - self.into_vec().into_iter() - } -} - -#[stable(feature = "boxed_slice_into_iter", since = "1.80.0")] -impl<'a, I, A: Allocator> IntoIterator for &'a Box<[I], A> { - type IntoIter = slice::Iter<'a, I>; - type Item = &'a I; - fn into_iter(self) -> slice::Iter<'a, I> { - self.iter() - } -} - -#[stable(feature = "boxed_slice_into_iter", since = "1.80.0")] -impl<'a, I, A: Allocator> IntoIterator for &'a mut Box<[I], A> { - type IntoIter = slice::IterMut<'a, I>; - type Item = &'a mut I; - fn into_iter(self) -> slice::IterMut<'a, I> { - self.iter_mut() - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "boxed_str_from_iter", since = "1.80.0")] -impl FromIterator for Box { - fn from_iter>(iter: T) -> Self { - String::from_iter(iter).into_boxed_str() - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "boxed_str_from_iter", since = "1.80.0")] -impl<'a> FromIterator<&'a char> for Box { - fn from_iter>(iter: T) -> Self { - String::from_iter(iter).into_boxed_str() - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "boxed_str_from_iter", since = "1.80.0")] -impl<'a> FromIterator<&'a str> for Box { - fn from_iter>(iter: T) -> Self { - String::from_iter(iter).into_boxed_str() - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "boxed_str_from_iter", since = "1.80.0")] -impl FromIterator for Box { - fn from_iter>(iter: T) -> Self { - String::from_iter(iter).into_boxed_str() - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "boxed_str_from_iter", since = "1.80.0")] -impl FromIterator> for Box { - fn from_iter>>(iter: T) -> Self { - String::from_iter(iter).into_boxed_str() - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "boxed_str_from_iter", since = "1.80.0")] -impl<'a> FromIterator> for Box { - fn from_iter>>(iter: T) -> Self { - String::from_iter(iter).into_boxed_str() - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "box_slice_clone", since = "1.3.0")] -impl Clone for Box<[T], A> { - fn clone(&self) -> Self { - let alloc = Box::allocator(self).clone(); - self.to_vec_in(alloc).into_boxed_slice() - } - - /// Copies `source`'s contents into `self` without creating a new allocation, - /// so long as the two are of the same length. - /// - /// # Examples - /// - /// ``` - /// let x = Box::new([5, 6, 7]); - /// let mut y = Box::new([8, 9, 10]); - /// let yp: *const [i32] = &*y; - /// - /// y.clone_from(&x); - /// - /// // The value is the same - /// assert_eq!(x, y); - /// - /// // And no allocation occurred - /// assert_eq!(yp, &*y); - /// ``` - fn clone_from(&mut self, source: &Self) { - if self.len() == source.len() { - self.clone_from_slice(&source); - } else { - *self = source.clone(); - } - } -} - #[stable(feature = "box_borrow", since = "1.1.0")] -impl borrow::Borrow for Box { +impl Borrow for Box { fn borrow(&self) -> &T { &**self } } #[stable(feature = "box_borrow", since = "1.1.0")] -impl borrow::BorrowMut for Box { +impl BorrowMut for Box { fn borrow_mut(&mut self) -> &mut T { &mut **self } @@ -2500,311 +2108,23 @@ impl Future for Box { } } -#[unstable(feature = "async_iterator", issue = "79024")] -impl AsyncIterator for Box { - type Item = S::Item; - - fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { - Pin::new(&mut **self).poll_next(cx) - } - - fn size_hint(&self) -> (usize, Option) { - (**self).size_hint() - } -} - -impl dyn Error { - #[inline] - #[stable(feature = "error_downcast", since = "1.3.0")] - #[rustc_allow_incoherent_impl] - /// Attempts to downcast the box to a concrete type. - pub fn downcast(self: Box) -> Result, Box> { - if self.is::() { - unsafe { - let raw: *mut dyn Error = Box::into_raw(self); - Ok(Box::from_raw(raw as *mut T)) - } - } else { - Err(self) - } - } -} - -impl dyn Error + Send { - #[inline] - #[stable(feature = "error_downcast", since = "1.3.0")] - #[rustc_allow_incoherent_impl] - /// Attempts to downcast the box to a concrete type. - pub fn downcast(self: Box) -> Result, Box> { - let err: Box = self; - ::downcast(err).map_err(|s| unsafe { - // Reapply the `Send` marker. - mem::transmute::, Box>(s) - }) - } -} - -impl dyn Error + Send + Sync { - #[inline] - #[stable(feature = "error_downcast", since = "1.3.0")] - #[rustc_allow_incoherent_impl] - /// Attempts to downcast the box to a concrete type. - pub fn downcast(self: Box) -> Result, Box> { - let err: Box = self; - ::downcast(err).map_err(|s| unsafe { - // Reapply the `Send + Sync` markers. - mem::transmute::, Box>(s) - }) - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "rust1", since = "1.0.0")] -impl<'a, E: Error + 'a> From for Box { - /// Converts a type of [`Error`] into a box of dyn [`Error`]. - /// - /// # Examples - /// - /// ``` - /// use std::error::Error; - /// use std::fmt; - /// use std::mem; - /// - /// #[derive(Debug)] - /// struct AnError; - /// - /// impl fmt::Display for AnError { - /// fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - /// write!(f, "An error") - /// } - /// } - /// - /// impl Error for AnError {} - /// - /// let an_error = AnError; - /// assert!(0 == mem::size_of_val(&an_error)); - /// let a_boxed_error = Box::::from(an_error); - /// assert!(mem::size_of::>() == mem::size_of_val(&a_boxed_error)) - /// ``` - fn from(err: E) -> Box { - Box::new(err) - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "rust1", since = "1.0.0")] -impl<'a, E: Error + Send + Sync + 'a> From for Box { - /// Converts a type of [`Error`] + [`Send`] + [`Sync`] into a box of - /// dyn [`Error`] + [`Send`] + [`Sync`]. - /// - /// # Examples - /// - /// ``` - /// use std::error::Error; - /// use std::fmt; - /// use std::mem; - /// - /// #[derive(Debug)] - /// struct AnError; - /// - /// impl fmt::Display for AnError { - /// fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - /// write!(f, "An error") - /// } - /// } - /// - /// impl Error for AnError {} - /// - /// unsafe impl Send for AnError {} - /// - /// unsafe impl Sync for AnError {} - /// - /// let an_error = AnError; - /// assert!(0 == mem::size_of_val(&an_error)); - /// let a_boxed_error = Box::::from(an_error); - /// assert!( - /// mem::size_of::>() == mem::size_of_val(&a_boxed_error)) - /// ``` - fn from(err: E) -> Box { - Box::new(err) - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "rust1", since = "1.0.0")] -impl<'a> From for Box { - /// Converts a [`String`] into a box of dyn [`Error`] + [`Send`] + [`Sync`]. - /// - /// # Examples - /// - /// ``` - /// use std::error::Error; - /// use std::mem; - /// - /// let a_string_error = "a string error".to_string(); - /// let a_boxed_error = Box::::from(a_string_error); - /// assert!( - /// mem::size_of::>() == mem::size_of_val(&a_boxed_error)) - /// ``` - #[inline] - fn from(err: String) -> Box { - struct StringError(String); - - impl Error for StringError { - #[allow(deprecated)] - fn description(&self) -> &str { - &self.0 - } - } - - impl fmt::Display for StringError { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - fmt::Display::fmt(&self.0, f) - } - } - - // Purposefully skip printing "StringError(..)" - impl fmt::Debug for StringError { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - fmt::Debug::fmt(&self.0, f) - } - } - - Box::new(StringError(err)) - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "string_box_error", since = "1.6.0")] -impl<'a> From for Box { - /// Converts a [`String`] into a box of dyn [`Error`]. - /// - /// # Examples - /// - /// ``` - /// use std::error::Error; - /// use std::mem; - /// - /// let a_string_error = "a string error".to_string(); - /// let a_boxed_error = Box::::from(a_string_error); - /// assert!(mem::size_of::>() == mem::size_of_val(&a_boxed_error)) - /// ``` - fn from(str_err: String) -> Box { - let err1: Box = From::from(str_err); - let err2: Box = err1; - err2 - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "rust1", since = "1.0.0")] -impl<'a> From<&str> for Box { - /// Converts a [`str`] into a box of dyn [`Error`] + [`Send`] + [`Sync`]. - /// - /// [`str`]: prim@str - /// - /// # Examples - /// - /// ``` - /// use std::error::Error; - /// use std::mem; - /// - /// let a_str_error = "a str error"; - /// let a_boxed_error = Box::::from(a_str_error); - /// assert!( - /// mem::size_of::>() == mem::size_of_val(&a_boxed_error)) - /// ``` - #[inline] - fn from(err: &str) -> Box { - From::from(String::from(err)) - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "string_box_error", since = "1.6.0")] -impl<'a> From<&str> for Box { - /// Converts a [`str`] into a box of dyn [`Error`]. - /// - /// [`str`]: prim@str - /// - /// # Examples - /// - /// ``` - /// use std::error::Error; - /// use std::mem; - /// - /// let a_str_error = "a str error"; - /// let a_boxed_error = Box::::from(a_str_error); - /// assert!(mem::size_of::>() == mem::size_of_val(&a_boxed_error)) - /// ``` - fn from(err: &str) -> Box { - From::from(String::from(err)) - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "cow_box_error", since = "1.22.0")] -impl<'a, 'b> From> for Box { - /// Converts a [`Cow`] into a box of dyn [`Error`] + [`Send`] + [`Sync`]. - /// - /// # Examples - /// - /// ``` - /// use std::error::Error; - /// use std::mem; - /// use std::borrow::Cow; - /// - /// let a_cow_str_error = Cow::from("a str error"); - /// let a_boxed_error = Box::::from(a_cow_str_error); - /// assert!( - /// mem::size_of::>() == mem::size_of_val(&a_boxed_error)) - /// ``` - fn from(err: Cow<'b, str>) -> Box { - From::from(String::from(err)) - } -} - -#[cfg(not(no_global_oom_handling))] -#[stable(feature = "cow_box_error", since = "1.22.0")] -impl<'a, 'b> From> for Box { - /// Converts a [`Cow`] into a box of dyn [`Error`]. - /// - /// # Examples - /// - /// ``` - /// use std::error::Error; - /// use std::mem; - /// use std::borrow::Cow; - /// - /// let a_cow_str_error = Cow::from("a str error"); - /// let a_boxed_error = Box::::from(a_cow_str_error); - /// assert!(mem::size_of::>() == mem::size_of_val(&a_boxed_error)) - /// ``` - fn from(err: Cow<'b, str>) -> Box { - From::from(String::from(err)) - } -} - #[stable(feature = "box_error", since = "1.8.0")] -impl core::error::Error for Box { +impl Error for Box { #[allow(deprecated, deprecated_in_future)] fn description(&self) -> &str { - core::error::Error::description(&**self) + Error::description(&**self) } #[allow(deprecated)] - fn cause(&self) -> Option<&dyn core::error::Error> { - core::error::Error::cause(&**self) + fn cause(&self) -> Option<&dyn Error> { + Error::cause(&**self) } - fn source(&self) -> Option<&(dyn core::error::Error + 'static)> { - core::error::Error::source(&**self) + fn source(&self) -> Option<&(dyn Error + 'static)> { + Error::source(&**self) } - fn provide<'b>(&'b self, request: &mut core::error::Request<'b>) { - core::error::Error::provide(&**self, request); + fn provide<'b>(&'b self, request: &mut error::Request<'b>) { + Error::provide(&**self, request); } } - -#[unstable(feature = "pin_coerce_unsized_trait", issue = "123430")] -unsafe impl PinCoerceUnsized for Box {} diff --git a/library/alloc/src/boxed/convert.rs b/library/alloc/src/boxed/convert.rs new file mode 100644 index 0000000000000..19a583ca546e4 --- /dev/null +++ b/library/alloc/src/boxed/convert.rs @@ -0,0 +1,747 @@ +use core::any::Any; +use core::error::Error; +use core::mem; +use core::pin::Pin; +#[cfg(not(no_global_oom_handling))] +use core::{fmt, ptr}; + +use crate::alloc::Allocator; +#[cfg(not(no_global_oom_handling))] +use crate::borrow::Cow; +use crate::boxed::Box; +#[cfg(not(no_global_oom_handling))] +use crate::raw_vec::RawVec; +#[cfg(not(no_global_oom_handling))] +use crate::str::from_boxed_utf8_unchecked; +#[cfg(not(no_global_oom_handling))] +use crate::string::String; +#[cfg(not(no_global_oom_handling))] +use crate::vec::Vec; + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "from_for_ptrs", since = "1.6.0")] +impl From for Box { + /// Converts a `T` into a `Box` + /// + /// The conversion allocates on the heap and moves `t` + /// from the stack into it. + /// + /// # Examples + /// + /// ```rust + /// let x = 5; + /// let boxed = Box::new(5); + /// + /// assert_eq!(Box::from(x), boxed); + /// ``` + fn from(t: T) -> Self { + Box::new(t) + } +} + +#[stable(feature = "pin", since = "1.33.0")] +impl From> for Pin> +where + A: 'static, +{ + /// Converts a `Box` into a `Pin>`. If `T` does not implement [`Unpin`], then + /// `*boxed` will be pinned in memory and unable to be moved. + /// + /// This conversion does not allocate on the heap and happens in place. + /// + /// This is also available via [`Box::into_pin`]. + /// + /// Constructing and pinning a `Box` with >>::from([Box::new]\(x)) + /// can also be written more concisely using [Box::pin]\(x). + /// This `From` implementation is useful if you already have a `Box`, or you are + /// constructing a (pinned) `Box` in a different way than with [`Box::new`]. + fn from(boxed: Box) -> Self { + Box::into_pin(boxed) + } +} + +/// Specialization trait used for `From<&[T]>`. +#[cfg(not(no_global_oom_handling))] +trait BoxFromSlice { + fn from_slice(slice: &[T]) -> Self; +} + +#[cfg(not(no_global_oom_handling))] +impl BoxFromSlice for Box<[T]> { + #[inline] + default fn from_slice(slice: &[T]) -> Self { + slice.to_vec().into_boxed_slice() + } +} + +#[cfg(not(no_global_oom_handling))] +impl BoxFromSlice for Box<[T]> { + #[inline] + fn from_slice(slice: &[T]) -> Self { + let len = slice.len(); + let buf = RawVec::with_capacity(len); + unsafe { + ptr::copy_nonoverlapping(slice.as_ptr(), buf.ptr(), len); + buf.into_box(slice.len()).assume_init() + } + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "box_from_slice", since = "1.17.0")] +impl From<&[T]> for Box<[T]> { + /// Converts a `&[T]` into a `Box<[T]>` + /// + /// This conversion allocates on the heap + /// and performs a copy of `slice` and its contents. + /// + /// # Examples + /// ```rust + /// // create a &[u8] which will be used to create a Box<[u8]> + /// let slice: &[u8] = &[104, 101, 108, 108, 111]; + /// let boxed_slice: Box<[u8]> = Box::from(slice); + /// + /// println!("{boxed_slice:?}"); + /// ``` + #[inline] + fn from(slice: &[T]) -> Box<[T]> { + >::from_slice(slice) + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "box_from_cow", since = "1.45.0")] +impl From> for Box<[T]> { + /// Converts a `Cow<'_, [T]>` into a `Box<[T]>` + /// + /// When `cow` is the `Cow::Borrowed` variant, this + /// conversion allocates on the heap and copies the + /// underlying slice. Otherwise, it will try to reuse the owned + /// `Vec`'s allocation. + #[inline] + fn from(cow: Cow<'_, [T]>) -> Box<[T]> { + match cow { + Cow::Borrowed(slice) => Box::from(slice), + Cow::Owned(slice) => Box::from(slice), + } + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "box_from_slice", since = "1.17.0")] +impl From<&str> for Box { + /// Converts a `&str` into a `Box` + /// + /// This conversion allocates on the heap + /// and performs a copy of `s`. + /// + /// # Examples + /// + /// ```rust + /// let boxed: Box = Box::from("hello"); + /// println!("{boxed}"); + /// ``` + #[inline] + fn from(s: &str) -> Box { + unsafe { from_boxed_utf8_unchecked(Box::from(s.as_bytes())) } + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "box_from_cow", since = "1.45.0")] +impl From> for Box { + /// Converts a `Cow<'_, str>` into a `Box` + /// + /// When `cow` is the `Cow::Borrowed` variant, this + /// conversion allocates on the heap and copies the + /// underlying `str`. Otherwise, it will try to reuse the owned + /// `String`'s allocation. + /// + /// # Examples + /// + /// ```rust + /// use std::borrow::Cow; + /// + /// let unboxed = Cow::Borrowed("hello"); + /// let boxed: Box = Box::from(unboxed); + /// println!("{boxed}"); + /// ``` + /// + /// ```rust + /// # use std::borrow::Cow; + /// let unboxed = Cow::Owned("hello".to_string()); + /// let boxed: Box = Box::from(unboxed); + /// println!("{boxed}"); + /// ``` + #[inline] + fn from(cow: Cow<'_, str>) -> Box { + match cow { + Cow::Borrowed(s) => Box::from(s), + Cow::Owned(s) => Box::from(s), + } + } +} + +#[stable(feature = "boxed_str_conv", since = "1.19.0")] +impl From> for Box<[u8], A> { + /// Converts a `Box` into a `Box<[u8]>` + /// + /// This conversion does not allocate on the heap and happens in place. + /// + /// # Examples + /// ```rust + /// // create a Box which will be used to create a Box<[u8]> + /// let boxed: Box = Box::from("hello"); + /// let boxed_str: Box<[u8]> = Box::from(boxed); + /// + /// // create a &[u8] which will be used to create a Box<[u8]> + /// let slice: &[u8] = &[104, 101, 108, 108, 111]; + /// let boxed_slice = Box::from(slice); + /// + /// assert_eq!(boxed_slice, boxed_str); + /// ``` + #[inline] + fn from(s: Box) -> Self { + let (raw, alloc) = Box::into_raw_with_allocator(s); + unsafe { Box::from_raw_in(raw as *mut [u8], alloc) } + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "box_from_array", since = "1.45.0")] +impl From<[T; N]> for Box<[T]> { + /// Converts a `[T; N]` into a `Box<[T]>` + /// + /// This conversion moves the array to newly heap-allocated memory. + /// + /// # Examples + /// + /// ```rust + /// let boxed: Box<[u8]> = Box::from([4, 2]); + /// println!("{boxed:?}"); + /// ``` + fn from(array: [T; N]) -> Box<[T]> { + Box::new(array) + } +} + +/// Casts a boxed slice to a boxed array. +/// +/// # Safety +/// +/// `boxed_slice.len()` must be exactly `N`. +unsafe fn boxed_slice_as_array_unchecked( + boxed_slice: Box<[T], A>, +) -> Box<[T; N], A> { + debug_assert_eq!(boxed_slice.len(), N); + + let (ptr, alloc) = Box::into_raw_with_allocator(boxed_slice); + // SAFETY: Pointer and allocator came from an existing box, + // and our safety condition requires that the length is exactly `N` + unsafe { Box::from_raw_in(ptr as *mut [T; N], alloc) } +} + +#[stable(feature = "boxed_slice_try_from", since = "1.43.0")] +impl TryFrom> for Box<[T; N]> { + type Error = Box<[T]>; + + /// Attempts to convert a `Box<[T]>` into a `Box<[T; N]>`. + /// + /// The conversion occurs in-place and does not require a + /// new memory allocation. + /// + /// # Errors + /// + /// Returns the old `Box<[T]>` in the `Err` variant if + /// `boxed_slice.len()` does not equal `N`. + fn try_from(boxed_slice: Box<[T]>) -> Result { + if boxed_slice.len() == N { + Ok(unsafe { boxed_slice_as_array_unchecked(boxed_slice) }) + } else { + Err(boxed_slice) + } + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "boxed_array_try_from_vec", since = "1.66.0")] +impl TryFrom> for Box<[T; N]> { + type Error = Vec; + + /// Attempts to convert a `Vec` into a `Box<[T; N]>`. + /// + /// Like [`Vec::into_boxed_slice`], this is in-place if `vec.capacity() == N`, + /// but will require a reallocation otherwise. + /// + /// # Errors + /// + /// Returns the original `Vec` in the `Err` variant if + /// `boxed_slice.len()` does not equal `N`. + /// + /// # Examples + /// + /// This can be used with [`vec!`] to create an array on the heap: + /// + /// ``` + /// let state: Box<[f32; 100]> = vec![1.0; 100].try_into().unwrap(); + /// assert_eq!(state.len(), 100); + /// ``` + fn try_from(vec: Vec) -> Result { + if vec.len() == N { + let boxed_slice = vec.into_boxed_slice(); + Ok(unsafe { boxed_slice_as_array_unchecked(boxed_slice) }) + } else { + Err(vec) + } + } +} + +impl Box { + /// Attempts to downcast the box to a concrete type. + /// + /// # Examples + /// + /// ``` + /// use std::any::Any; + /// + /// fn print_if_string(value: Box) { + /// if let Ok(string) = value.downcast::() { + /// println!("String ({}): {}", string.len(), string); + /// } + /// } + /// + /// let my_string = "Hello World".to_string(); + /// print_if_string(Box::new(my_string)); + /// print_if_string(Box::new(0i8)); + /// ``` + #[inline] + #[stable(feature = "rust1", since = "1.0.0")] + pub fn downcast(self) -> Result, Self> { + if self.is::() { unsafe { Ok(self.downcast_unchecked::()) } } else { Err(self) } + } + + /// Downcasts the box to a concrete type. + /// + /// For a safe alternative see [`downcast`]. + /// + /// # Examples + /// + /// ``` + /// #![feature(downcast_unchecked)] + /// + /// use std::any::Any; + /// + /// let x: Box = Box::new(1_usize); + /// + /// unsafe { + /// assert_eq!(*x.downcast_unchecked::(), 1); + /// } + /// ``` + /// + /// # Safety + /// + /// The contained value must be of type `T`. Calling this method + /// with the incorrect type is *undefined behavior*. + /// + /// [`downcast`]: Self::downcast + #[inline] + #[unstable(feature = "downcast_unchecked", issue = "90850")] + pub unsafe fn downcast_unchecked(self) -> Box { + debug_assert!(self.is::()); + unsafe { + let (raw, alloc): (*mut dyn Any, _) = Box::into_raw_with_allocator(self); + Box::from_raw_in(raw as *mut T, alloc) + } + } +} + +impl Box { + /// Attempts to downcast the box to a concrete type. + /// + /// # Examples + /// + /// ``` + /// use std::any::Any; + /// + /// fn print_if_string(value: Box) { + /// if let Ok(string) = value.downcast::() { + /// println!("String ({}): {}", string.len(), string); + /// } + /// } + /// + /// let my_string = "Hello World".to_string(); + /// print_if_string(Box::new(my_string)); + /// print_if_string(Box::new(0i8)); + /// ``` + #[inline] + #[stable(feature = "rust1", since = "1.0.0")] + pub fn downcast(self) -> Result, Self> { + if self.is::() { unsafe { Ok(self.downcast_unchecked::()) } } else { Err(self) } + } + + /// Downcasts the box to a concrete type. + /// + /// For a safe alternative see [`downcast`]. + /// + /// # Examples + /// + /// ``` + /// #![feature(downcast_unchecked)] + /// + /// use std::any::Any; + /// + /// let x: Box = Box::new(1_usize); + /// + /// unsafe { + /// assert_eq!(*x.downcast_unchecked::(), 1); + /// } + /// ``` + /// + /// # Safety + /// + /// The contained value must be of type `T`. Calling this method + /// with the incorrect type is *undefined behavior*. + /// + /// [`downcast`]: Self::downcast + #[inline] + #[unstable(feature = "downcast_unchecked", issue = "90850")] + pub unsafe fn downcast_unchecked(self) -> Box { + debug_assert!(self.is::()); + unsafe { + let (raw, alloc): (*mut (dyn Any + Send), _) = Box::into_raw_with_allocator(self); + Box::from_raw_in(raw as *mut T, alloc) + } + } +} + +impl Box { + /// Attempts to downcast the box to a concrete type. + /// + /// # Examples + /// + /// ``` + /// use std::any::Any; + /// + /// fn print_if_string(value: Box) { + /// if let Ok(string) = value.downcast::() { + /// println!("String ({}): {}", string.len(), string); + /// } + /// } + /// + /// let my_string = "Hello World".to_string(); + /// print_if_string(Box::new(my_string)); + /// print_if_string(Box::new(0i8)); + /// ``` + #[inline] + #[stable(feature = "box_send_sync_any_downcast", since = "1.51.0")] + pub fn downcast(self) -> Result, Self> { + if self.is::() { unsafe { Ok(self.downcast_unchecked::()) } } else { Err(self) } + } + + /// Downcasts the box to a concrete type. + /// + /// For a safe alternative see [`downcast`]. + /// + /// # Examples + /// + /// ``` + /// #![feature(downcast_unchecked)] + /// + /// use std::any::Any; + /// + /// let x: Box = Box::new(1_usize); + /// + /// unsafe { + /// assert_eq!(*x.downcast_unchecked::(), 1); + /// } + /// ``` + /// + /// # Safety + /// + /// The contained value must be of type `T`. Calling this method + /// with the incorrect type is *undefined behavior*. + /// + /// [`downcast`]: Self::downcast + #[inline] + #[unstable(feature = "downcast_unchecked", issue = "90850")] + pub unsafe fn downcast_unchecked(self) -> Box { + debug_assert!(self.is::()); + unsafe { + let (raw, alloc): (*mut (dyn Any + Send + Sync), _) = + Box::into_raw_with_allocator(self); + Box::from_raw_in(raw as *mut T, alloc) + } + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "rust1", since = "1.0.0")] +impl<'a, E: Error + 'a> From for Box { + /// Converts a type of [`Error`] into a box of dyn [`Error`]. + /// + /// # Examples + /// + /// ``` + /// use std::error::Error; + /// use std::fmt; + /// use std::mem; + /// + /// #[derive(Debug)] + /// struct AnError; + /// + /// impl fmt::Display for AnError { + /// fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + /// write!(f, "An error") + /// } + /// } + /// + /// impl Error for AnError {} + /// + /// let an_error = AnError; + /// assert!(0 == mem::size_of_val(&an_error)); + /// let a_boxed_error = Box::::from(an_error); + /// assert!(mem::size_of::>() == mem::size_of_val(&a_boxed_error)) + /// ``` + fn from(err: E) -> Box { + Box::new(err) + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "rust1", since = "1.0.0")] +impl<'a, E: Error + Send + Sync + 'a> From for Box { + /// Converts a type of [`Error`] + [`Send`] + [`Sync`] into a box of + /// dyn [`Error`] + [`Send`] + [`Sync`]. + /// + /// # Examples + /// + /// ``` + /// use std::error::Error; + /// use std::fmt; + /// use std::mem; + /// + /// #[derive(Debug)] + /// struct AnError; + /// + /// impl fmt::Display for AnError { + /// fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + /// write!(f, "An error") + /// } + /// } + /// + /// impl Error for AnError {} + /// + /// unsafe impl Send for AnError {} + /// + /// unsafe impl Sync for AnError {} + /// + /// let an_error = AnError; + /// assert!(0 == mem::size_of_val(&an_error)); + /// let a_boxed_error = Box::::from(an_error); + /// assert!( + /// mem::size_of::>() == mem::size_of_val(&a_boxed_error)) + /// ``` + fn from(err: E) -> Box { + Box::new(err) + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "rust1", since = "1.0.0")] +impl<'a> From for Box { + /// Converts a [`String`] into a box of dyn [`Error`] + [`Send`] + [`Sync`]. + /// + /// # Examples + /// + /// ``` + /// use std::error::Error; + /// use std::mem; + /// + /// let a_string_error = "a string error".to_string(); + /// let a_boxed_error = Box::::from(a_string_error); + /// assert!( + /// mem::size_of::>() == mem::size_of_val(&a_boxed_error)) + /// ``` + #[inline] + fn from(err: String) -> Box { + struct StringError(String); + + impl Error for StringError { + #[allow(deprecated)] + fn description(&self) -> &str { + &self.0 + } + } + + impl fmt::Display for StringError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + fmt::Display::fmt(&self.0, f) + } + } + + // Purposefully skip printing "StringError(..)" + impl fmt::Debug for StringError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + fmt::Debug::fmt(&self.0, f) + } + } + + Box::new(StringError(err)) + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "string_box_error", since = "1.6.0")] +impl<'a> From for Box { + /// Converts a [`String`] into a box of dyn [`Error`]. + /// + /// # Examples + /// + /// ``` + /// use std::error::Error; + /// use std::mem; + /// + /// let a_string_error = "a string error".to_string(); + /// let a_boxed_error = Box::::from(a_string_error); + /// assert!(mem::size_of::>() == mem::size_of_val(&a_boxed_error)) + /// ``` + fn from(str_err: String) -> Box { + let err1: Box = From::from(str_err); + let err2: Box = err1; + err2 + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "rust1", since = "1.0.0")] +impl<'a> From<&str> for Box { + /// Converts a [`str`] into a box of dyn [`Error`] + [`Send`] + [`Sync`]. + /// + /// [`str`]: prim@str + /// + /// # Examples + /// + /// ``` + /// use std::error::Error; + /// use std::mem; + /// + /// let a_str_error = "a str error"; + /// let a_boxed_error = Box::::from(a_str_error); + /// assert!( + /// mem::size_of::>() == mem::size_of_val(&a_boxed_error)) + /// ``` + #[inline] + fn from(err: &str) -> Box { + From::from(String::from(err)) + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "string_box_error", since = "1.6.0")] +impl<'a> From<&str> for Box { + /// Converts a [`str`] into a box of dyn [`Error`]. + /// + /// [`str`]: prim@str + /// + /// # Examples + /// + /// ``` + /// use std::error::Error; + /// use std::mem; + /// + /// let a_str_error = "a str error"; + /// let a_boxed_error = Box::::from(a_str_error); + /// assert!(mem::size_of::>() == mem::size_of_val(&a_boxed_error)) + /// ``` + fn from(err: &str) -> Box { + From::from(String::from(err)) + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "cow_box_error", since = "1.22.0")] +impl<'a, 'b> From> for Box { + /// Converts a [`Cow`] into a box of dyn [`Error`] + [`Send`] + [`Sync`]. + /// + /// # Examples + /// + /// ``` + /// use std::error::Error; + /// use std::mem; + /// use std::borrow::Cow; + /// + /// let a_cow_str_error = Cow::from("a str error"); + /// let a_boxed_error = Box::::from(a_cow_str_error); + /// assert!( + /// mem::size_of::>() == mem::size_of_val(&a_boxed_error)) + /// ``` + fn from(err: Cow<'b, str>) -> Box { + From::from(String::from(err)) + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "cow_box_error", since = "1.22.0")] +impl<'a, 'b> From> for Box { + /// Converts a [`Cow`] into a box of dyn [`Error`]. + /// + /// # Examples + /// + /// ``` + /// use std::error::Error; + /// use std::mem; + /// use std::borrow::Cow; + /// + /// let a_cow_str_error = Cow::from("a str error"); + /// let a_boxed_error = Box::::from(a_cow_str_error); + /// assert!(mem::size_of::>() == mem::size_of_val(&a_boxed_error)) + /// ``` + fn from(err: Cow<'b, str>) -> Box { + From::from(String::from(err)) + } +} + +impl dyn Error { + /// Attempts to downcast the box to a concrete type. + #[inline] + #[stable(feature = "error_downcast", since = "1.3.0")] + #[rustc_allow_incoherent_impl] + pub fn downcast(self: Box) -> Result, Box> { + if self.is::() { + unsafe { + let raw: *mut dyn Error = Box::into_raw(self); + Ok(Box::from_raw(raw as *mut T)) + } + } else { + Err(self) + } + } +} + +impl dyn Error + Send { + /// Attempts to downcast the box to a concrete type. + #[inline] + #[stable(feature = "error_downcast", since = "1.3.0")] + #[rustc_allow_incoherent_impl] + pub fn downcast(self: Box) -> Result, Box> { + let err: Box = self; + ::downcast(err).map_err(|s| unsafe { + // Reapply the `Send` marker. + mem::transmute::, Box>(s) + }) + } +} + +impl dyn Error + Send + Sync { + /// Attempts to downcast the box to a concrete type. + #[inline] + #[stable(feature = "error_downcast", since = "1.3.0")] + #[rustc_allow_incoherent_impl] + pub fn downcast(self: Box) -> Result, Box> { + let err: Box = self; + ::downcast(err).map_err(|s| unsafe { + // Reapply the `Send + Sync` markers. + mem::transmute::, Box>(s) + }) + } +} diff --git a/library/alloc/src/boxed/iter.rs b/library/alloc/src/boxed/iter.rs new file mode 100644 index 0000000000000..90582aa49c6d7 --- /dev/null +++ b/library/alloc/src/boxed/iter.rs @@ -0,0 +1,194 @@ +use core::async_iter::AsyncIterator; +use core::iter::FusedIterator; +use core::pin::Pin; +use core::slice; +use core::task::{Context, Poll}; + +use crate::alloc::Allocator; +#[cfg(not(no_global_oom_handling))] +use crate::borrow::Cow; +use crate::boxed::Box; +#[cfg(not(no_global_oom_handling))] +use crate::string::String; +use crate::vec; +#[cfg(not(no_global_oom_handling))] +use crate::vec::Vec; + +#[stable(feature = "rust1", since = "1.0.0")] +impl Iterator for Box { + type Item = I::Item; + fn next(&mut self) -> Option { + (**self).next() + } + fn size_hint(&self) -> (usize, Option) { + (**self).size_hint() + } + fn nth(&mut self, n: usize) -> Option { + (**self).nth(n) + } + fn last(self) -> Option { + BoxIter::last(self) + } +} + +trait BoxIter { + type Item; + fn last(self) -> Option; +} + +impl BoxIter for Box { + type Item = I::Item; + default fn last(self) -> Option { + #[inline] + fn some(_: Option, x: T) -> Option { + Some(x) + } + + self.fold(None, some) + } +} + +/// Specialization for sized `I`s that uses `I`s implementation of `last()` +/// instead of the default. +#[stable(feature = "rust1", since = "1.0.0")] +impl BoxIter for Box { + fn last(self) -> Option { + (*self).last() + } +} + +#[stable(feature = "rust1", since = "1.0.0")] +impl DoubleEndedIterator for Box { + fn next_back(&mut self) -> Option { + (**self).next_back() + } + fn nth_back(&mut self, n: usize) -> Option { + (**self).nth_back(n) + } +} +#[stable(feature = "rust1", since = "1.0.0")] +impl ExactSizeIterator for Box { + fn len(&self) -> usize { + (**self).len() + } + fn is_empty(&self) -> bool { + (**self).is_empty() + } +} + +#[stable(feature = "fused", since = "1.26.0")] +impl FusedIterator for Box {} + +#[unstable(feature = "async_iterator", issue = "79024")] +impl AsyncIterator for Box { + type Item = S::Item; + + fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + Pin::new(&mut **self).poll_next(cx) + } + + fn size_hint(&self) -> (usize, Option) { + (**self).size_hint() + } +} + +/// This implementation is required to make sure that the `Box<[I]>: IntoIterator` +/// implementation doesn't overlap with `IntoIterator for T where T: Iterator` blanket. +#[stable(feature = "boxed_slice_into_iter", since = "1.80.0")] +impl !Iterator for Box<[I], A> {} + +/// This implementation is required to make sure that the `&Box<[I]>: IntoIterator` +/// implementation doesn't overlap with `IntoIterator for T where T: Iterator` blanket. +#[stable(feature = "boxed_slice_into_iter", since = "1.80.0")] +impl<'a, I, A: Allocator> !Iterator for &'a Box<[I], A> {} + +/// This implementation is required to make sure that the `&mut Box<[I]>: IntoIterator` +/// implementation doesn't overlap with `IntoIterator for T where T: Iterator` blanket. +#[stable(feature = "boxed_slice_into_iter", since = "1.80.0")] +impl<'a, I, A: Allocator> !Iterator for &'a mut Box<[I], A> {} + +// Note: the `#[rustc_skip_during_method_dispatch(boxed_slice)]` on `trait IntoIterator` +// hides this implementation from explicit `.into_iter()` calls on editions < 2024, +// so those calls will still resolve to the slice implementation, by reference. +#[stable(feature = "boxed_slice_into_iter", since = "1.80.0")] +impl IntoIterator for Box<[I], A> { + type IntoIter = vec::IntoIter; + type Item = I; + fn into_iter(self) -> vec::IntoIter { + self.into_vec().into_iter() + } +} + +#[stable(feature = "boxed_slice_into_iter", since = "1.80.0")] +impl<'a, I, A: Allocator> IntoIterator for &'a Box<[I], A> { + type IntoIter = slice::Iter<'a, I>; + type Item = &'a I; + fn into_iter(self) -> slice::Iter<'a, I> { + self.iter() + } +} + +#[stable(feature = "boxed_slice_into_iter", since = "1.80.0")] +impl<'a, I, A: Allocator> IntoIterator for &'a mut Box<[I], A> { + type IntoIter = slice::IterMut<'a, I>; + type Item = &'a mut I; + fn into_iter(self) -> slice::IterMut<'a, I> { + self.iter_mut() + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "boxed_slice_from_iter", since = "1.32.0")] +impl FromIterator for Box<[I]> { + fn from_iter>(iter: T) -> Self { + iter.into_iter().collect::>().into_boxed_slice() + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "boxed_str_from_iter", since = "1.80.0")] +impl FromIterator for Box { + fn from_iter>(iter: T) -> Self { + String::from_iter(iter).into_boxed_str() + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "boxed_str_from_iter", since = "1.80.0")] +impl<'a> FromIterator<&'a char> for Box { + fn from_iter>(iter: T) -> Self { + String::from_iter(iter).into_boxed_str() + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "boxed_str_from_iter", since = "1.80.0")] +impl<'a> FromIterator<&'a str> for Box { + fn from_iter>(iter: T) -> Self { + String::from_iter(iter).into_boxed_str() + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "boxed_str_from_iter", since = "1.80.0")] +impl FromIterator for Box { + fn from_iter>(iter: T) -> Self { + String::from_iter(iter).into_boxed_str() + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "boxed_str_from_iter", since = "1.80.0")] +impl FromIterator> for Box { + fn from_iter>>(iter: T) -> Self { + String::from_iter(iter).into_boxed_str() + } +} + +#[cfg(not(no_global_oom_handling))] +#[stable(feature = "boxed_str_from_iter", since = "1.80.0")] +impl<'a> FromIterator> for Box { + fn from_iter>>(iter: T) -> Self { + String::from_iter(iter).into_boxed_str() + } +} diff --git a/library/alloc/src/boxed/thin.rs b/library/alloc/src/boxed/thin.rs index 9baded3a52141..78e5aec09b18d 100644 --- a/library/alloc/src/boxed/thin.rs +++ b/library/alloc/src/boxed/thin.rs @@ -186,7 +186,7 @@ impl ThinBox { fn with_header(&self) -> &WithHeader<::Metadata> { // SAFETY: both types are transparent to `NonNull` - unsafe { &*(core::ptr::addr_of!(self.ptr) as *const WithHeader<_>) } + unsafe { &*((&raw const self.ptr) as *const WithHeader<_>) } } } diff --git a/library/alloc/src/collections/binary_heap/mod.rs b/library/alloc/src/collections/binary_heap/mod.rs index fe9f1010d327c..59f10b09c73fd 100644 --- a/library/alloc/src/collections/binary_heap/mod.rs +++ b/library/alloc/src/collections/binary_heap/mod.rs @@ -145,7 +145,7 @@ use core::alloc::Allocator; use core::iter::{FusedIterator, InPlaceIterable, SourceIter, TrustedFused, TrustedLen}; -use core::mem::{self, swap, ManuallyDrop}; +use core::mem::{self, ManuallyDrop, swap}; use core::num::NonZero; use core::ops::{Deref, DerefMut}; use core::{fmt, ptr}; @@ -374,7 +374,10 @@ impl<'a, T: Ord, A: Allocator> PeekMut<'a, T, A> { // the caller could've mutated the element. It is removed from the // heap on the next line and pop() is not sensitive to its value. } - this.heap.pop().unwrap() + + // SAFETY: Have a `PeekMut` element proves that the associated binary heap being non-empty, + // so the `pop` operation will not fail. + unsafe { this.heap.pop().unwrap_unchecked() } } } @@ -959,6 +962,7 @@ impl BinaryHeap { /// } /// ``` #[stable(feature = "rust1", since = "1.0.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "binaryheap_iter")] pub fn iter(&self) -> Iter<'_, T> { Iter { iter: self.data.iter() } } diff --git a/library/alloc/src/collections/binary_heap/tests.rs b/library/alloc/src/collections/binary_heap/tests.rs index 1cb07c6214953..ad0a020a1a961 100644 --- a/library/alloc/src/collections/binary_heap/tests.rs +++ b/library/alloc/src/collections/binary_heap/tests.rs @@ -1,4 +1,4 @@ -use std::panic::{catch_unwind, AssertUnwindSafe}; +use std::panic::{AssertUnwindSafe, catch_unwind}; use super::*; use crate::boxed::Box; @@ -350,7 +350,7 @@ fn test_drain_forget() { mem::forget(it); })) .unwrap(); - // Behaviour after leaking is explicitly unspecified and order is arbitrary, + // Behavior after leaking is explicitly unspecified and order is arbitrary, // so it's fine if these start failing, but probably worth knowing. assert!(q.is_empty()); assert_eq!(a.dropped() + b.dropped() + c.dropped(), 1); @@ -377,7 +377,7 @@ fn test_drain_sorted_forget() { mem::forget(it); })) .unwrap(); - // Behaviour after leaking is explicitly unspecified, + // Behavior after leaking is explicitly unspecified, // so it's fine if these start failing, but probably worth knowing. assert_eq!(q.len(), 2); assert_eq!(a.dropped(), 0); diff --git a/library/alloc/src/collections/btree/append.rs b/library/alloc/src/collections/btree/append.rs index 47372938fbeda..d137d2721ee4f 100644 --- a/library/alloc/src/collections/btree/append.rs +++ b/library/alloc/src/collections/btree/append.rs @@ -79,7 +79,7 @@ impl Root { } open_node.push(key, value, right_tree); - // Go down to the right-most leaf again. + // Go down to the rightmost leaf again. cur_node = open_node.forget_type().last_leaf_edge().into_node(); } diff --git a/library/alloc/src/collections/btree/fix.rs b/library/alloc/src/collections/btree/fix.rs index 4c1e19ead4031..09edea3555ad5 100644 --- a/library/alloc/src/collections/btree/fix.rs +++ b/library/alloc/src/collections/btree/fix.rs @@ -3,7 +3,7 @@ use core::alloc::Allocator; use super::map::MIN_LEN; use super::node::ForceResult::*; use super::node::LeftOrRight::*; -use super::node::{marker, Handle, NodeRef, Root}; +use super::node::{Handle, NodeRef, Root, marker}; impl<'a, K: 'a, V: 'a> NodeRef, K, V, marker::LeafOrInternal> { /// Stocks up a possibly underfull node by merging with or stealing from a @@ -102,7 +102,7 @@ impl Root { pub fn fix_right_border_of_plentiful(&mut self) { let mut cur_node = self.borrow_mut(); while let Internal(internal) = cur_node.force() { - // Check if right-most child is underfull. + // Check if rightmost child is underfull. let mut last_kv = internal.last_kv().consider_for_balancing(); debug_assert!(last_kv.left_child_len() >= MIN_LEN * 2); let right_child_len = last_kv.right_child_len(); diff --git a/library/alloc/src/collections/btree/map.rs b/library/alloc/src/collections/btree/map.rs index 60e08b47e3d35..55649d865fc63 100644 --- a/library/alloc/src/collections/btree/map.rs +++ b/library/alloc/src/collections/btree/map.rs @@ -13,7 +13,7 @@ use super::borrow::DormantMutRef; use super::dedup_sorted_iter::DedupSortedIter; use super::navigate::{LazyLeafRange, LeafRange}; use super::node::ForceResult::*; -use super::node::{self, marker, Handle, NodeRef, Root}; +use super::node::{self, Handle, NodeRef, Root, marker}; use super::search::SearchBound; use super::search::SearchResult::*; use super::set_val::SetValZST; @@ -22,9 +22,9 @@ use crate::vec::Vec; mod entry; +use Entry::*; #[stable(feature = "rust1", since = "1.0.0")] pub use entry::{Entry, OccupiedEntry, OccupiedError, VacantEntry}; -use Entry::*; /// Minimum number of elements in a node that is not a root. /// We might temporarily have fewer elements during methods. @@ -916,6 +916,7 @@ impl BTreeMap { /// assert_eq!(map.contains_key(&2), false); /// ``` #[stable(feature = "rust1", since = "1.0.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "btreemap_contains_key")] pub fn contains_key(&self, key: &Q) -> bool where K: Borrow + Ord, @@ -981,6 +982,7 @@ impl BTreeMap { /// ``` #[stable(feature = "rust1", since = "1.0.0")] #[rustc_confusables("push", "put", "set")] + #[cfg_attr(not(test), rustc_diagnostic_item = "btreemap_insert")] pub fn insert(&mut self, key: K, value: V) -> Option where K: Ord, diff --git a/library/alloc/src/collections/btree/map/entry.rs b/library/alloc/src/collections/btree/map/entry.rs index d128ad8ee5f6d..75bb86916a887 100644 --- a/library/alloc/src/collections/btree/map/entry.rs +++ b/library/alloc/src/collections/btree/map/entry.rs @@ -5,7 +5,7 @@ use core::mem; use Entry::*; use super::super::borrow::DormantMutRef; -use super::super::node::{marker, Handle, NodeRef}; +use super::super::node::{Handle, NodeRef, marker}; use super::BTreeMap; use crate::alloc::{Allocator, Global}; diff --git a/library/alloc/src/collections/btree/map/tests.rs b/library/alloc/src/collections/btree/map/tests.rs index ff1254a5a0c42..db16d82be7dcc 100644 --- a/library/alloc/src/collections/btree/map/tests.rs +++ b/library/alloc/src/collections/btree/map/tests.rs @@ -1,7 +1,7 @@ use core::assert_matches::assert_matches; use std::iter; use std::ops::Bound::{Excluded, Included, Unbounded}; -use std::panic::{catch_unwind, AssertUnwindSafe}; +use std::panic::{AssertUnwindSafe, catch_unwind}; use std::sync::atomic::AtomicUsize; use std::sync::atomic::Ordering::SeqCst; @@ -1216,7 +1216,7 @@ mod test_extract_if { { let mut it = map.extract_if(|dummy, _| dummy.query(true)); catch_unwind(AssertUnwindSafe(|| while it.next().is_some() {})).unwrap_err(); - // Iterator behaviour after a panic is explicitly unspecified, + // Iterator behavior after a panic is explicitly unspecified, // so this is just the current implementation: let result = catch_unwind(AssertUnwindSafe(|| it.next())); assert!(matches!(result, Ok(None))); diff --git a/library/alloc/src/collections/btree/navigate.rs b/library/alloc/src/collections/btree/navigate.rs index f5c621e2c1759..14b7d4ad71f86 100644 --- a/library/alloc/src/collections/btree/navigate.rs +++ b/library/alloc/src/collections/btree/navigate.rs @@ -3,7 +3,7 @@ use core::ops::RangeBounds; use core::{hint, ptr}; use super::node::ForceResult::*; -use super::node::{marker, Handle, NodeRef}; +use super::node::{Handle, NodeRef, marker}; use super::search::SearchBound; use crate::alloc::Allocator; // `front` and `back` are always both `None` or both `Some`. diff --git a/library/alloc/src/collections/btree/node.rs b/library/alloc/src/collections/btree/node.rs index 78ccb3af66dbb..2a853ef421629 100644 --- a/library/alloc/src/collections/btree/node.rs +++ b/library/alloc/src/collections/btree/node.rs @@ -72,8 +72,8 @@ impl LeafNode { // be both slightly faster and easier to track in Valgrind. unsafe { // parent_idx, keys, and vals are all MaybeUninit - ptr::addr_of_mut!((*this).parent).write(None); - ptr::addr_of_mut!((*this).len).write(0); + (&raw mut (*this).parent).write(None); + (&raw mut (*this).len).write(0); } } @@ -114,7 +114,7 @@ impl InternalNode { unsafe { let mut node = Box::::new_uninit_in(alloc); // We only need to initialize the data; the edges are MaybeUninit. - LeafNode::init(ptr::addr_of_mut!((*node.as_mut_ptr()).data)); + LeafNode::init(&raw mut (*node.as_mut_ptr()).data); node.assume_init() } } @@ -525,8 +525,8 @@ impl<'a, K, V, Type> NodeRef, K, V, Type> { // to avoid aliasing with outstanding references to other elements, // in particular, those returned to the caller in earlier iterations. let leaf = Self::as_leaf_ptr(&mut self); - let keys = unsafe { ptr::addr_of!((*leaf).keys) }; - let vals = unsafe { ptr::addr_of_mut!((*leaf).vals) }; + let keys = unsafe { &raw const (*leaf).keys }; + let vals = unsafe { &raw mut (*leaf).vals }; // We must coerce to unsized array pointers because of Rust issue #74679. let keys: *const [_] = keys; let vals: *mut [_] = vals; @@ -1521,7 +1521,7 @@ impl<'a, K: 'a, V: 'a> BalancingContext<'a, K, V> { right_node.val_area_mut(..count - 1), ); - // Move the left-most stolen pair to the parent. + // Move the leftmost stolen pair to the parent. let k = left_node.key_area_mut(new_left_len).assume_init_read(); let v = left_node.val_area_mut(new_left_len).assume_init_read(); let (k, v) = self.parent.replace_kv(k, v); @@ -1570,7 +1570,7 @@ impl<'a, K: 'a, V: 'a> BalancingContext<'a, K, V> { // Move leaf data. { - // Move the right-most stolen pair to the parent. + // Move the rightmost stolen pair to the parent. let k = right_node.key_area_mut(count - 1).assume_init_read(); let v = right_node.val_area_mut(count - 1).assume_init_read(); let (k, v) = self.parent.replace_kv(k, v); diff --git a/library/alloc/src/collections/btree/remove.rs b/library/alloc/src/collections/btree/remove.rs index c46422c2f1d45..56f2824b782bd 100644 --- a/library/alloc/src/collections/btree/remove.rs +++ b/library/alloc/src/collections/btree/remove.rs @@ -3,7 +3,7 @@ use core::alloc::Allocator; use super::map::MIN_LEN; use super::node::ForceResult::*; use super::node::LeftOrRight::*; -use super::node::{marker, Handle, NodeRef}; +use super::node::{Handle, NodeRef, marker}; impl<'a, K: 'a, V: 'a> Handle, K, V, marker::LeafOrInternal>, marker::KV> { /// Removes a key-value pair from the tree, and returns that pair, as well as diff --git a/library/alloc/src/collections/btree/search.rs b/library/alloc/src/collections/btree/search.rs index 1d5c927175ea6..22e015edac3d2 100644 --- a/library/alloc/src/collections/btree/search.rs +++ b/library/alloc/src/collections/btree/search.rs @@ -6,7 +6,7 @@ use SearchBound::*; use SearchResult::*; use super::node::ForceResult::*; -use super::node::{marker, Handle, NodeRef}; +use super::node::{Handle, NodeRef, marker}; pub enum SearchBound { /// An inclusive bound to look for, just like `Bound::Included(T)`. diff --git a/library/alloc/src/collections/btree/set.rs b/library/alloc/src/collections/btree/set.rs index 2b5bebcd8cd08..a40209fa2e397 100644 --- a/library/alloc/src/collections/btree/set.rs +++ b/library/alloc/src/collections/btree/set.rs @@ -7,10 +7,10 @@ use core::iter::{FusedIterator, Peekable}; use core::mem::ManuallyDrop; use core::ops::{BitAnd, BitOr, BitXor, Bound, RangeBounds, Sub}; +use super::Recover; use super::map::{BTreeMap, Keys}; use super::merge_iter::MergeIterInner; use super::set_val::SetValZST; -use super::Recover; use crate::alloc::{Allocator, Global}; use crate::vec::Vec; @@ -1132,6 +1132,7 @@ impl BTreeSet { /// assert_eq!(set_iter.next(), None); /// ``` #[stable(feature = "rust1", since = "1.0.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "btreeset_iter")] pub fn iter(&self) -> Iter<'_, T> { Iter { iter: self.map.keys() } } diff --git a/library/alloc/src/collections/btree/set/tests.rs b/library/alloc/src/collections/btree/set/tests.rs index f947b6108c9a7..990044e069f64 100644 --- a/library/alloc/src/collections/btree/set/tests.rs +++ b/library/alloc/src/collections/btree/set/tests.rs @@ -1,5 +1,5 @@ use std::ops::Bound::{Excluded, Included}; -use std::panic::{catch_unwind, AssertUnwindSafe}; +use std::panic::{AssertUnwindSafe, catch_unwind}; use super::*; use crate::testing::crash_test::{CrashTestDummy, Panic}; @@ -132,11 +132,9 @@ fn test_difference() { check_difference(&[1, 3, 5, 9, 11], &[3, 6, 9], &[1, 5, 11]); check_difference(&[1, 3, 5, 9, 11], &[0, 1], &[3, 5, 9, 11]); check_difference(&[1, 3, 5, 9, 11], &[11, 12], &[1, 3, 5, 9]); - check_difference( - &[-5, 11, 22, 33, 40, 42], - &[-12, -5, 14, 23, 34, 38, 39, 50], - &[11, 22, 33, 40, 42], - ); + check_difference(&[-5, 11, 22, 33, 40, 42], &[-12, -5, 14, 23, 34, 38, 39, 50], &[ + 11, 22, 33, 40, 42, + ]); if cfg!(miri) { // Miri is too slow @@ -252,11 +250,9 @@ fn test_union() { check_union(&[], &[], &[]); check_union(&[1, 2, 3], &[2], &[1, 2, 3]); check_union(&[2], &[1, 2, 3], &[1, 2, 3]); - check_union( - &[1, 3, 5, 9, 11, 16, 19, 24], - &[-2, 1, 5, 9, 13, 19], - &[-2, 1, 3, 5, 9, 11, 13, 16, 19, 24], - ); + check_union(&[1, 3, 5, 9, 11, 16, 19, 24], &[-2, 1, 5, 9, 13, 19], &[ + -2, 1, 3, 5, 9, 11, 13, 16, 19, 24, + ]); } #[test] diff --git a/library/alloc/src/collections/linked_list.rs b/library/alloc/src/collections/linked_list.rs index 0cd410c0fb7c1..ca0ea1ec8b2ba 100644 --- a/library/alloc/src/collections/linked_list.rs +++ b/library/alloc/src/collections/linked_list.rs @@ -1082,7 +1082,7 @@ impl LinkedList { /// Retains only the elements specified by the predicate. /// - /// In other words, remove all elements `e` for which `f(&e)` returns false. + /// In other words, remove all elements `e` for which `f(&mut e)` returns false. /// This method operates in place, visiting each element exactly once in the /// original order, and preserves the order of the retained elements. /// diff --git a/library/alloc/src/collections/linked_list/tests.rs b/library/alloc/src/collections/linked_list/tests.rs index 9b3c9ac5ce52e..b7d4f8512a0f2 100644 --- a/library/alloc/src/collections/linked_list/tests.rs +++ b/library/alloc/src/collections/linked_list/tests.rs @@ -1,4 +1,7 @@ -use std::panic::{catch_unwind, AssertUnwindSafe}; +// FIXME(static_mut_refs): Do not allow `static_mut_refs` lint +#![allow(static_mut_refs)] + +use std::panic::{AssertUnwindSafe, catch_unwind}; use std::thread; use rand::RngCore; @@ -693,10 +696,9 @@ fn test_cursor_mut_insert() { cursor.splice_after(p); cursor.splice_before(q); check_links(&m); - assert_eq!( - m.iter().cloned().collect::>(), - &[200, 201, 202, 203, 1, 100, 101, 102, 103, 8, 2, 3, 4, 5, 6] - ); + assert_eq!(m.iter().cloned().collect::>(), &[ + 200, 201, 202, 203, 1, 100, 101, 102, 103, 8, 2, 3, 4, 5, 6 + ]); let mut cursor = m.cursor_front_mut(); cursor.move_prev(); let tmp = cursor.split_before(); @@ -913,10 +915,9 @@ fn extract_if_complex() { assert_eq!(removed, vec![2, 4, 6, 18, 20, 22, 24, 26, 34, 36]); assert_eq!(list.len(), 14); - assert_eq!( - list.into_iter().collect::>(), - vec![1, 7, 9, 11, 13, 15, 17, 27, 29, 31, 33, 35, 37, 39] - ); + assert_eq!(list.into_iter().collect::>(), vec![ + 1, 7, 9, 11, 13, 15, 17, 27, 29, 31, 33, 35, 37, 39 + ]); } { @@ -931,10 +932,9 @@ fn extract_if_complex() { assert_eq!(removed, vec![2, 4, 6, 18, 20, 22, 24, 26, 34, 36]); assert_eq!(list.len(), 13); - assert_eq!( - list.into_iter().collect::>(), - vec![7, 9, 11, 13, 15, 17, 27, 29, 31, 33, 35, 37, 39] - ); + assert_eq!(list.into_iter().collect::>(), vec![ + 7, 9, 11, 13, 15, 17, 27, 29, 31, 33, 35, 37, 39 + ]); } { @@ -949,10 +949,9 @@ fn extract_if_complex() { assert_eq!(removed, vec![2, 4, 6, 18, 20, 22, 24, 26, 34, 36]); assert_eq!(list.len(), 11); - assert_eq!( - list.into_iter().collect::>(), - vec![7, 9, 11, 13, 15, 17, 27, 29, 31, 33, 35] - ); + assert_eq!(list.into_iter().collect::>(), vec![ + 7, 9, 11, 13, 15, 17, 27, 29, 31, 33, 35 + ]); } { diff --git a/library/alloc/src/collections/vec_deque/iter.rs b/library/alloc/src/collections/vec_deque/iter.rs index bf4dd66f47638..6922ea9b79bfd 100644 --- a/library/alloc/src/collections/vec_deque/iter.rs +++ b/library/alloc/src/collections/vec_deque/iter.rs @@ -73,7 +73,7 @@ impl<'a, T> Iterator for Iter<'a, T> { fn advance_by(&mut self, n: usize) -> Result<(), NonZero> { let remaining = self.i1.advance_by(n); match remaining { - Ok(()) => return Ok(()), + Ok(()) => Ok(()), Err(n) => { mem::swap(&mut self.i1, &mut self.i2); self.i1.advance_by(n.get()) @@ -144,7 +144,7 @@ impl<'a, T> DoubleEndedIterator for Iter<'a, T> { fn advance_back_by(&mut self, n: usize) -> Result<(), NonZero> { match self.i2.advance_back_by(n) { - Ok(()) => return Ok(()), + Ok(()) => Ok(()), Err(n) => { mem::swap(&mut self.i1, &mut self.i2); self.i2.advance_back_by(n.get()) diff --git a/library/alloc/src/collections/vec_deque/iter_mut.rs b/library/alloc/src/collections/vec_deque/iter_mut.rs index 7a349a1b4edd0..84b7410958013 100644 --- a/library/alloc/src/collections/vec_deque/iter_mut.rs +++ b/library/alloc/src/collections/vec_deque/iter_mut.rs @@ -64,7 +64,7 @@ impl<'a, T> Iterator for IterMut<'a, T> { fn advance_by(&mut self, n: usize) -> Result<(), NonZero> { match self.i1.advance_by(n) { - Ok(()) => return Ok(()), + Ok(()) => Ok(()), Err(remaining) => { mem::swap(&mut self.i1, &mut self.i2); self.i1.advance_by(remaining.get()) @@ -135,7 +135,7 @@ impl<'a, T> DoubleEndedIterator for IterMut<'a, T> { fn advance_back_by(&mut self, n: usize) -> Result<(), NonZero> { match self.i2.advance_back_by(n) { - Ok(()) => return Ok(()), + Ok(()) => Ok(()), Err(remaining) => { mem::swap(&mut self.i1, &mut self.i2); self.i2.advance_back_by(remaining.get()) diff --git a/library/alloc/src/collections/vec_deque/mod.rs b/library/alloc/src/collections/vec_deque/mod.rs index dc725ec0f56e4..cf51a84bb6f24 100644 --- a/library/alloc/src/collections/vec_deque/mod.rs +++ b/library/alloc/src/collections/vec_deque/mod.rs @@ -9,7 +9,7 @@ use core::cmp::{self, Ordering}; use core::hash::{Hash, Hasher}; -use core::iter::{repeat_n, repeat_with, ByRefSized}; +use core::iter::{ByRefSized, repeat_n, repeat_with}; // This is used in a bunch of intra-doc links. // FIXME: For some reason, `#[cfg(doc)]` wasn't sufficient, resulting in // failures in linkchecker even though rustdoc built the docs just fine. @@ -103,6 +103,7 @@ pub struct VecDeque< #[stable(feature = "rust1", since = "1.0.0")] impl Clone for VecDeque { + #[track_caller] fn clone(&self) -> Self { let mut deq = Self::with_capacity_in(self.len(), self.allocator().clone()); deq.extend(self.iter().cloned()); @@ -113,6 +114,7 @@ impl Clone for VecDeque { /// /// This method is preferred over simply assigning `source.clone()` to `self`, /// as it avoids reallocation if possible. + #[track_caller] fn clone_from(&mut self, source: &Self) { self.clear(); self.extend(source.iter().cloned()); @@ -554,8 +556,8 @@ impl VecDeque { #[rustc_const_stable(feature = "const_vec_deque_new", since = "1.68.0")] #[must_use] pub const fn new() -> VecDeque { - // FIXME: This should just be `VecDeque::new_in(Global)` once that hits stable. - VecDeque { head: 0, len: 0, buf: RawVec::NEW } + // FIXME(const-hack): This should just be `VecDeque::new_in(Global)` once that hits stable. + VecDeque { head: 0, len: 0, buf: RawVec::new() } } /// Creates an empty deque with space for at least `capacity` elements. @@ -570,6 +572,7 @@ impl VecDeque { #[inline] #[stable(feature = "rust1", since = "1.0.0")] #[must_use] + #[track_caller] pub fn with_capacity(capacity: usize) -> VecDeque { Self::with_capacity_in(capacity, Global) } @@ -625,6 +628,7 @@ impl VecDeque { /// let deque: VecDeque = VecDeque::with_capacity(10); /// ``` #[unstable(feature = "allocator_api", issue = "32838")] + #[track_caller] pub fn with_capacity_in(capacity: usize, alloc: A) -> VecDeque { VecDeque { head: 0, len: 0, buf: RawVec::with_capacity_in(capacity, alloc) } } @@ -789,6 +793,7 @@ impl VecDeque { /// /// [`reserve`]: VecDeque::reserve #[stable(feature = "rust1", since = "1.0.0")] + #[track_caller] pub fn reserve_exact(&mut self, additional: usize) { let new_cap = self.len.checked_add(additional).expect("capacity overflow"); let old_cap = self.capacity(); @@ -818,6 +823,7 @@ impl VecDeque { /// assert!(buf.capacity() >= 11); /// ``` #[stable(feature = "rust1", since = "1.0.0")] + #[track_caller] pub fn reserve(&mut self, additional: usize) { let new_cap = self.len.checked_add(additional).expect("capacity overflow"); let old_cap = self.capacity(); @@ -949,6 +955,7 @@ impl VecDeque { /// assert!(buf.capacity() >= 4); /// ``` #[stable(feature = "deque_extras_15", since = "1.5.0")] + #[track_caller] pub fn shrink_to_fit(&mut self) { self.shrink_to(0); } @@ -974,6 +981,7 @@ impl VecDeque { /// assert!(buf.capacity() >= 4); /// ``` #[stable(feature = "shrink_to", since = "1.56.0")] + #[track_caller] pub fn shrink_to(&mut self, min_capacity: usize) { let target_cap = min_capacity.max(self.len); @@ -1201,6 +1209,7 @@ impl VecDeque { /// assert_eq!(&c[..], b); /// ``` #[stable(feature = "rust1", since = "1.0.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "vecdeque_iter")] pub fn iter(&self) -> Iter<'_, T> { let (a, b) = self.as_slices(); Iter::new(a.iter(), b.iter()) @@ -1739,6 +1748,7 @@ impl VecDeque { /// assert_eq!(d.front(), Some(&2)); /// ``` #[stable(feature = "rust1", since = "1.0.0")] + #[track_caller] pub fn push_front(&mut self, value: T) { if self.is_full() { self.grow(); @@ -1766,6 +1776,7 @@ impl VecDeque { /// ``` #[stable(feature = "rust1", since = "1.0.0")] #[rustc_confusables("push", "put", "append")] + #[track_caller] pub fn push_back(&mut self, value: T) { if self.is_full() { self.grow(); @@ -1875,6 +1886,7 @@ impl VecDeque { /// assert_eq!(vec_deque, &['a', 'd', 'b', 'c']); /// ``` #[stable(feature = "deque_extras_15", since = "1.5.0")] + #[track_caller] pub fn insert(&mut self, index: usize, value: T) { assert!(index <= self.len(), "index out of bounds"); if self.is_full() { @@ -1978,6 +1990,7 @@ impl VecDeque { #[inline] #[must_use = "use `.truncate()` if you don't need the other half"] #[stable(feature = "split_off", since = "1.4.0")] + #[track_caller] pub fn split_off(&mut self, at: usize) -> Self where A: Clone, @@ -2044,6 +2057,7 @@ impl VecDeque { /// ``` #[inline] #[stable(feature = "append", since = "1.4.0")] + #[track_caller] pub fn append(&mut self, other: &mut Self) { if T::IS_ZST { self.len = self.len.checked_add(other.len).expect("capacity overflow"); @@ -2108,7 +2122,7 @@ impl VecDeque { /// Retains only the elements specified by the predicate. /// - /// In other words, remove all elements `e` for which `f(&e)` returns false. + /// In other words, remove all elements `e` for which `f(&mut e)` returns false. /// This method operates in place, visiting each element exactly once in the /// original order, and preserves the order of the retained elements. /// @@ -2166,6 +2180,7 @@ impl VecDeque { // be called in cold paths. // This may panic or abort #[inline(never)] + #[track_caller] fn grow(&mut self) { // Extend or possibly remove this assertion when valid use-cases for growing the // buffer without it being full emerge @@ -2204,6 +2219,7 @@ impl VecDeque { /// assert_eq!(buf, [5, 10, 101, 102, 103]); /// ``` #[stable(feature = "vec_resize_with", since = "1.33.0")] + #[track_caller] pub fn resize_with(&mut self, new_len: usize, generator: impl FnMut() -> T) { let len = self.len; @@ -2750,6 +2766,7 @@ impl VecDeque { /// assert_eq!(buf, [5, 10, 20, 20, 20]); /// ``` #[stable(feature = "deque_extras", since = "1.16.0")] + #[track_caller] pub fn resize(&mut self, new_len: usize, value: T) { if new_len > self.len() { let extra = new_len - self.len(); @@ -2869,6 +2886,7 @@ impl IndexMut for VecDeque { #[stable(feature = "rust1", since = "1.0.0")] impl FromIterator for VecDeque { + #[track_caller] fn from_iter>(iter: I) -> VecDeque { SpecFromIter::spec_from_iter(iter.into_iter()) } @@ -2908,16 +2926,19 @@ impl<'a, T, A: Allocator> IntoIterator for &'a mut VecDeque { #[stable(feature = "rust1", since = "1.0.0")] impl Extend for VecDeque { + #[track_caller] fn extend>(&mut self, iter: I) { >::spec_extend(self, iter.into_iter()); } #[inline] + #[track_caller] fn extend_one(&mut self, elem: T) { self.push_back(elem); } #[inline] + #[track_caller] fn extend_reserve(&mut self, additional: usize) { self.reserve(additional); } @@ -2933,16 +2954,19 @@ impl Extend for VecDeque { #[stable(feature = "extend_ref", since = "1.2.0")] impl<'a, T: 'a + Copy, A: Allocator> Extend<&'a T> for VecDeque { + #[track_caller] fn extend>(&mut self, iter: I) { self.spec_extend(iter.into_iter()); } #[inline] + #[track_caller] fn extend_one(&mut self, &elem: &'a T) { self.push_back(elem); } #[inline] + #[track_caller] fn extend_reserve(&mut self, additional: usize) { self.reserve(additional); } @@ -3040,6 +3064,7 @@ impl From<[T; N]> for VecDeque { /// let deq2: VecDeque<_> = [1, 2, 3, 4].into(); /// assert_eq!(deq1, deq2); /// ``` + #[track_caller] fn from(arr: [T; N]) -> Self { let mut deq = VecDeque::with_capacity(N); let arr = ManuallyDrop::new(arr); diff --git a/library/alloc/src/collections/vec_deque/spec_extend.rs b/library/alloc/src/collections/vec_deque/spec_extend.rs index a9b0fd073b548..d246385ca8419 100644 --- a/library/alloc/src/collections/vec_deque/spec_extend.rs +++ b/library/alloc/src/collections/vec_deque/spec_extend.rs @@ -7,6 +7,7 @@ use crate::vec; // Specialization trait used for VecDeque::extend pub(super) trait SpecExtend { + #[track_caller] fn spec_extend(&mut self, iter: I); } @@ -14,6 +15,7 @@ impl SpecExtend for VecDeque where I: Iterator, { + #[track_caller] default fn spec_extend(&mut self, mut iter: I) { // This function should be the moral equivalent of: // @@ -44,6 +46,7 @@ impl SpecExtend for VecDeque where I: TrustedLen, { + #[track_caller] default fn spec_extend(&mut self, iter: I) { // This is the case for a TrustedLen iterator. let (low, high) = iter.size_hint(); @@ -76,6 +79,7 @@ where } impl SpecExtend> for VecDeque { + #[track_caller] fn spec_extend(&mut self, mut iterator: vec::IntoIter) { let slice = iterator.as_slice(); self.reserve(slice.len()); @@ -93,6 +97,7 @@ where I: Iterator, T: Copy, { + #[track_caller] default fn spec_extend(&mut self, iterator: I) { self.spec_extend(iterator.copied()) } @@ -102,6 +107,7 @@ impl<'a, T: 'a, A: Allocator> SpecExtend<&'a T, slice::Iter<'a, T>> for VecDeque where T: Copy, { + #[track_caller] fn spec_extend(&mut self, iterator: slice::Iter<'a, T>) { let slice = iterator.as_slice(); self.reserve(slice.len()); diff --git a/library/alloc/src/collections/vec_deque/spec_from_iter.rs b/library/alloc/src/collections/vec_deque/spec_from_iter.rs index 2708c7fe10259..1efe84d6d7d7d 100644 --- a/library/alloc/src/collections/vec_deque/spec_from_iter.rs +++ b/library/alloc/src/collections/vec_deque/spec_from_iter.rs @@ -9,6 +9,7 @@ impl SpecFromIter for VecDeque where I: Iterator, { + #[track_caller] default fn spec_from_iter(iterator: I) -> Self { // Since converting is O(1) now, just re-use the `Vec` logic for // anything where we can't do something extra-special for `VecDeque`, diff --git a/library/alloc/src/collections/vec_deque/tests.rs b/library/alloc/src/collections/vec_deque/tests.rs index f8ce4ca97884e..6328b3b4db867 100644 --- a/library/alloc/src/collections/vec_deque/tests.rs +++ b/library/alloc/src/collections/vec_deque/tests.rs @@ -1,3 +1,6 @@ +// FIXME(static_mut_refs): Do not allow `static_mut_refs` lint +#![allow(static_mut_refs)] + use core::iter::TrustedLen; use super::*; @@ -559,10 +562,9 @@ fn make_contiguous_head_to_end() { tester.push_front(i as char); } - assert_eq!( - tester, - ['P', 'O', 'N', 'M', 'L', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K'] - ); + assert_eq!(tester, [ + 'P', 'O', 'N', 'M', 'L', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K' + ]); // ABCDEFGHIJKPONML let expected_start = 0; diff --git a/library/alloc/src/ffi/c_str.rs b/library/alloc/src/ffi/c_str.rs index e32676a65432b..d7e99f4a1a638 100644 --- a/library/alloc/src/ffi/c_str.rs +++ b/library/alloc/src/ffi/c_str.rs @@ -4,10 +4,10 @@ mod tests; use core::borrow::Borrow; -use core::ffi::{c_char, CStr}; +use core::ffi::{CStr, c_char}; use core::num::NonZero; use core::slice::memchr; -use core::str::{self, Utf8Error}; +use core::str::{self, FromStr, Utf8Error}; use core::{fmt, mem, ops, ptr, slice}; use crate::borrow::{Cow, ToOwned}; @@ -576,6 +576,7 @@ impl CString { #[inline] #[must_use] #[stable(feature = "as_c_str", since = "1.20.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "cstring_as_c_str")] pub fn as_c_str(&self) -> &CStr { &*self } @@ -695,6 +696,7 @@ impl CString { // memory-unsafe code from working by accident. Inline // to prevent LLVM from optimizing it away in debug builds. #[stable(feature = "cstring_drop", since = "1.13.0")] +#[rustc_insignificant_dtor] impl Drop for CString { #[inline] fn drop(&mut self) { @@ -815,6 +817,30 @@ impl From>> for CString { } } +impl FromStr for CString { + type Err = NulError; + + /// Converts a string `s` into a [`CString`]. + /// + /// This method is equivalent to [`CString::new`]. + #[inline] + fn from_str(s: &str) -> Result { + Self::new(s) + } +} + +impl TryFrom for String { + type Error = IntoStringError; + + /// Converts a [`CString`] into a [`String`] if it contains valid UTF-8 data. + /// + /// This method is equivalent to [`CString::into_string`]. + #[inline] + fn try_from(value: CString) -> Result { + value.into_string() + } +} + #[cfg(not(test))] #[stable(feature = "more_box_slice_clone", since = "1.29.0")] impl Clone for Box { diff --git a/library/alloc/src/fmt.rs b/library/alloc/src/fmt.rs index 571fcd177aae7..3da71038a5e76 100644 --- a/library/alloc/src/fmt.rs +++ b/library/alloc/src/fmt.rs @@ -580,10 +580,8 @@ pub use core::fmt::Alignment; #[stable(feature = "rust1", since = "1.0.0")] pub use core::fmt::Error; -#[unstable(feature = "debug_closure_helpers", issue = "117729")] -pub use core::fmt::{from_fn, FromFn}; #[stable(feature = "rust1", since = "1.0.0")] -pub use core::fmt::{write, Arguments}; +pub use core::fmt::{Arguments, write}; #[stable(feature = "rust1", since = "1.0.0")] pub use core::fmt::{Binary, Octal}; #[stable(feature = "rust1", since = "1.0.0")] @@ -592,6 +590,8 @@ pub use core::fmt::{Debug, Display}; pub use core::fmt::{DebugList, DebugMap, DebugSet, DebugStruct, DebugTuple}; #[stable(feature = "rust1", since = "1.0.0")] pub use core::fmt::{Formatter, Result, Write}; +#[unstable(feature = "debug_closure_helpers", issue = "117729")] +pub use core::fmt::{FromFn, from_fn}; #[stable(feature = "rust1", since = "1.0.0")] pub use core::fmt::{LowerExp, UpperExp}; #[stable(feature = "rust1", since = "1.0.0")] diff --git a/library/alloc/src/lib.rs b/library/alloc/src/lib.rs index d28c5a29df2b9..67ac172742c95 100644 --- a/library/alloc/src/lib.rs +++ b/library/alloc/src/lib.rs @@ -94,6 +94,7 @@ #![cfg_attr(kani, feature(kani))] #![cfg_attr(not(no_global_oom_handling), feature(const_alloc_error))] #![cfg_attr(not(no_global_oom_handling), feature(const_btree_len))] +#![cfg_attr(test, feature(str_as_str))] #![feature(alloc_layout_extra)] #![feature(allocator_api)] #![feature(array_chunks)] @@ -104,19 +105,16 @@ #![feature(async_closure)] #![feature(async_fn_traits)] #![feature(async_iterator)] +#![feature(box_uninit_write)] #![feature(clone_to_uninit)] #![feature(coerce_unsized)] #![feature(const_align_of_val)] #![feature(const_box)] -#![feature(const_cow_is_borrowed)] #![feature(const_eval_select)] #![feature(const_heap)] -#![feature(const_maybe_uninit_as_mut_ptr)] #![feature(const_maybe_uninit_write)] -#![feature(const_option)] -#![feature(const_pin)] -#![feature(const_refs_to_cell)] #![feature(const_size_of_val)] +#![feature(const_vec_string_slice)] #![feature(core_intrinsics)] #![feature(deprecated_suggestion)] #![feature(deref_pure_trait)] @@ -132,6 +130,7 @@ #![feature(iter_advance_by)] #![feature(iter_next_chunk)] #![feature(layout_for_ptr)] +#![feature(legacy_receiver_trait)] #![feature(local_waker)] #![feature(maybe_uninit_slice)] #![feature(maybe_uninit_uninit_array_transpose)] @@ -141,7 +140,6 @@ #![feature(ptr_internals)] #![feature(ptr_metadata)] #![feature(ptr_sub_ptr)] -#![feature(receiver_trait)] #![feature(set_ptr_value)] #![feature(sized_type_properties)] #![feature(slice_from_ptr_range)] @@ -150,7 +148,6 @@ #![feature(slice_range)] #![feature(std_internals)] #![feature(str_internals)] -#![feature(strict_provenance)] #![feature(trusted_fused)] #![feature(trusted_len)] #![feature(trusted_random_access)] @@ -165,15 +162,15 @@ // // Language features: // tidy-alphabetical-start +#![cfg_attr(bootstrap, feature(strict_provenance))] +#![cfg_attr(not(bootstrap), feature(strict_provenance_lints))] #![cfg_attr(not(test), feature(coroutine_trait))] #![cfg_attr(test, feature(panic_update_hook))] #![cfg_attr(test, feature(test))] #![feature(allocator_internals)] #![feature(allow_internal_unstable)] #![feature(cfg_sanitize)] -#![feature(const_mut_refs)] #![feature(const_precise_live_drops)] -#![feature(const_ptr_write)] #![feature(const_try)] #![feature(decl_macro)] #![feature(dropck_eyepatch)] diff --git a/library/alloc/src/raw_vec.rs b/library/alloc/src/raw_vec.rs index a651ba067e47c..85a9120c7e255 100644 --- a/library/alloc/src/raw_vec.rs +++ b/library/alloc/src/raw_vec.rs @@ -20,6 +20,7 @@ mod tests; // only one location which panics rather than a bunch throughout the module. #[cfg(not(no_global_oom_handling))] #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))] +#[track_caller] fn capacity_overflow() -> ! { panic!("capacity overflow"); } @@ -96,22 +97,15 @@ struct RawVecInner { } impl RawVec { - /// HACK(Centril): This exists because stable `const fn` can only call stable `const fn`, so - /// they cannot call `Self::new()`. - /// - /// If you change `RawVec::new` or dependencies, please take care to not introduce anything - /// that would truly const-call something unstable. - pub const NEW: Self = Self::new(); - /// Creates the biggest possible `RawVec` (on the system heap) /// without allocating. If `T` has positive size, then this makes a /// `RawVec` with capacity `0`. If `T` is zero-sized, then it makes a /// `RawVec` with capacity `usize::MAX`. Useful for implementing /// delayed allocation. #[must_use] - #[rustc_const_stable(feature = "raw_vec_internals_const", since = "1.81")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "raw_vec_internals_const", since = "1.81"))] pub const fn new() -> Self { - Self { inner: RawVecInner::new::(), _marker: PhantomData } + Self::new_in(Global) } /// Creates a `RawVec` (on the system heap) with exactly the @@ -132,6 +126,7 @@ impl RawVec { #[cfg(not(any(no_global_oom_handling, test)))] #[must_use] #[inline] + #[track_caller] pub fn with_capacity(capacity: usize) -> Self { Self { inner: RawVecInner::with_capacity(capacity, T::LAYOUT), _marker: PhantomData } } @@ -140,6 +135,7 @@ impl RawVec { #[cfg(not(any(no_global_oom_handling, test)))] #[must_use] #[inline] + #[track_caller] pub fn with_capacity_zeroed(capacity: usize) -> Self { Self { inner: RawVecInner::with_capacity_zeroed_in(capacity, Global, T::LAYOUT), @@ -149,15 +145,10 @@ impl RawVec { } impl RawVecInner { - #[must_use] - #[rustc_const_stable(feature = "raw_vec_internals_const", since = "1.81")] - const fn new() -> Self { - Self::new_in(Global, core::mem::align_of::()) - } - #[cfg(not(any(no_global_oom_handling, test)))] #[must_use] #[inline] + #[track_caller] fn with_capacity(capacity: usize, elem_layout: Layout) -> Self { match Self::try_allocate_in(capacity, AllocInit::Uninitialized, Global, elem_layout) { Ok(res) => res, @@ -188,7 +179,7 @@ impl RawVec { /// Like `new`, but parameterized over the choice of allocator for /// the returned `RawVec`. #[inline] - #[rustc_const_stable(feature = "raw_vec_internals_const", since = "1.81")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "raw_vec_internals_const", since = "1.81"))] pub const fn new_in(alloc: A) -> Self { Self { inner: RawVecInner::new_in(alloc, align_of::()), _marker: PhantomData } } @@ -197,6 +188,7 @@ impl RawVec { /// allocator for the returned `RawVec`. #[cfg(not(no_global_oom_handling))] #[inline] + #[track_caller] pub fn with_capacity_in(capacity: usize, alloc: A) -> Self { Self { inner: RawVecInner::with_capacity_in(capacity, alloc, T::LAYOUT), @@ -218,6 +210,7 @@ impl RawVec { /// of allocator for the returned `RawVec`. #[cfg(not(no_global_oom_handling))] #[inline] + #[track_caller] pub fn with_capacity_zeroed_in(capacity: usize, alloc: A) -> Self { Self { inner: RawVecInner::with_capacity_zeroed_in(capacity, alloc, T::LAYOUT), @@ -293,7 +286,7 @@ impl RawVec { /// `Unique::dangling()` if `capacity == 0` or `T` is zero-sized. In the former case, you must /// be careful. #[inline] - pub fn ptr(&self) -> *mut T { + pub const fn ptr(&self) -> *mut T { self.inner.ptr() } @@ -306,7 +299,7 @@ impl RawVec { /// /// This will always be `usize::MAX` if `T` is zero-sized. #[inline] - pub fn capacity(&self) -> usize { + pub const fn capacity(&self) -> usize { self.inner.capacity(size_of::()) } @@ -337,6 +330,7 @@ impl RawVec { /// Aborts on OOM. #[cfg(not(no_global_oom_handling))] #[inline] + #[track_caller] pub fn reserve(&mut self, len: usize, additional: usize) { self.inner.reserve(len, additional, T::LAYOUT) } @@ -345,6 +339,7 @@ impl RawVec { /// caller to ensure `len == self.capacity()`. #[cfg(not(no_global_oom_handling))] #[inline(never)] + #[track_caller] pub fn grow_one(&mut self) { self.inner.grow_one(T::LAYOUT) } @@ -372,6 +367,7 @@ impl RawVec { /// /// Aborts on OOM. #[cfg(not(no_global_oom_handling))] + #[track_caller] pub fn reserve_exact(&mut self, len: usize, additional: usize) { self.inner.reserve_exact(len, additional, T::LAYOUT) } @@ -396,6 +392,7 @@ impl RawVec { /// /// Aborts on OOM. #[cfg(not(no_global_oom_handling))] + #[track_caller] #[inline] pub fn shrink_to_fit(&mut self, cap: usize) { self.inner.shrink_to_fit(cap, T::LAYOUT) @@ -412,7 +409,7 @@ unsafe impl<#[may_dangle] T, A: Allocator> Drop for RawVec { impl RawVecInner { #[inline] - #[rustc_const_stable(feature = "raw_vec_internals_const", since = "1.81")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "raw_vec_internals_const", since = "1.81"))] const fn new_in(alloc: A, align: usize) -> Self { let ptr = unsafe { core::mem::transmute(align) }; // `cap: 0` means "unallocated". zero-sized types are ignored. @@ -421,6 +418,7 @@ impl RawVecInner { #[cfg(not(no_global_oom_handling))] #[inline] + #[track_caller] fn with_capacity_in(capacity: usize, alloc: A, elem_layout: Layout) -> Self { match Self::try_allocate_in(capacity, AllocInit::Uninitialized, alloc, elem_layout) { Ok(this) => { @@ -445,6 +443,7 @@ impl RawVecInner { #[cfg(not(no_global_oom_handling))] #[inline] + #[track_caller] fn with_capacity_zeroed_in(capacity: usize, alloc: A, elem_layout: Layout) -> Self { match Self::try_allocate_in(capacity, AllocInit::Zeroed, alloc, elem_layout) { Ok(res) => res, @@ -501,17 +500,17 @@ impl RawVecInner { } #[inline] - fn ptr(&self) -> *mut T { + const fn ptr(&self) -> *mut T { self.non_null::().as_ptr() } #[inline] - fn non_null(&self) -> NonNull { - self.ptr.cast().into() + const fn non_null(&self) -> NonNull { + self.ptr.cast().as_non_null_ptr() } #[inline] - fn capacity(&self, elem_size: usize) -> usize { + const fn capacity(&self, elem_size: usize) -> usize { if elem_size == 0 { usize::MAX } else { self.cap.0 } } @@ -539,6 +538,7 @@ impl RawVecInner { #[cfg(not(no_global_oom_handling))] #[inline] + #[track_caller] fn reserve(&mut self, len: usize, additional: usize, elem_layout: Layout) { // Callers expect this function to be very cheap when there is already sufficient capacity. // Therefore, we move all the resizing and error-handling logic from grow_amortized and @@ -563,6 +563,7 @@ impl RawVecInner { #[cfg(not(no_global_oom_handling))] #[inline] + #[track_caller] fn grow_one(&mut self, elem_layout: Layout) { if let Err(err) = self.grow_amortized(self.cap.0, 1, elem_layout) { handle_error(err); @@ -586,6 +587,7 @@ impl RawVecInner { } #[cfg(not(no_global_oom_handling))] + #[track_caller] fn reserve_exact(&mut self, len: usize, additional: usize, elem_layout: Layout) { if let Err(err) = self.try_reserve_exact(len, additional, elem_layout) { handle_error(err); @@ -610,6 +612,7 @@ impl RawVecInner { #[cfg(not(no_global_oom_handling))] #[inline] + #[track_caller] fn shrink_to_fit(&mut self, cap: usize, elem_layout: Layout) { if let Err(err) = self.shrink(cap, elem_layout) { handle_error(err); @@ -783,6 +786,7 @@ where #[cfg(not(no_global_oom_handling))] #[cold] #[optimize(size)] +#[track_caller] fn handle_error(e: TryReserveError) -> ! { match e.kind() { CapacityOverflow => capacity_overflow(), diff --git a/library/alloc/src/rc.rs b/library/alloc/src/rc.rs index 88c7a12db23ca..fc8646e96d948 100644 --- a/library/alloc/src/rc.rs +++ b/library/alloc/src/rc.rs @@ -251,13 +251,13 @@ use core::intrinsics::abort; #[cfg(not(no_global_oom_handling))] use core::iter; use core::marker::{PhantomData, Unsize}; -use core::mem::{self, align_of_val_raw, ManuallyDrop}; -use core::ops::{CoerceUnsized, Deref, DerefMut, DerefPure, DispatchFromDyn, Receiver}; +use core::mem::{self, ManuallyDrop, align_of_val_raw}; +use core::ops::{CoerceUnsized, Deref, DerefMut, DerefPure, DispatchFromDyn, LegacyReceiver}; use core::panic::{RefUnwindSafe, UnwindSafe}; #[cfg(not(no_global_oom_handling))] use core::pin::Pin; use core::pin::PinCoerceUnsized; -use core::ptr::{self, drop_in_place, NonNull}; +use core::ptr::{self, NonNull, drop_in_place}; #[cfg(not(no_global_oom_handling))] use core::slice::from_raw_parts_mut; use core::{borrow, fmt, hint}; @@ -282,19 +282,19 @@ mod tests; // would interfere with otherwise safe [into|from]_raw() of transmutable // inner types. #[repr(C)] -struct RcBox { +struct RcInner { strong: Cell, weak: Cell, value: T, } -/// Calculate layout for `RcBox` using the inner value's layout -fn rcbox_layout_for_value_layout(layout: Layout) -> Layout { +/// Calculate layout for `RcInner` using the inner value's layout +fn rc_inner_layout_for_value_layout(layout: Layout) -> Layout { // Calculate layout using the given value layout. // Previously, layout was calculated on the expression - // `&*(ptr as *const RcBox)`, but this created a misaligned + // `&*(ptr as *const RcInner)`, but this created a misaligned // reference (see #54908). - Layout::new::>().extend(layout).unwrap().0.pad_to_align() + Layout::new::>().extend(layout).unwrap().0.pad_to_align() } /// A single-threaded reference-counting pointer. 'Rc' stands for 'Reference @@ -314,8 +314,8 @@ pub struct Rc< T: ?Sized, #[unstable(feature = "allocator_api", issue = "32838")] A: Allocator = Global, > { - ptr: NonNull>, - phantom: PhantomData>, + ptr: NonNull>, + phantom: PhantomData>, alloc: A, } @@ -343,39 +343,54 @@ impl, U: ?Sized> DispatchFromDyn> for Rc {} impl Rc { #[inline] - unsafe fn from_inner(ptr: NonNull>) -> Self { + unsafe fn from_inner(ptr: NonNull>) -> Self { unsafe { Self::from_inner_in(ptr, Global) } } #[inline] - unsafe fn from_ptr(ptr: *mut RcBox) -> Self { + unsafe fn from_ptr(ptr: *mut RcInner) -> Self { unsafe { Self::from_inner(NonNull::new_unchecked(ptr)) } } } impl Rc { #[inline(always)] - fn inner(&self) -> &RcBox { + fn inner(&self) -> &RcInner { // This unsafety is ok because while this Rc is alive we're guaranteed // that the inner pointer is valid. unsafe { self.ptr.as_ref() } } #[inline] - fn into_inner_with_allocator(this: Self) -> (NonNull>, A) { + fn into_inner_with_allocator(this: Self) -> (NonNull>, A) { let this = mem::ManuallyDrop::new(this); (this.ptr, unsafe { ptr::read(&this.alloc) }) } #[inline] - unsafe fn from_inner_in(ptr: NonNull>, alloc: A) -> Self { + unsafe fn from_inner_in(ptr: NonNull>, alloc: A) -> Self { Self { ptr, phantom: PhantomData, alloc } } #[inline] - unsafe fn from_ptr_in(ptr: *mut RcBox, alloc: A) -> Self { + unsafe fn from_ptr_in(ptr: *mut RcInner, alloc: A) -> Self { unsafe { Self::from_inner_in(NonNull::new_unchecked(ptr), alloc) } } + + // Non-inlined part of `drop`. + #[inline(never)] + unsafe fn drop_slow(&mut self) { + // Reconstruct the "strong weak" pointer and drop it when this + // variable goes out of scope. This ensures that the memory is + // deallocated even if the destructor of `T` panics. + let _weak = Weak { ptr: self.ptr, alloc: &self.alloc }; + + // Destroy the contained object. + // We cannot use `get_mut_unchecked` here, because `self.alloc` is borrowed. + unsafe { + ptr::drop_in_place(&mut (*self.ptr.as_ptr()).value); + } + } } impl Rc { @@ -397,7 +412,7 @@ impl Rc { // if the weak pointer is stored inside the strong one. unsafe { Self::from_inner( - Box::leak(Box::new(RcBox { strong: Cell::new(1), weak: Cell::new(1), value })) + Box::leak(Box::new(RcInner { strong: Cell::new(1), weak: Cell::new(1), value })) .into(), ) } @@ -460,42 +475,7 @@ impl Rc { where F: FnOnce(&Weak) -> T, { - // Construct the inner in the "uninitialized" state with a single - // weak reference. - let uninit_ptr: NonNull<_> = Box::leak(Box::new(RcBox { - strong: Cell::new(0), - weak: Cell::new(1), - value: mem::MaybeUninit::::uninit(), - })) - .into(); - - let init_ptr: NonNull> = uninit_ptr.cast(); - - let weak = Weak { ptr: init_ptr, alloc: Global }; - - // It's important we don't give up ownership of the weak pointer, or - // else the memory might be freed by the time `data_fn` returns. If - // we really wanted to pass ownership, we could create an additional - // weak pointer for ourselves, but this would result in additional - // updates to the weak reference count which might not be necessary - // otherwise. - let data = data_fn(&weak); - - let strong = unsafe { - let inner = init_ptr.as_ptr(); - ptr::write(ptr::addr_of_mut!((*inner).value), data); - - let prev_value = (*inner).strong.get(); - debug_assert_eq!(prev_value, 0, "No prior strong references should exist"); - (*inner).strong.set(1); - - Rc::from_inner(init_ptr) - }; - - // Strong references should collectively own a shared weak reference, - // so don't run the destructor for our old weak reference. - mem::forget(weak); - strong + Self::new_cyclic_in(data_fn, Global) } /// Constructs a new `Rc` with uninitialized contents. @@ -581,8 +561,12 @@ impl Rc { // if the weak pointer is stored inside the strong one. unsafe { Ok(Self::from_inner( - Box::leak(Box::try_new(RcBox { strong: Cell::new(1), weak: Cell::new(1), value })?) - .into(), + Box::leak(Box::try_new(RcInner { + strong: Cell::new(1), + weak: Cell::new(1), + value, + })?) + .into(), )) } } @@ -681,7 +665,7 @@ impl Rc { // That would make code size bigger. match Self::try_new_in(value, alloc) { Ok(m) => m, - Err(_) => handle_alloc_error(Layout::new::>()), + Err(_) => handle_alloc_error(Layout::new::>()), } } @@ -762,6 +746,84 @@ impl Rc { } } + /// Constructs a new `Rc` in the given allocator while giving you a `Weak` to the allocation, + /// to allow you to construct a `T` which holds a weak pointer to itself. + /// + /// Generally, a structure circularly referencing itself, either directly or + /// indirectly, should not hold a strong reference to itself to prevent a memory leak. + /// Using this function, you get access to the weak pointer during the + /// initialization of `T`, before the `Rc` is created, such that you can + /// clone and store it inside the `T`. + /// + /// `new_cyclic_in` first allocates the managed allocation for the `Rc`, + /// then calls your closure, giving it a `Weak` to this allocation, + /// and only afterwards completes the construction of the `Rc` by placing + /// the `T` returned from your closure into the allocation. + /// + /// Since the new `Rc` is not fully-constructed until `Rc::new_cyclic_in` + /// returns, calling [`upgrade`] on the weak reference inside your closure will + /// fail and result in a `None` value. + /// + /// # Panics + /// + /// If `data_fn` panics, the panic is propagated to the caller, and the + /// temporary [`Weak`] is dropped normally. + /// + /// # Examples + /// + /// See [`new_cyclic`]. + /// + /// [`new_cyclic`]: Rc::new_cyclic + /// [`upgrade`]: Weak::upgrade + #[cfg(not(no_global_oom_handling))] + #[unstable(feature = "allocator_api", issue = "32838")] + pub fn new_cyclic_in(data_fn: F, alloc: A) -> Rc + where + F: FnOnce(&Weak) -> T, + { + // Construct the inner in the "uninitialized" state with a single + // weak reference. + let (uninit_raw_ptr, alloc) = Box::into_raw_with_allocator(Box::new_in( + RcInner { + strong: Cell::new(0), + weak: Cell::new(1), + value: mem::MaybeUninit::::uninit(), + }, + alloc, + )); + let uninit_ptr: NonNull<_> = (unsafe { &mut *uninit_raw_ptr }).into(); + let init_ptr: NonNull> = uninit_ptr.cast(); + + let weak = Weak { ptr: init_ptr, alloc: alloc }; + + // It's important we don't give up ownership of the weak pointer, or + // else the memory might be freed by the time `data_fn` returns. If + // we really wanted to pass ownership, we could create an additional + // weak pointer for ourselves, but this would result in additional + // updates to the weak reference count which might not be necessary + // otherwise. + let data = data_fn(&weak); + + let strong = unsafe { + let inner = init_ptr.as_ptr(); + ptr::write(&raw mut (*inner).value, data); + + let prev_value = (*inner).strong.get(); + debug_assert_eq!(prev_value, 0, "No prior strong references should exist"); + (*inner).strong.set(1); + + // Strong references should collectively own a shared weak reference, + // so don't run the destructor for our old weak reference. + // Calling into_raw_with_allocator has the double effect of giving us back the allocator, + // and forgetting the weak reference. + let alloc = weak.into_raw_with_allocator().1; + + Rc::from_inner_in(init_ptr, alloc) + }; + + strong + } + /// Constructs a new `Rc` in the provided allocator, returning an error if the allocation /// fails /// @@ -783,7 +845,7 @@ impl Rc { // the allocation while the strong destructor is running, even // if the weak pointer is stored inside the strong one. let (ptr, alloc) = Box::into_unique(Box::try_new_in( - RcBox { strong: Cell::new(1), weak: Cell::new(1), value }, + RcInner { strong: Cell::new(1), weak: Cell::new(1), value }, alloc, )?); Ok(unsafe { Self::from_inner_in(ptr.into(), alloc) }) @@ -1016,7 +1078,7 @@ impl Rc<[T]> { |layout| Global.allocate_zeroed(layout), |mem| { ptr::slice_from_raw_parts_mut(mem.cast::(), len) - as *mut RcBox<[mem::MaybeUninit]> + as *mut RcInner<[mem::MaybeUninit]> }, )) } @@ -1089,7 +1151,7 @@ impl Rc<[T], A> { |layout| alloc.allocate_zeroed(layout), |mem| { ptr::slice_from_raw_parts_mut(mem.cast::(), len) - as *mut RcBox<[mem::MaybeUninit]> + as *mut RcInner<[mem::MaybeUninit]> }, ), alloc, @@ -1394,12 +1456,12 @@ impl Rc { #[stable(feature = "weak_into_raw", since = "1.45.0")] #[rustc_never_returns_null_ptr] pub fn as_ptr(this: &Self) -> *const T { - let ptr: *mut RcBox = NonNull::as_ptr(this.ptr); + let ptr: *mut RcInner = NonNull::as_ptr(this.ptr); // SAFETY: This cannot go through Deref::deref or Rc::inner because // this is required to retain raw/mut provenance such that e.g. `get_mut` can // write through the pointer after the Rc is recovered through `from_raw`. - unsafe { ptr::addr_of_mut!((*ptr).value) } + unsafe { &raw mut (*ptr).value } } /// Constructs an `Rc` from a raw pointer in the provided allocator. @@ -1473,8 +1535,8 @@ impl Rc { pub unsafe fn from_raw_in(ptr: *const T, alloc: A) -> Self { let offset = unsafe { data_offset(ptr) }; - // Reverse the offset to find the original RcBox. - let rc_ptr = unsafe { ptr.byte_sub(offset) as *mut RcBox }; + // Reverse the offset to find the original RcInner. + let rc_ptr = unsafe { ptr.byte_sub(offset) as *mut RcInner }; unsafe { Self::from_ptr_in(rc_ptr, alloc) } } @@ -1959,48 +2021,48 @@ impl Rc { } impl Rc { - /// Allocates an `RcBox` with sufficient space for + /// Allocates an `RcInner` with sufficient space for /// a possibly-unsized inner value where the value has the layout provided. /// - /// The function `mem_to_rcbox` is called with the data pointer - /// and must return back a (potentially fat)-pointer for the `RcBox`. + /// The function `mem_to_rc_inner` is called with the data pointer + /// and must return back a (potentially fat)-pointer for the `RcInner`. #[cfg(not(no_global_oom_handling))] unsafe fn allocate_for_layout( value_layout: Layout, allocate: impl FnOnce(Layout) -> Result, AllocError>, - mem_to_rcbox: impl FnOnce(*mut u8) -> *mut RcBox, - ) -> *mut RcBox { - let layout = rcbox_layout_for_value_layout(value_layout); + mem_to_rc_inner: impl FnOnce(*mut u8) -> *mut RcInner, + ) -> *mut RcInner { + let layout = rc_inner_layout_for_value_layout(value_layout); unsafe { - Rc::try_allocate_for_layout(value_layout, allocate, mem_to_rcbox) + Rc::try_allocate_for_layout(value_layout, allocate, mem_to_rc_inner) .unwrap_or_else(|_| handle_alloc_error(layout)) } } - /// Allocates an `RcBox` with sufficient space for + /// Allocates an `RcInner` with sufficient space for /// a possibly-unsized inner value where the value has the layout provided, /// returning an error if allocation fails. /// - /// The function `mem_to_rcbox` is called with the data pointer - /// and must return back a (potentially fat)-pointer for the `RcBox`. + /// The function `mem_to_rc_inner` is called with the data pointer + /// and must return back a (potentially fat)-pointer for the `RcInner`. #[inline] unsafe fn try_allocate_for_layout( value_layout: Layout, allocate: impl FnOnce(Layout) -> Result, AllocError>, - mem_to_rcbox: impl FnOnce(*mut u8) -> *mut RcBox, - ) -> Result<*mut RcBox, AllocError> { - let layout = rcbox_layout_for_value_layout(value_layout); + mem_to_rc_inner: impl FnOnce(*mut u8) -> *mut RcInner, + ) -> Result<*mut RcInner, AllocError> { + let layout = rc_inner_layout_for_value_layout(value_layout); // Allocate for the layout. let ptr = allocate(layout)?; - // Initialize the RcBox - let inner = mem_to_rcbox(ptr.as_non_null_ptr().as_ptr()); + // Initialize the RcInner + let inner = mem_to_rc_inner(ptr.as_non_null_ptr().as_ptr()); unsafe { debug_assert_eq!(Layout::for_value_raw(inner), layout); - ptr::addr_of_mut!((*inner).strong).write(Cell::new(1)); - ptr::addr_of_mut!((*inner).weak).write(Cell::new(1)); + (&raw mut (*inner).strong).write(Cell::new(1)); + (&raw mut (*inner).weak).write(Cell::new(1)); } Ok(inner) @@ -2008,15 +2070,15 @@ impl Rc { } impl Rc { - /// Allocates an `RcBox` with sufficient space for an unsized inner value + /// Allocates an `RcInner` with sufficient space for an unsized inner value #[cfg(not(no_global_oom_handling))] - unsafe fn allocate_for_ptr_in(ptr: *const T, alloc: &A) -> *mut RcBox { - // Allocate for the `RcBox` using the given value. + unsafe fn allocate_for_ptr_in(ptr: *const T, alloc: &A) -> *mut RcInner { + // Allocate for the `RcInner` using the given value. unsafe { Rc::::allocate_for_layout( Layout::for_value_raw(ptr), |layout| alloc.allocate(layout), - |mem| mem.with_metadata_of(ptr as *const RcBox), + |mem| mem.with_metadata_of(ptr as *const RcInner), ) } } @@ -2029,8 +2091,8 @@ impl Rc { // Copy value as bytes ptr::copy_nonoverlapping( - core::ptr::addr_of!(*src) as *const u8, - ptr::addr_of_mut!((*ptr).value) as *mut u8, + (&raw const *src) as *const u8, + (&raw mut (*ptr).value) as *mut u8, value_size, ); @@ -2045,14 +2107,14 @@ impl Rc { } impl Rc<[T]> { - /// Allocates an `RcBox<[T]>` with the given length. + /// Allocates an `RcInner<[T]>` with the given length. #[cfg(not(no_global_oom_handling))] - unsafe fn allocate_for_slice(len: usize) -> *mut RcBox<[T]> { + unsafe fn allocate_for_slice(len: usize) -> *mut RcInner<[T]> { unsafe { Self::allocate_for_layout( Layout::array::(len).unwrap(), |layout| Global.allocate(layout), - |mem| ptr::slice_from_raw_parts_mut(mem.cast::(), len) as *mut RcBox<[T]>, + |mem| ptr::slice_from_raw_parts_mut(mem.cast::(), len) as *mut RcInner<[T]>, ) } } @@ -2064,11 +2126,7 @@ impl Rc<[T]> { unsafe fn copy_from_slice(v: &[T]) -> Rc<[T]> { unsafe { let ptr = Self::allocate_for_slice(v.len()); - ptr::copy_nonoverlapping( - v.as_ptr(), - ptr::addr_of_mut!((*ptr).value) as *mut T, - v.len(), - ); + ptr::copy_nonoverlapping(v.as_ptr(), (&raw mut (*ptr).value) as *mut T, v.len()); Self::from_ptr(ptr) } } @@ -2080,7 +2138,7 @@ impl Rc<[T]> { unsafe fn from_iter_exact(iter: impl Iterator, len: usize) -> Rc<[T]> { // Panic guard while cloning T elements. // In the event of a panic, elements that have been written - // into the new RcBox will be dropped, then the memory freed. + // into the new RcInner will be dropped, then the memory freed. struct Guard { mem: NonNull, elems: *mut T, @@ -2106,7 +2164,7 @@ impl Rc<[T]> { let layout = Layout::for_value_raw(ptr); // Pointer to first element - let elems = ptr::addr_of_mut!((*ptr).value) as *mut T; + let elems = (&raw mut (*ptr).value) as *mut T; let mut guard = Guard { mem: NonNull::new_unchecked(mem), elems, layout, n_elems: 0 }; @@ -2115,7 +2173,7 @@ impl Rc<[T]> { guard.n_elems += 1; } - // All clear. Forget the guard so it doesn't free the new RcBox. + // All clear. Forget the guard so it doesn't free the new RcInner. mem::forget(guard); Self::from_ptr(ptr) @@ -2124,15 +2182,15 @@ impl Rc<[T]> { } impl Rc<[T], A> { - /// Allocates an `RcBox<[T]>` with the given length. + /// Allocates an `RcInner<[T]>` with the given length. #[inline] #[cfg(not(no_global_oom_handling))] - unsafe fn allocate_for_slice_in(len: usize, alloc: &A) -> *mut RcBox<[T]> { + unsafe fn allocate_for_slice_in(len: usize, alloc: &A) -> *mut RcInner<[T]> { unsafe { Rc::<[T]>::allocate_for_layout( Layout::array::(len).unwrap(), |layout| alloc.allocate(layout), - |mem| ptr::slice_from_raw_parts_mut(mem.cast::(), len) as *mut RcBox<[T]>, + |mem| ptr::slice_from_raw_parts_mut(mem.cast::(), len) as *mut RcInner<[T]>, ) } } @@ -2179,8 +2237,8 @@ unsafe impl PinCoerceUnsized for Weak {} #[unstable(feature = "deref_pure_trait", issue = "87121")] unsafe impl DerefPure for Rc {} -#[unstable(feature = "receiver_trait", issue = "none")] -impl Receiver for Rc {} +#[unstable(feature = "legacy_receiver_trait", issue = "none")] +impl LegacyReceiver for Rc {} #[stable(feature = "rust1", since = "1.0.0")] unsafe impl<#[may_dangle] T: ?Sized, A: Allocator> Drop for Rc { @@ -2209,21 +2267,12 @@ unsafe impl<#[may_dangle] T: ?Sized, A: Allocator> Drop for Rc { /// drop(foo); // Doesn't print anything /// drop(foo2); // Prints "dropped!" /// ``` + #[inline] fn drop(&mut self) { unsafe { self.inner().dec_strong(); if self.inner().strong() == 0 { - // destroy the contained object - ptr::drop_in_place(Self::get_mut_unchecked(self)); - - // remove the implicit "strong weak" pointer now that we've - // destroyed the contents. - self.inner().dec_weak(); - - if self.inner().weak() == 0 { - self.alloc - .deallocate(self.ptr.cast(), Layout::for_value_raw(self.ptr.as_ptr())); - } + self.drop_slow(); } } } @@ -2269,7 +2318,16 @@ impl Default for Rc { /// ``` #[inline] fn default() -> Rc { - Rc::new(Default::default()) + unsafe { + Self::from_inner( + Box::leak(Box::write(Box::new_uninit(), RcInner { + strong: Cell::new(1), + weak: Cell::new(1), + value: T::default(), + })) + .into(), + ) + } } } @@ -2534,7 +2592,7 @@ impl fmt::Debug for Rc { #[stable(feature = "rust1", since = "1.0.0")] impl fmt::Pointer for Rc { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - fmt::Pointer::fmt(&core::ptr::addr_of!(**self), f) + fmt::Pointer::fmt(&(&raw const **self), f) } } @@ -2675,7 +2733,7 @@ impl From> for Rc<[T], A> { let (vec_ptr, len, cap, alloc) = v.into_raw_parts_with_alloc(); let rc_ptr = Self::allocate_for_slice_in(len, &alloc); - ptr::copy_nonoverlapping(vec_ptr, ptr::addr_of_mut!((*rc_ptr).value) as *mut T, len); + ptr::copy_nonoverlapping(vec_ptr, (&raw mut (*rc_ptr).value) as *mut T, len); // Create a `Vec` with length 0, to deallocate the buffer // without dropping its contents or the allocator @@ -2862,9 +2920,9 @@ pub struct Weak< // but it is not necessarily a valid pointer. // `Weak::new` sets this to `usize::MAX` so that it doesn’t need // to allocate space on the heap. That's not a value a real pointer - // will ever have because RcBox has alignment at least 2. + // will ever have because RcInner has alignment at least 2. // This is only possible when `T: Sized`; unsized `T` never dangle. - ptr: NonNull>, + ptr: NonNull>, alloc: A, } @@ -2900,7 +2958,7 @@ impl Weak { pub const fn new() -> Weak { Weak { ptr: unsafe { - NonNull::new_unchecked(ptr::without_provenance_mut::>(usize::MAX)) + NonNull::new_unchecked(ptr::without_provenance_mut::>(usize::MAX)) }, alloc: Global, } @@ -2927,7 +2985,7 @@ impl Weak { pub fn new_in(alloc: A) -> Weak { Weak { ptr: unsafe { - NonNull::new_unchecked(ptr::without_provenance_mut::>(usize::MAX)) + NonNull::new_unchecked(ptr::without_provenance_mut::>(usize::MAX)) }, alloc, } @@ -3023,7 +3081,7 @@ impl Weak { /// /// drop(strong); /// // But not any more. We can do weak.as_ptr(), but accessing the pointer would lead to - /// // undefined behaviour. + /// // undefined behavior. /// // assert_eq!("hello", unsafe { &*weak.as_ptr() }); /// ``` /// @@ -3031,17 +3089,17 @@ impl Weak { #[must_use] #[stable(feature = "rc_as_ptr", since = "1.45.0")] pub fn as_ptr(&self) -> *const T { - let ptr: *mut RcBox = NonNull::as_ptr(self.ptr); + let ptr: *mut RcInner = NonNull::as_ptr(self.ptr); if is_dangling(ptr) { // If the pointer is dangling, we return the sentinel directly. This cannot be - // a valid payload address, as the payload is at least as aligned as RcBox (usize). + // a valid payload address, as the payload is at least as aligned as RcInner (usize). ptr as *const T } else { // SAFETY: if is_dangling returns false, then the pointer is dereferenceable. // The payload may be dropped at this point, and we have to maintain provenance, // so use raw pointer manipulation. - unsafe { ptr::addr_of_mut!((*ptr).value) } + unsafe { &raw mut (*ptr).value } } } @@ -3167,14 +3225,14 @@ impl Weak { let ptr = if is_dangling(ptr) { // This is a dangling Weak. - ptr as *mut RcBox + ptr as *mut RcInner } else { // Otherwise, we're guaranteed the pointer came from a nondangling Weak. // SAFETY: data_offset is safe to call, as ptr references a real (potentially dropped) T. let offset = unsafe { data_offset(ptr) }; - // Thus, we reverse the offset to get the whole RcBox. + // Thus, we reverse the offset to get the whole RcInner. // SAFETY: the pointer originated from a Weak, so this offset is safe. - unsafe { ptr.byte_sub(offset) as *mut RcBox } + unsafe { ptr.byte_sub(offset) as *mut RcInner } }; // SAFETY: we now have recovered the original Weak pointer, so can create the Weak. @@ -3249,7 +3307,7 @@ impl Weak { } } - /// Returns `None` when the pointer is dangling and there is no allocated `RcBox`, + /// Returns `None` when the pointer is dangling and there is no allocated `RcInner`, /// (i.e., when this `Weak` was created by `Weak::new`). #[inline] fn inner(&self) -> Option> { @@ -3483,7 +3541,7 @@ trait RcInnerPtr { } } -impl RcInnerPtr for RcBox { +impl RcInnerPtr for RcInner { #[inline(always)] fn weak_ref(&self) -> &Cell { &self.weak @@ -3524,15 +3582,15 @@ impl AsRef for Rc { #[stable(feature = "pin", since = "1.33.0")] impl Unpin for Rc {} -/// Gets the offset within an `RcBox` for the payload behind a pointer. +/// Gets the offset within an `RcInner` for the payload behind a pointer. /// /// # Safety /// /// The pointer must point to (and have valid metadata for) a previously /// valid instance of T, but the T is allowed to be dropped. unsafe fn data_offset(ptr: *const T) -> usize { - // Align the unsized value to the end of the RcBox. - // Because RcBox is repr(C), it will always be the last field in memory. + // Align the unsized value to the end of the RcInner. + // Because RcInner is repr(C), it will always be the last field in memory. // SAFETY: since the only unsized types possible are slices, trait objects, // and extern types, the input safety requirement is currently enough to // satisfy the requirements of align_of_val_raw; this is an implementation @@ -3542,7 +3600,7 @@ unsafe fn data_offset(ptr: *const T) -> usize { #[inline] fn data_offset_align(align: usize) -> usize { - let layout = Layout::new::>(); + let layout = Layout::new::>(); layout.size() + layout.padding_needed_for(align) } @@ -3588,8 +3646,8 @@ pub struct UniqueRc< T: ?Sized, #[unstable(feature = "allocator_api", issue = "32838")] A: Allocator = Global, > { - ptr: NonNull>, - phantom: PhantomData>, + ptr: NonNull>, + phantom: PhantomData>, alloc: A, } @@ -3625,7 +3683,7 @@ impl UniqueRc { #[unstable(feature = "unique_rc_arc", issue = "112566")] pub fn new_in(value: T, alloc: A) -> Self { let (ptr, alloc) = Box::into_unique(Box::new_in( - RcBox { + RcInner { strong: Cell::new(0), // keep one weak reference so if all the weak pointers that are created are dropped // the UniqueRc still stays valid. @@ -3720,7 +3778,7 @@ unsafe impl<#[may_dangle] T: ?Sized, A: Allocator> Drop for UniqueRc { } } -/// A unique owning pointer to a [`RcBox`] **that does not imply the contents are initialized,** +/// A unique owning pointer to a [`RcInner`] **that does not imply the contents are initialized,** /// but will deallocate it (without dropping the value) when dropped. /// /// This is a helper for [`Rc::make_mut()`] to ensure correct cleanup on panic. @@ -3728,21 +3786,21 @@ unsafe impl<#[may_dangle] T: ?Sized, A: Allocator> Drop for UniqueRc { /// which `MaybeUninit` does not. #[cfg(not(no_global_oom_handling))] struct UniqueRcUninit { - ptr: NonNull>, + ptr: NonNull>, layout_for_value: Layout, alloc: Option, } #[cfg(not(no_global_oom_handling))] impl UniqueRcUninit { - /// Allocates a RcBox with layout suitable to contain `for_value` or a clone of it. + /// Allocates a RcInner with layout suitable to contain `for_value` or a clone of it. fn new(for_value: &T, alloc: A) -> UniqueRcUninit { let layout = Layout::for_value(for_value); let ptr = unsafe { Rc::allocate_for_layout( layout, - |layout_for_rcbox| alloc.allocate(layout_for_rcbox), - |mem| mem.with_metadata_of(ptr::from_ref(for_value) as *const RcBox), + |layout_for_rc_inner| alloc.allocate(layout_for_rc_inner), + |mem| mem.with_metadata_of(ptr::from_ref(for_value) as *const RcInner), ) }; Self { ptr: NonNull::new(ptr).unwrap(), layout_for_value: layout, alloc: Some(alloc) } @@ -3777,10 +3835,10 @@ impl Drop for UniqueRcUninit { // * new() produced a pointer safe to deallocate. // * We own the pointer unless into_rc() was called, which forgets us. unsafe { - self.alloc - .take() - .unwrap() - .deallocate(self.ptr.cast(), rcbox_layout_for_value_layout(self.layout_for_value)); + self.alloc.take().unwrap().deallocate( + self.ptr.cast(), + rc_inner_layout_for_value_layout(self.layout_for_value), + ); } } } diff --git a/library/alloc/src/rc/tests.rs b/library/alloc/src/rc/tests.rs index 84e8b325f71fc..333e1bde31c1e 100644 --- a/library/alloc/src/rc/tests.rs +++ b/library/alloc/src/rc/tests.rs @@ -448,7 +448,11 @@ fn test_from_box_str() { use std::string::String; let s = String::from("foo").into_boxed_str(); + assert_eq!((&&&s).as_str(), "foo"); + let r: Rc = Rc::from(s); + assert_eq!((&r).as_str(), "foo"); + assert_eq!(r.as_str(), "foo"); assert_eq!(&r[..], "foo"); } diff --git a/library/alloc/src/slice.rs b/library/alloc/src/slice.rs index 9d70487032699..e3c7835f1d10b 100644 --- a/library/alloc/src/slice.rs +++ b/library/alloc/src/slice.rs @@ -19,20 +19,6 @@ use core::cmp::Ordering::{self, Less}; use core::mem::{self, MaybeUninit}; #[cfg(not(no_global_oom_handling))] use core::ptr; -#[cfg(not(no_global_oom_handling))] -use core::slice::sort; - -use crate::alloc::Allocator; -#[cfg(not(no_global_oom_handling))] -use crate::alloc::Global; -#[cfg(not(no_global_oom_handling))] -use crate::borrow::ToOwned; -use crate::boxed::Box; -use crate::vec::Vec; - -#[cfg(test)] -mod tests; - #[unstable(feature = "array_chunks", issue = "74985")] pub use core::slice::ArrayChunks; #[unstable(feature = "array_chunks", issue = "74985")] @@ -43,14 +29,8 @@ pub use core::slice::ArrayWindows; pub use core::slice::EscapeAscii; #[stable(feature = "slice_get_slice", since = "1.28.0")] pub use core::slice::SliceIndex; -#[stable(feature = "from_ref", since = "1.28.0")] -pub use core::slice::{from_mut, from_ref}; -#[unstable(feature = "slice_from_ptr_range", issue = "89792")] -pub use core::slice::{from_mut_ptr_range, from_ptr_range}; -#[stable(feature = "rust1", since = "1.0.0")] -pub use core::slice::{from_raw_parts, from_raw_parts_mut}; -#[unstable(feature = "slice_range", issue = "76393")] -pub use core::slice::{range, try_range}; +#[cfg(not(no_global_oom_handling))] +use core::slice::sort; #[stable(feature = "slice_group_by", since = "1.77.0")] pub use core::slice::{ChunkBy, ChunkByMut}; #[stable(feature = "rust1", since = "1.0.0")] @@ -69,6 +49,14 @@ pub use core::slice::{RSplit, RSplitMut}; pub use core::slice::{RSplitN, RSplitNMut, SplitN, SplitNMut}; #[stable(feature = "split_inclusive", since = "1.51.0")] pub use core::slice::{SplitInclusive, SplitInclusiveMut}; +#[stable(feature = "from_ref", since = "1.28.0")] +pub use core::slice::{from_mut, from_ref}; +#[unstable(feature = "slice_from_ptr_range", issue = "89792")] +pub use core::slice::{from_mut_ptr_range, from_ptr_range}; +#[stable(feature = "rust1", since = "1.0.0")] +pub use core::slice::{from_raw_parts, from_raw_parts_mut}; +#[unstable(feature = "slice_range", issue = "76393")] +pub use core::slice::{range, try_range}; //////////////////////////////////////////////////////////////////////////////// // Basic slice extension methods @@ -83,6 +71,14 @@ pub use hack::into_vec; #[cfg(test)] pub use hack::to_vec; +use crate::alloc::Allocator; +#[cfg(not(no_global_oom_handling))] +use crate::alloc::Global; +#[cfg(not(no_global_oom_handling))] +use crate::borrow::ToOwned; +use crate::boxed::Box; +use crate::vec::Vec; + // HACK(japaric): With cfg(test) `impl [T]` is not available, these three // functions are actually methods that are in `impl [T]` but not in // `core::slice::SliceExt` - we need to supply these functions for the @@ -96,6 +92,7 @@ pub(crate) mod hack { // We shouldn't add inline attribute to this since this is used in // `vec!` macro mostly and causes perf regression. See #71204 for // discussion and perf results. + #[allow(missing_docs)] pub fn into_vec(b: Box<[T], A>) -> Vec { unsafe { let len = b.len(); @@ -105,6 +102,7 @@ pub(crate) mod hack { } #[cfg(not(no_global_oom_handling))] + #[allow(missing_docs)] #[inline] pub fn to_vec(s: &[T], alloc: A) -> Vec { T::to_vec(s, alloc) @@ -178,10 +176,9 @@ impl [T] { /// This sort is stable (i.e., does not reorder equal elements) and *O*(*n* \* log(*n*)) /// worst-case. /// - /// If the implementation of [`Ord`] for `T` does not implement a [total order] the resulting - /// order of elements in the slice is unspecified. All original elements will remain in the - /// slice and any possible modifications via interior mutability are observed in the input. Same - /// is true if the implementation of [`Ord`] for `T` panics. + /// If the implementation of [`Ord`] for `T` does not implement a [total order], the function + /// may panic; even if the function exits normally, the resulting order of elements in the slice + /// is unspecified. See also the note on panicking below. /// /// When applicable, unstable sorting is preferred because it is generally faster than stable /// sorting and it doesn't allocate auxiliary memory. See @@ -210,7 +207,15 @@ impl [T] { /// /// # Panics /// - /// May panic if the implementation of [`Ord`] for `T` does not implement a [total order]. + /// May panic if the implementation of [`Ord`] for `T` does not implement a [total order], or if + /// the [`Ord`] implementation itself panics. + /// + /// All safe functions on slices preserve the invariant that even if the function panics, all + /// original elements will remain in the slice and any possible modifications via interior + /// mutability are observed in the input. This ensures that recovery code (for instance inside + /// of a `Drop` or following a `catch_unwind`) will still have access to all the original + /// elements. For instance, if the slice belongs to a `Vec`, the `Vec::drop` method will be able + /// to dispose of all contained elements. /// /// # Examples /// @@ -239,10 +244,9 @@ impl [T] { /// This sort is stable (i.e., does not reorder equal elements) and *O*(*n* \* log(*n*)) /// worst-case. /// - /// If the comparison function `compare` does not implement a [total order] the resulting order - /// of elements in the slice is unspecified. All original elements will remain in the slice and - /// any possible modifications via interior mutability are observed in the input. Same is true - /// if `compare` panics. + /// If the comparison function `compare` does not implement a [total order], the function may + /// panic; even if the function exits normally, the resulting order of elements in the slice is + /// unspecified. See also the note on panicking below. /// /// For example `|a, b| (a - b).cmp(a)` is a comparison function that is neither transitive nor /// reflexive nor total, `a < b < c < a` with `a = 1, b = 2, c = 3`. For more information and @@ -261,7 +265,14 @@ impl [T] { /// /// # Panics /// - /// May panic if `compare` does not implement a [total order]. + /// May panic if `compare` does not implement a [total order], or if `compare` itself panics. + /// + /// All safe functions on slices preserve the invariant that even if the function panics, all + /// original elements will remain in the slice and any possible modifications via interior + /// mutability are observed in the input. This ensures that recovery code (for instance inside + /// of a `Drop` or following a `catch_unwind`) will still have access to all the original + /// elements. For instance, if the slice belongs to a `Vec`, the `Vec::drop` method will be able + /// to dispose of all contained elements. /// /// # Examples /// @@ -293,10 +304,9 @@ impl [T] { /// This sort is stable (i.e., does not reorder equal elements) and *O*(*m* \* *n* \* log(*n*)) /// worst-case, where the key function is *O*(*m*). /// - /// If the implementation of [`Ord`] for `K` does not implement a [total order] the resulting - /// order of elements in the slice is unspecified. All original elements will remain in the - /// slice and any possible modifications via interior mutability are observed in the input. Same - /// is true if the implementation of [`Ord`] for `K` panics. + /// If the implementation of [`Ord`] for `K` does not implement a [total order], the function + /// may panic; even if the function exits normally, the resulting order of elements in the slice + /// is unspecified. See also the note on panicking below. /// /// # Current implementation /// @@ -311,7 +321,15 @@ impl [T] { /// /// # Panics /// - /// May panic if the implementation of [`Ord`] for `K` does not implement a [total order]. + /// May panic if the implementation of [`Ord`] for `K` does not implement a [total order], or if + /// the [`Ord`] implementation or the key-function `f` panics. + /// + /// All safe functions on slices preserve the invariant that even if the function panics, all + /// original elements will remain in the slice and any possible modifications via interior + /// mutability are observed in the input. This ensures that recovery code (for instance inside + /// of a `Drop` or following a `catch_unwind`) will still have access to all the original + /// elements. For instance, if the slice belongs to a `Vec`, the `Vec::drop` method will be able + /// to dispose of all contained elements. /// /// # Examples /// @@ -345,10 +363,9 @@ impl [T] { /// storage to remember the results of key evaluation. The order of calls to the key function is /// unspecified and may change in future versions of the standard library. /// - /// If the implementation of [`Ord`] for `K` does not implement a [total order] the resulting - /// order of elements in the slice is unspecified. All original elements will remain in the - /// slice and any possible modifications via interior mutability are observed in the input. Same - /// is true if the implementation of [`Ord`] for `K` panics. + /// If the implementation of [`Ord`] for `K` does not implement a [total order], the function + /// may panic; even if the function exits normally, the resulting order of elements in the slice + /// is unspecified. See also the note on panicking below. /// /// For simple key functions (e.g., functions that are property accesses or basic operations), /// [`sort_by_key`](slice::sort_by_key) is likely to be faster. @@ -367,7 +384,15 @@ impl [T] { /// /// # Panics /// - /// May panic if the implementation of [`Ord`] for `K` does not implement a [total order]. + /// May panic if the implementation of [`Ord`] for `K` does not implement a [total order], or if + /// the [`Ord`] implementation panics. + /// + /// All safe functions on slices preserve the invariant that even if the function panics, all + /// original elements will remain in the slice and any possible modifications via interior + /// mutability are observed in the input. This ensures that recovery code (for instance inside + /// of a `Drop` or following a `catch_unwind`) will still have access to all the original + /// elements. For instance, if the slice belongs to a `Vec`, the `Vec::drop` method will be able + /// to dispose of all contained elements. /// /// # Examples /// @@ -494,6 +519,7 @@ impl [T] { #[rustc_allow_incoherent_impl] #[stable(feature = "rust1", since = "1.0.0")] #[inline] + #[cfg_attr(not(test), rustc_diagnostic_item = "slice_into_vec")] pub fn into_vec(self: Box) -> Vec { // N.B., see the `hack` module in this file for more details. hack::into_vec(self) diff --git a/library/alloc/src/slice/tests.rs b/library/alloc/src/slice/tests.rs deleted file mode 100644 index 786704caeb0ad..0000000000000 --- a/library/alloc/src/slice/tests.rs +++ /dev/null @@ -1,369 +0,0 @@ -use core::cell::Cell; -use core::cmp::Ordering::{self, Equal, Greater, Less}; -use core::convert::identity; -use core::sync::atomic::AtomicUsize; -use core::sync::atomic::Ordering::Relaxed; -use core::{fmt, mem}; -use std::panic; - -use rand::distributions::Standard; -use rand::prelude::*; -use rand::{Rng, RngCore}; - -use crate::borrow::ToOwned; -use crate::rc::Rc; -use crate::string::ToString; -use crate::test_helpers::test_rng; -use crate::vec::Vec; - -macro_rules! do_test { - ($input:ident, $func:ident) => { - let len = $input.len(); - - // Work out the total number of comparisons required to sort - // this array... - let mut count = 0usize; - $input.to_owned().$func(|a, b| { - count += 1; - a.cmp(b) - }); - - // ... and then panic on each and every single one. - for panic_countdown in 0..count { - // Refresh the counters. - VERSIONS.store(0, Relaxed); - for i in 0..len { - DROP_COUNTS[i].store(0, Relaxed); - } - - let v = $input.to_owned(); - let _ = panic::catch_unwind(move || { - let mut v = v; - let mut panic_countdown = panic_countdown; - v.$func(|a, b| { - if panic_countdown == 0 { - SILENCE_PANIC.with(|s| s.set(true)); - panic!(); - } - panic_countdown -= 1; - a.cmp(b) - }) - }); - - // Check that the number of things dropped is exactly - // what we expect (i.e., the contents of `v`). - for (i, c) in DROP_COUNTS.iter().enumerate().take(len) { - let count = c.load(Relaxed); - assert!(count == 1, "found drop count == {} for i == {}, len == {}", count, i, len); - } - - // Check that the most recent versions of values were dropped. - assert_eq!(VERSIONS.load(Relaxed), 0); - } - }; -} - -const MAX_LEN: usize = 80; - -static DROP_COUNTS: [AtomicUsize; MAX_LEN] = [ - // FIXME(RFC 1109): AtomicUsize is not Copy. - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), - AtomicUsize::new(0), -]; - -static VERSIONS: AtomicUsize = AtomicUsize::new(0); - -#[derive(Clone, Eq)] -struct DropCounter { - x: u32, - id: usize, - version: Cell, -} - -impl PartialEq for DropCounter { - fn eq(&self, other: &Self) -> bool { - self.partial_cmp(other) == Some(Ordering::Equal) - } -} - -impl PartialOrd for DropCounter { - fn partial_cmp(&self, other: &Self) -> Option { - self.version.set(self.version.get() + 1); - other.version.set(other.version.get() + 1); - VERSIONS.fetch_add(2, Relaxed); - self.x.partial_cmp(&other.x) - } -} - -impl Ord for DropCounter { - fn cmp(&self, other: &Self) -> Ordering { - self.partial_cmp(other).unwrap() - } -} - -impl Drop for DropCounter { - fn drop(&mut self) { - DROP_COUNTS[self.id].fetch_add(1, Relaxed); - VERSIONS.fetch_sub(self.version.get(), Relaxed); - } -} - -std::thread_local!(static SILENCE_PANIC: Cell = Cell::new(false)); - -#[test] -#[cfg_attr(target_os = "emscripten", ignore)] // no threads -#[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] -fn panic_safe() { - panic::update_hook(move |prev, info| { - if !SILENCE_PANIC.with(|s| s.get()) { - prev(info); - } - }); - - let mut rng = test_rng(); - - // Miri is too slow (but still need to `chain` to make the types match) - let lens = if cfg!(miri) { (1..10).chain(0..0) } else { (1..20).chain(70..MAX_LEN) }; - let moduli: &[u32] = if cfg!(miri) { &[5] } else { &[5, 20, 50] }; - - for len in lens { - for &modulus in moduli { - for &has_runs in &[false, true] { - let mut input = (0..len) - .map(|id| DropCounter { - x: rng.next_u32() % modulus, - id: id, - version: Cell::new(0), - }) - .collect::>(); - - if has_runs { - for c in &mut input { - c.x = c.id as u32; - } - - for _ in 0..5 { - let a = rng.gen::() % len; - let b = rng.gen::() % len; - if a < b { - input[a..b].reverse(); - } else { - input.swap(a, b); - } - } - } - - do_test!(input, sort_by); - do_test!(input, sort_unstable_by); - } - } - } - - // Set default panic hook again. - drop(panic::take_hook()); -} - -#[test] -#[cfg_attr(miri, ignore)] // Miri is too slow -#[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] -fn test_sort() { - let mut rng = test_rng(); - - for len in (2..25).chain(500..510) { - for &modulus in &[5, 10, 100, 1000] { - for _ in 0..10 { - let orig: Vec<_> = (&mut rng) - .sample_iter::(&Standard) - .map(|x| x % modulus) - .take(len) - .collect(); - - // Sort in default order. - let mut v = orig.clone(); - v.sort(); - assert!(v.windows(2).all(|w| w[0] <= w[1])); - - // Sort in ascending order. - let mut v = orig.clone(); - v.sort_by(|a, b| a.cmp(b)); - assert!(v.windows(2).all(|w| w[0] <= w[1])); - - // Sort in descending order. - let mut v = orig.clone(); - v.sort_by(|a, b| b.cmp(a)); - assert!(v.windows(2).all(|w| w[0] >= w[1])); - - // Sort in lexicographic order. - let mut v1 = orig.clone(); - let mut v2 = orig.clone(); - v1.sort_by_key(|x| x.to_string()); - v2.sort_by_cached_key(|x| x.to_string()); - assert!(v1.windows(2).all(|w| w[0].to_string() <= w[1].to_string())); - assert!(v1 == v2); - - // Sort with many pre-sorted runs. - let mut v = orig.clone(); - v.sort(); - v.reverse(); - for _ in 0..5 { - let a = rng.gen::() % len; - let b = rng.gen::() % len; - if a < b { - v[a..b].reverse(); - } else { - v.swap(a, b); - } - } - v.sort(); - assert!(v.windows(2).all(|w| w[0] <= w[1])); - } - } - } - - const ORD_VIOLATION_MAX_LEN: usize = 500; - let mut v = [0; ORD_VIOLATION_MAX_LEN]; - for i in 0..ORD_VIOLATION_MAX_LEN { - v[i] = i as i32; - } - - // Sort using a completely random comparison function. This will reorder the elements *somehow*, - // it may panic but the original elements must still be present. - let _ = panic::catch_unwind(move || { - v.sort_by(|_, _| *[Less, Equal, Greater].choose(&mut rng).unwrap()); - }); - - v.sort(); - for i in 0..ORD_VIOLATION_MAX_LEN { - assert_eq!(v[i], i as i32); - } - - // Should not panic. - [0i32; 0].sort(); - [(); 10].sort(); - [(); 100].sort(); - - let mut v = [0xDEADBEEFu64]; - v.sort(); - assert!(v == [0xDEADBEEF]); -} - -#[test] -fn test_sort_stability() { - // Miri is too slow - let large_range = if cfg!(miri) { 0..0 } else { 500..510 }; - let rounds = if cfg!(miri) { 1 } else { 10 }; - - let mut rng = test_rng(); - for len in (2..25).chain(large_range) { - for _ in 0..rounds { - let mut counts = [0; 10]; - - // create a vector like [(6, 1), (5, 1), (6, 2), ...], - // where the first item of each tuple is random, but - // the second item represents which occurrence of that - // number this element is, i.e., the second elements - // will occur in sorted order. - let orig: Vec<_> = (0..len) - .map(|_| { - let n = rng.gen::() % 10; - counts[n] += 1; - (n, counts[n]) - }) - .collect(); - - let mut v = orig.clone(); - // Only sort on the first element, so an unstable sort - // may mix up the counts. - v.sort_by(|&(a, _), &(b, _)| a.cmp(&b)); - - // This comparison includes the count (the second item - // of the tuple), so elements with equal first items - // will need to be ordered with increasing - // counts... i.e., exactly asserting that this sort is - // stable. - assert!(v.windows(2).all(|w| w[0] <= w[1])); - - let mut v = orig.clone(); - v.sort_by_cached_key(|&(x, _)| x); - assert!(v.windows(2).all(|w| w[0] <= w[1])); - } - } -} diff --git a/library/alloc/src/str.rs b/library/alloc/src/str.rs index d7fba3ae159c6..26c1ba2a5c485 100644 --- a/library/alloc/src/str.rs +++ b/library/alloc/src/str.rs @@ -9,9 +9,7 @@ use core::borrow::{Borrow, BorrowMut}; use core::iter::FusedIterator; -#[stable(feature = "rust1", since = "1.0.0")] -pub use core::str::pattern; -use core::str::pattern::{DoubleEndedSearcher, Pattern, ReverseSearcher, Searcher}; +use core::mem::MaybeUninit; #[stable(feature = "encode_utf16", since = "1.8.0")] pub use core::str::EncodeUtf16; #[stable(feature = "split_ascii_whitespace", since = "1.34.0")] @@ -20,12 +18,11 @@ pub use core::str::SplitAsciiWhitespace; pub use core::str::SplitInclusive; #[stable(feature = "rust1", since = "1.0.0")] pub use core::str::SplitWhitespace; -#[unstable(feature = "str_from_raw_parts", issue = "119206")] -pub use core::str::{from_raw_parts, from_raw_parts_mut}; #[stable(feature = "rust1", since = "1.0.0")] -pub use core::str::{from_utf8, from_utf8_mut, Bytes, CharIndices, Chars}; +pub use core::str::pattern; +use core::str::pattern::{DoubleEndedSearcher, Pattern, ReverseSearcher, Searcher, Utf8Pattern}; #[stable(feature = "rust1", since = "1.0.0")] -pub use core::str::{from_utf8_unchecked, from_utf8_unchecked_mut, ParseBoolError}; +pub use core::str::{Bytes, CharIndices, Chars, from_utf8, from_utf8_mut}; #[stable(feature = "str_escape", since = "1.34.0")] pub use core::str::{EscapeDebug, EscapeDefault, EscapeUnicode}; #[stable(feature = "rust1", since = "1.0.0")] @@ -38,6 +35,8 @@ pub use core::str::{MatchIndices, RMatchIndices}; #[stable(feature = "rust1", since = "1.0.0")] pub use core::str::{Matches, RMatches}; #[stable(feature = "rust1", since = "1.0.0")] +pub use core::str::{ParseBoolError, from_utf8_unchecked, from_utf8_unchecked_mut}; +#[stable(feature = "rust1", since = "1.0.0")] pub use core::str::{RSplit, Split}; #[stable(feature = "rust1", since = "1.0.0")] pub use core::str::{RSplitN, SplitN}; @@ -45,6 +44,8 @@ pub use core::str::{RSplitN, SplitN}; pub use core::str::{RSplitTerminator, SplitTerminator}; #[stable(feature = "utf8_chunks", since = "1.79.0")] pub use core::str::{Utf8Chunk, Utf8Chunks}; +#[unstable(feature = "str_from_raw_parts", issue = "119206")] +pub use core::str::{from_raw_parts, from_raw_parts_mut}; use core::unicode::conversions; use core::{mem, ptr}; @@ -268,7 +269,23 @@ impl str { #[stable(feature = "rust1", since = "1.0.0")] #[inline] pub fn replace(&self, from: P, to: &str) -> String { - let mut result = String::new(); + // Fast path for replacing a single ASCII character with another. + if let Some(from_byte) = match from.as_utf8_pattern() { + Some(Utf8Pattern::StringPattern([from_byte])) => Some(*from_byte), + Some(Utf8Pattern::CharPattern(c)) => c.as_ascii().map(|ascii_char| ascii_char.to_u8()), + _ => None, + } { + if let [to_byte] = to.as_bytes() { + return unsafe { replace_ascii(self.as_bytes(), from_byte, *to_byte) }; + } + } + // Set result capacity to self.len() when from.len() <= to.len() + let default_capacity = match from.as_utf8_pattern() { + Some(Utf8Pattern::StringPattern(s)) if s.len() <= to.len() => self.len(), + Some(Utf8Pattern::CharPattern(c)) if c.len_utf8() <= to.len() => self.len(), + _ => 0, + }; + let mut result = String::with_capacity(default_capacity); let mut last_end = 0; for (start, part) in self.match_indices(from) { result.push_str(unsafe { self.get_unchecked(last_end..start) }); @@ -365,14 +382,9 @@ impl str { without modifying the original"] #[stable(feature = "unicode_case_mapping", since = "1.2.0")] pub fn to_lowercase(&self) -> String { - let out = convert_while_ascii(self.as_bytes(), u8::to_ascii_lowercase); - - // Safety: we know this is a valid char boundary since - // out.len() is only progressed if ascii bytes are found - let rest = unsafe { self.get_unchecked(out.len()..) }; + let (mut s, rest) = convert_while_ascii(self, u8::to_ascii_lowercase); - // Safety: We have written only valid ASCII to our vec - let mut s = unsafe { String::from_utf8_unchecked(out) }; + let prefix_len = s.len(); for (i, c) in rest.char_indices() { if c == 'Σ' { @@ -381,8 +393,7 @@ impl str { // in `SpecialCasing.txt`, // so hard-code it rather than have a generic "condition" mechanism. // See https://github.com/rust-lang/rust/issues/26035 - let out_len = self.len() - rest.len(); - let sigma_lowercase = map_uppercase_sigma(&self, i + out_len); + let sigma_lowercase = map_uppercase_sigma(self, prefix_len + i); s.push(sigma_lowercase); } else { match conversions::to_lower(c) { @@ -458,14 +469,7 @@ impl str { without modifying the original"] #[stable(feature = "unicode_case_mapping", since = "1.2.0")] pub fn to_uppercase(&self) -> String { - let out = convert_while_ascii(self.as_bytes(), u8::to_ascii_uppercase); - - // Safety: we know this is a valid char boundary since - // out.len() is only progressed if ascii bytes are found - let rest = unsafe { self.get_unchecked(out.len()..) }; - - // Safety: We have written only valid ASCII to our vec - let mut s = unsafe { String::from_utf8_unchecked(out) }; + let (mut s, rest) = convert_while_ascii(self, u8::to_ascii_uppercase); for c in rest.chars() { match conversions::to_upper(c) { @@ -614,50 +618,98 @@ pub unsafe fn from_boxed_utf8_unchecked(v: Box<[u8]>) -> Box { unsafe { Box::from_raw(Box::into_raw(v) as *mut str) } } -/// Converts the bytes while the bytes are still ascii. +/// Converts leading ascii bytes in `s` by calling the `convert` function. +/// /// For better average performance, this happens in chunks of `2*size_of::()`. -/// Returns a vec with the converted bytes. +/// +/// Returns a tuple of the converted prefix and the remainder starting from +/// the first non-ascii character. +/// +/// This function is only public so that it can be verified in a codegen test, +/// see `issue-123712-str-to-lower-autovectorization.rs`. +#[unstable(feature = "str_internals", issue = "none")] +#[doc(hidden)] #[inline] #[cfg(not(test))] #[cfg(not(no_global_oom_handling))] -fn convert_while_ascii(b: &[u8], convert: fn(&u8) -> u8) -> Vec { - let mut out = Vec::with_capacity(b.len()); +pub fn convert_while_ascii(s: &str, convert: fn(&u8) -> u8) -> (String, &str) { + // Process the input in chunks of 16 bytes to enable auto-vectorization. + // Previously the chunk size depended on the size of `usize`, + // but on 32-bit platforms with sse or neon is also the better choice. + // The only downside on other platforms would be a bit more loop-unrolling. + const N: usize = 16; + + let mut slice = s.as_bytes(); + let mut out = Vec::with_capacity(slice.len()); + let mut out_slice = out.spare_capacity_mut(); + + let mut ascii_prefix_len = 0_usize; + let mut is_ascii = [false; N]; + + while slice.len() >= N { + // SAFETY: checked in loop condition + let chunk = unsafe { slice.get_unchecked(..N) }; + // SAFETY: out_slice has at least same length as input slice and gets sliced with the same offsets + let out_chunk = unsafe { out_slice.get_unchecked_mut(..N) }; + + for j in 0..N { + is_ascii[j] = chunk[j] <= 127; + } - const USIZE_SIZE: usize = mem::size_of::(); - const MAGIC_UNROLL: usize = 2; - const N: usize = USIZE_SIZE * MAGIC_UNROLL; - const NONASCII_MASK: usize = usize::from_ne_bytes([0x80; USIZE_SIZE]); + // Auto-vectorization for this check is a bit fragile, sum and comparing against the chunk + // size gives the best result, specifically a pmovmsk instruction on x86. + // See https://github.com/llvm/llvm-project/issues/96395 for why llvm currently does not + // currently recognize other similar idioms. + if is_ascii.iter().map(|x| *x as u8).sum::() as usize != N { + break; + } - let mut i = 0; - unsafe { - while i + N <= b.len() { - // Safety: we have checks the sizes `b` and `out` to know that our - let in_chunk = b.get_unchecked(i..i + N); - let out_chunk = out.spare_capacity_mut().get_unchecked_mut(i..i + N); - - let mut bits = 0; - for j in 0..MAGIC_UNROLL { - // read the bytes 1 usize at a time (unaligned since we haven't checked the alignment) - // safety: in_chunk is valid bytes in the range - bits |= in_chunk.as_ptr().cast::().add(j).read_unaligned(); - } - // if our chunks aren't ascii, then return only the prior bytes as init - if bits & NONASCII_MASK != 0 { - break; - } + for j in 0..N { + out_chunk[j] = MaybeUninit::new(convert(&chunk[j])); + } - // perform the case conversions on N bytes (gets heavily autovec'd) - for j in 0..N { - // safety: in_chunk and out_chunk is valid bytes in the range - let out = out_chunk.get_unchecked_mut(j); - out.write(convert(in_chunk.get_unchecked(j))); - } + ascii_prefix_len += N; + slice = unsafe { slice.get_unchecked(N..) }; + out_slice = unsafe { out_slice.get_unchecked_mut(N..) }; + } - // mark these bytes as initialised - i += N; + // handle the remainder as individual bytes + while slice.len() > 0 { + let byte = slice[0]; + if byte > 127 { + break; } - out.set_len(i); + // SAFETY: out_slice has at least same length as input slice + unsafe { + *out_slice.get_unchecked_mut(0) = MaybeUninit::new(convert(&byte)); + } + ascii_prefix_len += 1; + slice = unsafe { slice.get_unchecked(1..) }; + out_slice = unsafe { out_slice.get_unchecked_mut(1..) }; } - out + unsafe { + // SAFETY: ascii_prefix_len bytes have been initialized above + out.set_len(ascii_prefix_len); + + // SAFETY: We have written only valid ascii to the output vec + let ascii_string = String::from_utf8_unchecked(out); + + // SAFETY: we know this is a valid char boundary + // since we only skipped over leading ascii bytes + let rest = core::str::from_utf8_unchecked(slice); + + (ascii_string, rest) + } +} +#[inline] +#[cfg(not(test))] +#[cfg(not(no_global_oom_handling))] +#[allow(dead_code)] +/// Faster implementation of string replacement for ASCII to ASCII cases. +/// Should produce fast vectorized code. +unsafe fn replace_ascii(utf8_bytes: &[u8], from: u8, to: u8) -> String { + let result: Vec = utf8_bytes.iter().map(|b| if *b == from { to } else { *b }).collect(); + // SAFETY: We replaced ascii with ascii on valid utf8 strings. + unsafe { String::from_utf8_unchecked(result) } } diff --git a/library/alloc/src/string.rs b/library/alloc/src/string.rs index bc8b7e24bf12b..b042720933b6d 100644 --- a/library/alloc/src/string.rs +++ b/library/alloc/src/string.rs @@ -43,9 +43,9 @@ #![stable(feature = "rust1", since = "1.0.0")] use core::error::Error; +use core::iter::FusedIterator; #[cfg(not(no_global_oom_handling))] use core::iter::from_fn; -use core::iter::FusedIterator; #[cfg(not(no_global_oom_handling))] use core::ops::Add; #[cfg(not(no_global_oom_handling))] @@ -53,7 +53,7 @@ use core::ops::AddAssign; #[cfg(not(no_global_oom_handling))] use core::ops::Bound::{Excluded, Included, Unbounded}; use core::ops::{self, Range, RangeBounds}; -use core::str::pattern::Pattern; +use core::str::pattern::{Pattern, Utf8Pattern}; use core::{fmt, hash, ptr, slice}; #[cfg(not(no_global_oom_handling))] @@ -62,9 +62,9 @@ use crate::alloc::Allocator; use crate::borrow::{Cow, ToOwned}; use crate::boxed::Box; use crate::collections::TryReserveError; -use crate::str::{self, from_utf8_unchecked_mut, Chars, Utf8Error}; +use crate::str::{self, Chars, Utf8Error, from_utf8_unchecked_mut}; #[cfg(not(no_global_oom_handling))] -use crate::str::{from_boxed_utf8_unchecked, FromStr}; +use crate::str::{FromStr, from_boxed_utf8_unchecked}; use crate::vec::Vec; /// A UTF-8–encoded, growable string. @@ -440,6 +440,7 @@ impl String { /// ``` #[inline] #[rustc_const_stable(feature = "const_string_new", since = "1.39.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "string_new")] #[stable(feature = "rust1", since = "1.0.0")] #[must_use] pub const fn new() -> String { @@ -508,6 +509,7 @@ impl String { // NB see the slice::hack module in slice.rs for more information #[inline] #[cfg(test)] + #[allow(missing_docs)] pub fn from_str(_: &str) -> String { panic!("not available with cfg(test)"); } @@ -570,6 +572,7 @@ impl String { /// [`into_bytes`]: String::into_bytes #[inline] #[stable(feature = "rust1", since = "1.0.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "string_from_utf8")] pub fn from_utf8(vec: Vec) -> Result { match str::from_utf8(&vec) { Ok(..) => Ok(String { vec }), @@ -659,6 +662,56 @@ impl String { Cow::Owned(res) } + /// Converts a [`Vec`] to a `String`, substituting invalid UTF-8 + /// sequences with replacement characters. + /// + /// See [`from_utf8_lossy`] for more details. + /// + /// [`from_utf8_lossy`]: String::from_utf8_lossy + /// + /// Note that this function does not guarantee reuse of the original `Vec` + /// allocation. + /// + /// # Examples + /// + /// Basic usage: + /// + /// ``` + /// #![feature(string_from_utf8_lossy_owned)] + /// // some bytes, in a vector + /// let sparkle_heart = vec![240, 159, 146, 150]; + /// + /// let sparkle_heart = String::from_utf8_lossy_owned(sparkle_heart); + /// + /// assert_eq!(String::from("💖"), sparkle_heart); + /// ``` + /// + /// Incorrect bytes: + /// + /// ``` + /// #![feature(string_from_utf8_lossy_owned)] + /// // some invalid bytes + /// let input: Vec = b"Hello \xF0\x90\x80World".into(); + /// let output = String::from_utf8_lossy_owned(input); + /// + /// assert_eq!(String::from("Hello �World"), output); + /// ``` + #[must_use] + #[cfg(not(no_global_oom_handling))] + #[unstable(feature = "string_from_utf8_lossy_owned", issue = "129436")] + pub fn from_utf8_lossy_owned(v: Vec) -> String { + if let Cow::Owned(string) = String::from_utf8_lossy(&v) { + string + } else { + // SAFETY: `String::from_utf8_lossy`'s contract ensures that if + // it returns a `Cow::Borrowed`, it is a valid UTF-8 string. + // Otherwise, it returns a new allocation of an owned `String`, with + // replacement characters for invalid sequences, which is returned + // above. + unsafe { String::from_utf8_unchecked(v) } + } + } + /// Decode a UTF-16–encoded vector `v` into a `String`, returning [`Err`] /// if `v` contains any invalid data. /// @@ -1006,7 +1059,8 @@ impl String { #[inline] #[must_use = "`self` will be dropped if the result is not used"] #[stable(feature = "rust1", since = "1.0.0")] - pub fn into_bytes(self) -> Vec { + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] + pub const fn into_bytes(self) -> Vec { self.vec } @@ -1022,8 +1076,12 @@ impl String { #[inline] #[must_use] #[stable(feature = "string_as_str", since = "1.7.0")] - pub fn as_str(&self) -> &str { - self + #[cfg_attr(not(test), rustc_diagnostic_item = "string_as_str")] + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] + pub const fn as_str(&self) -> &str { + // SAFETY: String contents are stipulated to be valid UTF-8, invalid contents are an error + // at construction. + unsafe { str::from_utf8_unchecked(self.vec.as_slice()) } } /// Converts a `String` into a mutable string slice. @@ -1041,8 +1099,12 @@ impl String { #[inline] #[must_use] #[stable(feature = "string_as_str", since = "1.7.0")] - pub fn as_mut_str(&mut self) -> &mut str { - self + #[cfg_attr(not(test), rustc_diagnostic_item = "string_as_mut_str")] + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] + pub const fn as_mut_str(&mut self) -> &mut str { + // SAFETY: String contents are stipulated to be valid UTF-8, invalid contents are an error + // at construction. + unsafe { str::from_utf8_unchecked_mut(self.vec.as_mut_slice()) } } /// Appends a given string slice onto the end of this `String`. @@ -1060,6 +1122,7 @@ impl String { #[inline] #[stable(feature = "rust1", since = "1.0.0")] #[rustc_confusables("append", "push")] + #[cfg_attr(not(test), rustc_diagnostic_item = "string_push_str")] pub fn push_str(&mut self, string: &str) { self.vec.extend_from_slice(string.as_bytes()) } @@ -1112,7 +1175,8 @@ impl String { #[inline] #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - pub fn capacity(&self) -> usize { + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] + pub const fn capacity(&self) -> usize { self.vec.capacity() } @@ -1375,8 +1439,9 @@ impl String { #[inline] #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - pub fn as_bytes(&self) -> &[u8] { - &self.vec + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] + pub const fn as_bytes(&self) -> &[u8] { + self.vec.as_slice() } /// Shortens this `String` to the specified length. @@ -1694,6 +1759,7 @@ impl String { #[cfg(not(no_global_oom_handling))] #[inline] #[stable(feature = "insert_str", since = "1.16.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "string_insert_str")] pub fn insert_str(&mut self, idx: usize, string: &str) { assert!(self.is_char_boundary(idx)); @@ -1727,7 +1793,8 @@ impl String { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - pub unsafe fn as_mut_vec(&mut self) -> &mut Vec { + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] + pub const unsafe fn as_mut_vec(&mut self) -> &mut Vec { &mut self.vec } @@ -1748,8 +1815,9 @@ impl String { #[inline] #[must_use] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] #[rustc_confusables("length", "size")] - pub fn len(&self) -> usize { + pub const fn len(&self) -> usize { self.vec.len() } @@ -1767,7 +1835,8 @@ impl String { #[inline] #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - pub fn is_empty(&self) -> bool { + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] + pub const fn is_empty(&self) -> bool { self.len() == 0 } @@ -2009,6 +2078,54 @@ impl FromUtf8Error { &self.bytes[..] } + /// Converts the bytes into a `String` lossily, substituting invalid UTF-8 + /// sequences with replacement characters. + /// + /// See [`String::from_utf8_lossy`] for more details on replacement of + /// invalid sequences, and [`String::from_utf8_lossy_owned`] for the + /// `String` function which corresponds to this function. + /// + /// # Examples + /// + /// ``` + /// #![feature(string_from_utf8_lossy_owned)] + /// // some invalid bytes + /// let input: Vec = b"Hello \xF0\x90\x80World".into(); + /// let output = String::from_utf8(input).unwrap_or_else(|e| e.into_utf8_lossy()); + /// + /// assert_eq!(String::from("Hello �World"), output); + /// ``` + #[must_use] + #[cfg(not(no_global_oom_handling))] + #[unstable(feature = "string_from_utf8_lossy_owned", issue = "129436")] + pub fn into_utf8_lossy(self) -> String { + const REPLACEMENT: &str = "\u{FFFD}"; + + let mut res = { + let mut v = Vec::with_capacity(self.bytes.len()); + + // `Utf8Error::valid_up_to` returns the maximum index of validated + // UTF-8 bytes. Copy the valid bytes into the output buffer. + v.extend_from_slice(&self.bytes[..self.error.valid_up_to()]); + + // SAFETY: This is safe because the only bytes present in the buffer + // were validated as UTF-8 by the call to `String::from_utf8` which + // produced this `FromUtf8Error`. + unsafe { String::from_utf8_unchecked(v) } + }; + + let iter = self.bytes[self.error.valid_up_to()..].utf8_chunks(); + + for chunk in iter { + res.push_str(chunk.valid()); + if !chunk.invalid().is_empty() { + res.push_str(REPLACEMENT); + } + } + + res + } + /// Returns the bytes that were attempted to convert to a `String`. /// /// This method is carefully constructed to avoid allocation. It will @@ -2319,6 +2436,11 @@ impl<'b> Pattern for &'b String { { self[..].strip_suffix_of(haystack) } + + #[inline] + fn as_utf8_pattern(&self) -> Option> { + Some(Utf8Pattern::StringPattern(self.as_bytes())) + } } macro_rules! impl_eq { @@ -2484,7 +2606,7 @@ impl ops::Deref for String { #[inline] fn deref(&self) -> &str { - unsafe { str::from_utf8_unchecked(&self.vec) } + self.as_str() } } @@ -2495,7 +2617,7 @@ unsafe impl ops::DerefPure for String {} impl ops::DerefMut for String { #[inline] fn deref_mut(&mut self) -> &mut str { - unsafe { str::from_utf8_unchecked_mut(&mut *self.vec) } + self.as_mut_str() } } diff --git a/library/alloc/src/sync.rs b/library/alloc/src/sync.rs index 43684f31cb723..98a2fe242570f 100644 --- a/library/alloc/src/sync.rs +++ b/library/alloc/src/sync.rs @@ -17,8 +17,8 @@ use core::intrinsics::abort; #[cfg(not(no_global_oom_handling))] use core::iter; use core::marker::{PhantomData, Unsize}; -use core::mem::{self, align_of_val_raw, ManuallyDrop}; -use core::ops::{CoerceUnsized, Deref, DerefPure, DispatchFromDyn, Receiver}; +use core::mem::{self, ManuallyDrop, align_of_val_raw}; +use core::ops::{CoerceUnsized, Deref, DerefPure, DispatchFromDyn, LegacyReceiver}; use core::panic::{RefUnwindSafe, UnwindSafe}; use core::pin::{Pin, PinCoerceUnsized}; use core::ptr::{self, NonNull}; @@ -237,6 +237,7 @@ macro_rules! acquire { /// [rc_examples]: crate::rc#examples #[cfg_attr(not(test), rustc_diagnostic_item = "Arc")] #[stable(feature = "rust1", since = "1.0.0")] +#[rustc_insignificant_dtor] pub struct Arc< T: ?Sized, #[unstable(feature = "allocator_api", issue = "32838")] A: Allocator = Global, @@ -318,7 +319,7 @@ pub struct Weak< // but it is not necessarily a valid pointer. // `Weak::new` sets this to `usize::MAX` so that it doesn’t need // to allocate space on the heap. That's not a value a real pointer - // will ever have because RcBox has alignment at least 2. + // will ever have because RcInner has alignment at least 2. // This is only possible when `T: Sized`; unsized `T` never dangle. ptr: NonNull>, alloc: A, @@ -450,54 +451,7 @@ impl Arc { where F: FnOnce(&Weak) -> T, { - // Construct the inner in the "uninitialized" state with a single - // weak reference. - let uninit_ptr: NonNull<_> = Box::leak(Box::new(ArcInner { - strong: atomic::AtomicUsize::new(0), - weak: atomic::AtomicUsize::new(1), - data: mem::MaybeUninit::::uninit(), - })) - .into(); - let init_ptr: NonNull> = uninit_ptr.cast(); - - let weak = Weak { ptr: init_ptr, alloc: Global }; - - // It's important we don't give up ownership of the weak pointer, or - // else the memory might be freed by the time `data_fn` returns. If - // we really wanted to pass ownership, we could create an additional - // weak pointer for ourselves, but this would result in additional - // updates to the weak reference count which might not be necessary - // otherwise. - let data = data_fn(&weak); - - // Now we can properly initialize the inner value and turn our weak - // reference into a strong reference. - let strong = unsafe { - let inner = init_ptr.as_ptr(); - ptr::write(ptr::addr_of_mut!((*inner).data), data); - - // The above write to the data field must be visible to any threads which - // observe a non-zero strong count. Therefore we need at least "Release" ordering - // in order to synchronize with the `compare_exchange_weak` in `Weak::upgrade`. - // - // "Acquire" ordering is not required. When considering the possible behaviours - // of `data_fn` we only need to look at what it could do with a reference to a - // non-upgradeable `Weak`: - // - It can *clone* the `Weak`, increasing the weak reference count. - // - It can drop those clones, decreasing the weak reference count (but never to zero). - // - // These side effects do not impact us in any way, and no other side effects are - // possible with safe code alone. - let prev_value = (*inner).strong.fetch_add(1, Release); - debug_assert_eq!(prev_value, 0, "No prior strong references should exist"); - - Arc::from_inner(init_ptr) - }; - - // Strong references should collectively own a shared weak reference, - // so don't run the destructor for our old weak reference. - mem::forget(weak); - strong + Self::new_cyclic_in(data_fn, Global) } /// Constructs a new `Arc` with uninitialized contents. @@ -781,6 +735,98 @@ impl Arc { } } + /// Constructs a new `Arc` in the given allocator while giving you a `Weak` to the allocation, + /// to allow you to construct a `T` which holds a weak pointer to itself. + /// + /// Generally, a structure circularly referencing itself, either directly or + /// indirectly, should not hold a strong reference to itself to prevent a memory leak. + /// Using this function, you get access to the weak pointer during the + /// initialization of `T`, before the `Arc` is created, such that you can + /// clone and store it inside the `T`. + /// + /// `new_cyclic_in` first allocates the managed allocation for the `Arc`, + /// then calls your closure, giving it a `Weak` to this allocation, + /// and only afterwards completes the construction of the `Arc` by placing + /// the `T` returned from your closure into the allocation. + /// + /// Since the new `Arc` is not fully-constructed until `Arc::new_cyclic_in` + /// returns, calling [`upgrade`] on the weak reference inside your closure will + /// fail and result in a `None` value. + /// + /// # Panics + /// + /// If `data_fn` panics, the panic is propagated to the caller, and the + /// temporary [`Weak`] is dropped normally. + /// + /// # Example + /// + /// See [`new_cyclic`] + /// + /// [`new_cyclic`]: Arc::new_cyclic + /// [`upgrade`]: Weak::upgrade + #[cfg(not(no_global_oom_handling))] + #[inline] + #[unstable(feature = "allocator_api", issue = "32838")] + pub fn new_cyclic_in(data_fn: F, alloc: A) -> Arc + where + F: FnOnce(&Weak) -> T, + { + // Construct the inner in the "uninitialized" state with a single + // weak reference. + let (uninit_raw_ptr, alloc) = Box::into_raw_with_allocator(Box::new_in( + ArcInner { + strong: atomic::AtomicUsize::new(0), + weak: atomic::AtomicUsize::new(1), + data: mem::MaybeUninit::::uninit(), + }, + alloc, + )); + let uninit_ptr: NonNull<_> = (unsafe { &mut *uninit_raw_ptr }).into(); + let init_ptr: NonNull> = uninit_ptr.cast(); + + let weak = Weak { ptr: init_ptr, alloc: alloc }; + + // It's important we don't give up ownership of the weak pointer, or + // else the memory might be freed by the time `data_fn` returns. If + // we really wanted to pass ownership, we could create an additional + // weak pointer for ourselves, but this would result in additional + // updates to the weak reference count which might not be necessary + // otherwise. + let data = data_fn(&weak); + + // Now we can properly initialize the inner value and turn our weak + // reference into a strong reference. + let strong = unsafe { + let inner = init_ptr.as_ptr(); + ptr::write(&raw mut (*inner).data, data); + + // The above write to the data field must be visible to any threads which + // observe a non-zero strong count. Therefore we need at least "Release" ordering + // in order to synchronize with the `compare_exchange_weak` in `Weak::upgrade`. + // + // "Acquire" ordering is not required. When considering the possible behaviors + // of `data_fn` we only need to look at what it could do with a reference to a + // non-upgradeable `Weak`: + // - It can *clone* the `Weak`, increasing the weak reference count. + // - It can drop those clones, decreasing the weak reference count (but never to zero). + // + // These side effects do not impact us in any way, and no other side effects are + // possible with safe code alone. + let prev_value = (*inner).strong.fetch_add(1, Release); + debug_assert_eq!(prev_value, 0, "No prior strong references should exist"); + + // Strong references should collectively own a shared weak reference, + // so don't run the destructor for our old weak reference. + // Calling into_raw_with_allocator has the double effect of giving us back the allocator, + // and forgetting the weak reference. + let alloc = weak.into_raw_with_allocator().1; + + Arc::from_inner_in(init_ptr, alloc) + }; + + strong + } + /// Constructs a new `Pin>` in the provided allocator. If `T` does not implement `Unpin`, /// then `data` will be pinned in memory and unable to be moved. #[cfg(not(no_global_oom_handling))] @@ -1535,10 +1581,10 @@ impl Arc { pub fn as_ptr(this: &Self) -> *const T { let ptr: *mut ArcInner = NonNull::as_ptr(this.ptr); - // SAFETY: This cannot go through Deref::deref or RcBoxPtr::inner because + // SAFETY: This cannot go through Deref::deref or RcInnerPtr::inner because // this is required to retain raw/mut provenance such that e.g. `get_mut` can // write through the pointer after the Rc is recovered through `from_raw`. - unsafe { ptr::addr_of_mut!((*ptr).data) } + unsafe { &raw mut (*ptr).data } } /// Constructs an `Arc` from a raw pointer. @@ -1826,15 +1872,17 @@ impl Arc { // Non-inlined part of `drop`. #[inline(never)] unsafe fn drop_slow(&mut self) { + // Drop the weak ref collectively held by all strong references when this + // variable goes out of scope. This ensures that the memory is deallocated + // even if the destructor of `T` panics. + // Take a reference to `self.alloc` instead of cloning because 1. it'll last long + // enough, and 2. you should be able to drop `Arc`s with unclonable allocators + let _weak = Weak { ptr: self.ptr, alloc: &self.alloc }; + // Destroy the data at this time, even though we must not free the box // allocation itself (there might still be weak pointers lying around). - unsafe { ptr::drop_in_place(Self::get_mut_unchecked(self)) }; - - // Drop the weak ref collectively held by all strong references - // Take a reference to `self.alloc` instead of cloning because 1. it'll - // last long enough, and 2. you should be able to drop `Arc`s with - // unclonable allocators - drop(Weak { ptr: self.ptr, alloc: &self.alloc }); + // We cannot use `get_mut_unchecked` here, because `self.alloc` is borrowed. + unsafe { ptr::drop_in_place(&mut (*self.ptr.as_ptr()).data) }; } /// Returns `true` if the two `Arc`s point to the same allocation in a vein similar to @@ -1910,8 +1958,8 @@ impl Arc { debug_assert_eq!(unsafe { Layout::for_value_raw(inner) }, layout); unsafe { - ptr::addr_of_mut!((*inner).strong).write(atomic::AtomicUsize::new(1)); - ptr::addr_of_mut!((*inner).weak).write(atomic::AtomicUsize::new(1)); + (&raw mut (*inner).strong).write(atomic::AtomicUsize::new(1)); + (&raw mut (*inner).weak).write(atomic::AtomicUsize::new(1)); } inner @@ -1941,8 +1989,8 @@ impl Arc { // Copy value as bytes ptr::copy_nonoverlapping( - core::ptr::addr_of!(*src) as *const u8, - ptr::addr_of_mut!((*ptr).data) as *mut u8, + (&raw const *src) as *const u8, + (&raw mut (*ptr).data) as *mut u8, value_size, ); @@ -1977,7 +2025,7 @@ impl Arc<[T]> { unsafe { let ptr = Self::allocate_for_slice(v.len()); - ptr::copy_nonoverlapping(v.as_ptr(), ptr::addr_of_mut!((*ptr).data) as *mut T, v.len()); + ptr::copy_nonoverlapping(v.as_ptr(), (&raw mut (*ptr).data) as *mut T, v.len()); Self::from_ptr(ptr) } @@ -2016,7 +2064,7 @@ impl Arc<[T]> { let layout = Layout::for_value_raw(ptr); // Pointer to first element - let elems = ptr::addr_of_mut!((*ptr).data) as *mut T; + let elems = (&raw mut (*ptr).data) as *mut T; let mut guard = Guard { mem: NonNull::new_unchecked(mem), elems, layout, n_elems: 0 }; @@ -2143,8 +2191,8 @@ unsafe impl PinCoerceUnsized for Weak {} #[unstable(feature = "deref_pure_trait", issue = "87121")] unsafe impl DerefPure for Arc {} -#[unstable(feature = "receiver_trait", issue = "none")] -impl Receiver for Arc {} +#[unstable(feature = "legacy_receiver_trait", issue = "none")] +impl LegacyReceiver for Arc {} #[cfg(not(no_global_oom_handling))] impl Arc { @@ -2742,7 +2790,7 @@ impl Weak { /// /// drop(strong); /// // But not any more. We can do weak.as_ptr(), but accessing the pointer would lead to - /// // undefined behaviour. + /// // undefined behavior. /// // assert_eq!("hello", unsafe { &*weak.as_ptr() }); /// ``` /// @@ -2760,7 +2808,7 @@ impl Weak { // SAFETY: if is_dangling returns false, then the pointer is dereferenceable. // The payload may be dropped at this point, and we have to maintain provenance, // so use raw pointer manipulation. - unsafe { ptr::addr_of_mut!((*ptr).data) } + unsafe { &raw mut (*ptr).data } } } @@ -2890,7 +2938,7 @@ impl Weak { // Otherwise, we're guaranteed the pointer came from a nondangling Weak. // SAFETY: data_offset is safe to call, as ptr references a real (potentially dropped) T. let offset = unsafe { data_offset(ptr) }; - // Thus, we reverse the offset to get the whole RcBox. + // Thus, we reverse the offset to get the whole RcInner. // SAFETY: the pointer originated from a Weak, so this offset is safe. unsafe { ptr.byte_sub(offset) as *mut ArcInner } }; @@ -3383,7 +3431,7 @@ impl fmt::Debug for Arc { #[stable(feature = "rust1", since = "1.0.0")] impl fmt::Pointer for Arc { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - fmt::Pointer::fmt(&core::ptr::addr_of!(**self), f) + fmt::Pointer::fmt(&(&raw const **self), f) } } @@ -3401,7 +3449,16 @@ impl Default for Arc { /// assert_eq!(*x, 0); /// ``` fn default() -> Arc { - Arc::new(Default::default()) + unsafe { + Self::from_inner( + Box::leak(Box::write(Box::new_uninit(), ArcInner { + strong: atomic::AtomicUsize::new(1), + weak: atomic::AtomicUsize::new(1), + data: T::default(), + })) + .into(), + ) + } } } @@ -3633,7 +3690,7 @@ impl From> for Arc<[T], A> { let (vec_ptr, len, cap, alloc) = v.into_raw_parts_with_alloc(); let rc_ptr = Self::allocate_for_slice_in(len, &alloc); - ptr::copy_nonoverlapping(vec_ptr, ptr::addr_of_mut!((*rc_ptr).data) as *mut T, len); + ptr::copy_nonoverlapping(vec_ptr, (&raw mut (*rc_ptr).data) as *mut T, len); // Create a `Vec` with length 0, to deallocate the buffer // without dropping its contents or the allocator @@ -3815,7 +3872,7 @@ impl Unpin for Arc {} /// valid instance of T, but the T is allowed to be dropped. unsafe fn data_offset(ptr: *const T) -> usize { // Align the unsized value to the end of the ArcInner. - // Because RcBox is repr(C), it will always be the last field in memory. + // Because RcInner is repr(C), it will always be the last field in memory. // SAFETY: since the only unsized types possible are slices, trait objects, // and extern types, the input safety requirement is currently enough to // satisfy the requirements of align_of_val_raw; this is an implementation diff --git a/library/alloc/src/sync/tests.rs b/library/alloc/src/sync/tests.rs index d6b3de875771e..3f66c88992344 100644 --- a/library/alloc/src/sync/tests.rs +++ b/library/alloc/src/sync/tests.rs @@ -1,10 +1,10 @@ use std::clone::Clone; use std::mem::MaybeUninit; use std::option::Option::None; +use std::sync::Mutex; use std::sync::atomic::AtomicUsize; use std::sync::atomic::Ordering::SeqCst; use std::sync::mpsc::channel; -use std::sync::Mutex; use std::thread; use super::*; diff --git a/library/alloc/src/vec/cow.rs b/library/alloc/src/vec/cow.rs index c18091705a636..4deb35efffc14 100644 --- a/library/alloc/src/vec/cow.rs +++ b/library/alloc/src/vec/cow.rs @@ -58,6 +58,7 @@ impl<'a, T> FromIterator for Cow<'a, [T]> where T: Clone, { + #[track_caller] fn from_iter>(it: I) -> Cow<'a, [T]> { Cow::Owned(FromIterator::from_iter(it)) } diff --git a/library/alloc/src/vec/in_place_collect.rs b/library/alloc/src/vec/in_place_collect.rs index d119e6ca397c5..a7dba16944e7d 100644 --- a/library/alloc/src/vec/in_place_collect.rs +++ b/library/alloc/src/vec/in_place_collect.rs @@ -163,7 +163,7 @@ use core::num::NonZero; use core::ptr; use super::{InPlaceDrop, InPlaceDstDataSrcBufDrop, SpecFromIter, SpecFromIterNested, Vec}; -use crate::alloc::{handle_alloc_error, Global}; +use crate::alloc::{Global, handle_alloc_error}; const fn in_place_collectible( step_merge: Option>, @@ -191,7 +191,7 @@ const fn in_place_collectible( const fn needs_realloc(src_cap: usize, dst_cap: usize) -> bool { if const { mem::align_of::() != mem::align_of::() } { - // FIXME: use unreachable! once that works in const + // FIXME(const-hack): use unreachable! once that works in const panic!("in_place_collectible() prevents this"); } @@ -208,7 +208,7 @@ const fn needs_realloc(src_cap: usize, dst_cap: usize) -> bool { // type layouts don't guarantee a fit, so do a runtime check to see if // the allocations happen to match - return src_cap > 0 && src_cap * mem::size_of::() != dst_cap * mem::size_of::(); + src_cap > 0 && src_cap * mem::size_of::() != dst_cap * mem::size_of::() } /// This provides a shorthand for the source type since local type aliases aren't a thing. @@ -229,6 +229,7 @@ where I: Iterator + InPlaceCollect, ::Source: AsVecIntoIter, { + #[track_caller] default fn from_iter(iterator: I) -> Self { // Select the implementation in const eval to avoid codegen of the dead branch to improve compile times. let fun: fn(I) -> Vec = const { @@ -246,6 +247,7 @@ where } } +#[track_caller] fn from_iter_in_place(mut iterator: I) -> Vec where I: Iterator + InPlaceCollect, @@ -328,7 +330,7 @@ where mem::forget(dst_guard); - let vec = unsafe { Vec::from_nonnull(dst_buf, len, dst_cap) }; + let vec = unsafe { Vec::from_parts(dst_buf, len, dst_cap) }; vec } diff --git a/library/alloc/src/vec/in_place_drop.rs b/library/alloc/src/vec/in_place_drop.rs index 27f7597931045..4d5b4e47d39e4 100644 --- a/library/alloc/src/vec/in_place_drop.rs +++ b/library/alloc/src/vec/in_place_drop.rs @@ -1,5 +1,5 @@ use core::marker::PhantomData; -use core::ptr::{self, drop_in_place, NonNull}; +use core::ptr::{self, NonNull, drop_in_place}; use core::slice::{self}; use crate::alloc::Global; diff --git a/library/alloc/src/vec/into_iter.rs b/library/alloc/src/vec/into_iter.rs index fad8abad49353..9a6745fdbc0a3 100644 --- a/library/alloc/src/vec/into_iter.rs +++ b/library/alloc/src/vec/into_iter.rs @@ -21,11 +21,11 @@ use crate::raw_vec::RawVec; macro non_null { (mut $place:expr, $t:ident) => {{ #![allow(unused_unsafe)] // we're sometimes used within an unsafe block - unsafe { &mut *(ptr::addr_of_mut!($place) as *mut NonNull<$t>) } + unsafe { &mut *((&raw mut $place) as *mut NonNull<$t>) } }}, ($place:expr, $t:ident) => {{ #![allow(unused_unsafe)] // we're sometimes used within an unsafe block - unsafe { *(ptr::addr_of!($place) as *const NonNull<$t>) } + unsafe { *((&raw const $place) as *const NonNull<$t>) } }}, } @@ -142,7 +142,7 @@ impl IntoIter { // struct and then overwriting &mut self. // this creates less assembly self.cap = 0; - self.buf = RawVec::NEW.non_null(); + self.buf = RawVec::new().non_null(); self.ptr = self.buf; self.end = self.buf.as_ptr(); @@ -288,11 +288,11 @@ impl Iterator for IntoIter { // Safety: `len` is larger than the array size. Copy a fixed amount here to fully initialize // the array. - return unsafe { + unsafe { ptr::copy_nonoverlapping(self.ptr.as_ptr(), raw_ary.as_mut_ptr() as *mut T, N); self.ptr = self.ptr.add(N); Ok(raw_ary.transpose().assume_init()) - }; + } } fn fold(mut self, mut accum: B, mut f: F) -> B diff --git a/library/alloc/src/vec/is_zero.rs b/library/alloc/src/vec/is_zero.rs index bcc5bf4d65bb4..ba57d940d8c99 100644 --- a/library/alloc/src/vec/is_zero.rs +++ b/library/alloc/src/vec/is_zero.rs @@ -172,7 +172,7 @@ macro_rules! impl_is_zero_option_of_bool { fn is_zero(&self) -> bool { // SAFETY: This is *not* a stable layout guarantee, but // inside `core` we're allowed to rely on the current rustc - // behaviour that options of bools will be one byte with + // behavior that options of bools will be one byte with // no padding, so long as they're nested less than 254 deep. let raw: u8 = unsafe { core::mem::transmute(*self) }; raw == 0 diff --git a/library/alloc/src/vec/mod.rs b/library/alloc/src/vec/mod.rs index 162791ba59d03..07a1bd4932138 100644 --- a/library/alloc/src/vec/mod.rs +++ b/library/alloc/src/vec/mod.rs @@ -416,10 +416,11 @@ impl Vec { /// ``` #[inline] #[rustc_const_stable(feature = "const_vec_new", since = "1.39.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "vec_new")] #[stable(feature = "rust1", since = "1.0.0")] #[must_use] pub const fn new() -> Self { - Vec { buf: RawVec::NEW, len: 0 } + Vec { buf: RawVec::new(), len: 0 } } /// Constructs a new, empty `Vec` with at least the specified capacity. @@ -476,6 +477,8 @@ impl Vec { #[inline] #[stable(feature = "rust1", since = "1.0.0")] #[must_use] + #[cfg_attr(not(test), rustc_diagnostic_item = "vec_with_capacity")] + #[track_caller] pub fn with_capacity(capacity: usize) -> Self { Self::with_capacity_in(capacity, Global) } @@ -603,15 +606,116 @@ impl Vec { unsafe { Self::from_raw_parts_in(ptr, length, capacity, Global) } } - /// A convenience method for hoisting the non-null precondition out of [`Vec::from_raw_parts`]. + #[doc(alias = "from_non_null_parts")] + /// Creates a `Vec` directly from a `NonNull` pointer, a length, and a capacity. /// /// # Safety /// - /// See [`Vec::from_raw_parts`]. + /// This is highly unsafe, due to the number of invariants that aren't + /// checked: + /// + /// * `ptr` must have been allocated using the global allocator, such as via + /// the [`alloc::alloc`] function. + /// * `T` needs to have the same alignment as what `ptr` was allocated with. + /// (`T` having a less strict alignment is not sufficient, the alignment really + /// needs to be equal to satisfy the [`dealloc`] requirement that memory must be + /// allocated and deallocated with the same layout.) + /// * The size of `T` times the `capacity` (ie. the allocated size in bytes) needs + /// to be the same size as the pointer was allocated with. (Because similar to + /// alignment, [`dealloc`] must be called with the same layout `size`.) + /// * `length` needs to be less than or equal to `capacity`. + /// * The first `length` values must be properly initialized values of type `T`. + /// * `capacity` needs to be the capacity that the pointer was allocated with. + /// * The allocated size in bytes must be no larger than `isize::MAX`. + /// See the safety documentation of [`pointer::offset`]. + /// + /// These requirements are always upheld by any `ptr` that has been allocated + /// via `Vec`. Other allocation sources are allowed if the invariants are + /// upheld. + /// + /// Violating these may cause problems like corrupting the allocator's + /// internal data structures. For example it is normally **not** safe + /// to build a `Vec` from a pointer to a C `char` array with length + /// `size_t`, doing so is only safe if the array was initially allocated by + /// a `Vec` or `String`. + /// It's also not safe to build one from a `Vec` and its length, because + /// the allocator cares about the alignment, and these two types have different + /// alignments. The buffer was allocated with alignment 2 (for `u16`), but after + /// turning it into a `Vec` it'll be deallocated with alignment 1. To avoid + /// these issues, it is often preferable to do casting/transmuting using + /// [`NonNull::slice_from_raw_parts`] instead. + /// + /// The ownership of `ptr` is effectively transferred to the + /// `Vec` which may then deallocate, reallocate or change the + /// contents of memory pointed to by the pointer at will. Ensure + /// that nothing else uses the pointer after calling this + /// function. + /// + /// [`String`]: crate::string::String + /// [`alloc::alloc`]: crate::alloc::alloc + /// [`dealloc`]: crate::alloc::GlobalAlloc::dealloc + /// + /// # Examples + /// + /// ``` + /// #![feature(box_vec_non_null)] + /// + /// use std::ptr::NonNull; + /// use std::mem; + /// + /// let v = vec![1, 2, 3]; + /// + // FIXME Update this when vec_into_raw_parts is stabilized + /// // Prevent running `v`'s destructor so we are in complete control + /// // of the allocation. + /// let mut v = mem::ManuallyDrop::new(v); + /// + /// // Pull out the various important pieces of information about `v` + /// let p = unsafe { NonNull::new_unchecked(v.as_mut_ptr()) }; + /// let len = v.len(); + /// let cap = v.capacity(); + /// + /// unsafe { + /// // Overwrite memory with 4, 5, 6 + /// for i in 0..len { + /// p.add(i).write(4 + i); + /// } + /// + /// // Put everything back together into a Vec + /// let rebuilt = Vec::from_parts(p, len, cap); + /// assert_eq!(rebuilt, [4, 5, 6]); + /// } + /// ``` + /// + /// Using memory that was allocated elsewhere: + /// + /// ```rust + /// #![feature(box_vec_non_null)] + /// + /// use std::alloc::{alloc, Layout}; + /// use std::ptr::NonNull; + /// + /// fn main() { + /// let layout = Layout::array::(16).expect("overflow cannot happen"); + /// + /// let vec = unsafe { + /// let Some(mem) = NonNull::new(alloc(layout).cast::()) else { + /// return; + /// }; + /// + /// mem.write(1_000_000); + /// + /// Vec::from_parts(mem, 1, 16) + /// }; + /// + /// assert_eq!(vec, &[1_000_000]); + /// assert_eq!(vec.capacity(), 16); + /// } + /// ``` #[inline] - #[cfg(not(no_global_oom_handling))] // required by tests/run-make/alloc-no-oom-handling - pub(crate) unsafe fn from_nonnull(ptr: NonNull, length: usize, capacity: usize) -> Self { - unsafe { Self::from_nonnull_in(ptr, length, capacity, Global) } + #[unstable(feature = "box_vec_non_null", reason = "new API", issue = "130364")] + pub unsafe fn from_parts(ptr: NonNull, length: usize, capacity: usize) -> Self { + unsafe { Self::from_parts_in(ptr, length, capacity, Global) } } } @@ -694,6 +798,7 @@ impl Vec { #[cfg(not(no_global_oom_handling))] #[inline] #[unstable(feature = "allocator_api", issue = "32838")] + #[track_caller] pub fn with_capacity_in(capacity: usize, alloc: A) -> Self { Vec { buf: RawVec::with_capacity_in(capacity, alloc), len: 0 } } @@ -830,19 +935,119 @@ impl Vec { unsafe { Vec { buf: RawVec::from_raw_parts_in(ptr, capacity, alloc), len: length } } } - /// A convenience method for hoisting the non-null precondition out of [`Vec::from_raw_parts_in`]. + #[doc(alias = "from_non_null_parts_in")] + /// Creates a `Vec` directly from a `NonNull` pointer, a length, a capacity, + /// and an allocator. /// /// # Safety /// - /// See [`Vec::from_raw_parts_in`]. + /// This is highly unsafe, due to the number of invariants that aren't + /// checked: + /// + /// * `ptr` must be [*currently allocated*] via the given allocator `alloc`. + /// * `T` needs to have the same alignment as what `ptr` was allocated with. + /// (`T` having a less strict alignment is not sufficient, the alignment really + /// needs to be equal to satisfy the [`dealloc`] requirement that memory must be + /// allocated and deallocated with the same layout.) + /// * The size of `T` times the `capacity` (ie. the allocated size in bytes) needs + /// to be the same size as the pointer was allocated with. (Because similar to + /// alignment, [`dealloc`] must be called with the same layout `size`.) + /// * `length` needs to be less than or equal to `capacity`. + /// * The first `length` values must be properly initialized values of type `T`. + /// * `capacity` needs to [*fit*] the layout size that the pointer was allocated with. + /// * The allocated size in bytes must be no larger than `isize::MAX`. + /// See the safety documentation of [`pointer::offset`]. + /// + /// These requirements are always upheld by any `ptr` that has been allocated + /// via `Vec`. Other allocation sources are allowed if the invariants are + /// upheld. + /// + /// Violating these may cause problems like corrupting the allocator's + /// internal data structures. For example it is **not** safe + /// to build a `Vec` from a pointer to a C `char` array with length `size_t`. + /// It's also not safe to build one from a `Vec` and its length, because + /// the allocator cares about the alignment, and these two types have different + /// alignments. The buffer was allocated with alignment 2 (for `u16`), but after + /// turning it into a `Vec` it'll be deallocated with alignment 1. + /// + /// The ownership of `ptr` is effectively transferred to the + /// `Vec` which may then deallocate, reallocate or change the + /// contents of memory pointed to by the pointer at will. Ensure + /// that nothing else uses the pointer after calling this + /// function. + /// + /// [`String`]: crate::string::String + /// [`dealloc`]: crate::alloc::GlobalAlloc::dealloc + /// [*currently allocated*]: crate::alloc::Allocator#currently-allocated-memory + /// [*fit*]: crate::alloc::Allocator#memory-fitting + /// + /// # Examples + /// + /// ``` + /// #![feature(allocator_api, box_vec_non_null)] + /// + /// use std::alloc::System; + /// + /// use std::ptr::NonNull; + /// use std::mem; + /// + /// let mut v = Vec::with_capacity_in(3, System); + /// v.push(1); + /// v.push(2); + /// v.push(3); + /// + // FIXME Update this when vec_into_raw_parts is stabilized + /// // Prevent running `v`'s destructor so we are in complete control + /// // of the allocation. + /// let mut v = mem::ManuallyDrop::new(v); + /// + /// // Pull out the various important pieces of information about `v` + /// let p = unsafe { NonNull::new_unchecked(v.as_mut_ptr()) }; + /// let len = v.len(); + /// let cap = v.capacity(); + /// let alloc = v.allocator(); + /// + /// unsafe { + /// // Overwrite memory with 4, 5, 6 + /// for i in 0..len { + /// p.add(i).write(4 + i); + /// } + /// + /// // Put everything back together into a Vec + /// let rebuilt = Vec::from_parts_in(p, len, cap, alloc.clone()); + /// assert_eq!(rebuilt, [4, 5, 6]); + /// } + /// ``` + /// + /// Using memory that was allocated elsewhere: + /// + /// ```rust + /// #![feature(allocator_api, box_vec_non_null)] + /// + /// use std::alloc::{AllocError, Allocator, Global, Layout}; + /// + /// fn main() { + /// let layout = Layout::array::(16).expect("overflow cannot happen"); + /// + /// let vec = unsafe { + /// let mem = match Global.allocate(layout) { + /// Ok(mem) => mem.cast::(), + /// Err(AllocError) => return, + /// }; + /// + /// mem.write(1_000_000); + /// + /// Vec::from_parts_in(mem, 1, 16, Global) + /// }; + /// + /// assert_eq!(vec, &[1_000_000]); + /// assert_eq!(vec.capacity(), 16); + /// } + /// ``` #[inline] - #[cfg(not(no_global_oom_handling))] // required by tests/run-make/alloc-no-oom-handling - pub(crate) unsafe fn from_nonnull_in( - ptr: NonNull, - length: usize, - capacity: usize, - alloc: A, - ) -> Self { + #[unstable(feature = "allocator_api", reason = "new API", issue = "32838")] + // #[unstable(feature = "box_vec_non_null", issue = "130364")] + pub unsafe fn from_parts_in(ptr: NonNull, length: usize, capacity: usize, alloc: A) -> Self { unsafe { Vec { buf: RawVec::from_nonnull_in(ptr, capacity, alloc), len: length } } } @@ -885,6 +1090,49 @@ impl Vec { (me.as_mut_ptr(), me.len(), me.capacity()) } + #[doc(alias = "into_non_null_parts")] + /// Decomposes a `Vec` into its raw components: `(NonNull pointer, length, capacity)`. + /// + /// Returns the `NonNull` pointer to the underlying data, the length of + /// the vector (in elements), and the allocated capacity of the + /// data (in elements). These are the same arguments in the same + /// order as the arguments to [`from_parts`]. + /// + /// After calling this function, the caller is responsible for the + /// memory previously managed by the `Vec`. The only way to do + /// this is to convert the `NonNull` pointer, length, and capacity back + /// into a `Vec` with the [`from_parts`] function, allowing + /// the destructor to perform the cleanup. + /// + /// [`from_parts`]: Vec::from_parts + /// + /// # Examples + /// + /// ``` + /// #![feature(vec_into_raw_parts, box_vec_non_null)] + /// + /// let v: Vec = vec![-1, 0, 1]; + /// + /// let (ptr, len, cap) = v.into_parts(); + /// + /// let rebuilt = unsafe { + /// // We can now make changes to the components, such as + /// // transmuting the raw pointer to a compatible type. + /// let ptr = ptr.cast::(); + /// + /// Vec::from_parts(ptr, len, cap) + /// }; + /// assert_eq!(rebuilt, [4294967295, 0, 1]); + /// ``` + #[must_use = "losing the pointer will leak memory"] + #[unstable(feature = "box_vec_non_null", reason = "new API", issue = "130364")] + // #[unstable(feature = "vec_into_raw_parts", reason = "new API", issue = "65816")] + pub fn into_parts(self) -> (NonNull, usize, usize) { + let (ptr, len, capacity) = self.into_raw_parts(); + // SAFETY: A `Vec` always has a non-null pointer. + (unsafe { NonNull::new_unchecked(ptr) }, len, capacity) + } + /// Decomposes a `Vec` into its raw components: `(pointer, length, capacity, allocator)`. /// /// Returns the raw pointer to the underlying data, the length of the vector (in elements), @@ -934,6 +1182,54 @@ impl Vec { (ptr, len, capacity, alloc) } + #[doc(alias = "into_non_null_parts_with_alloc")] + /// Decomposes a `Vec` into its raw components: `(NonNull pointer, length, capacity, allocator)`. + /// + /// Returns the `NonNull` pointer to the underlying data, the length of the vector (in elements), + /// the allocated capacity of the data (in elements), and the allocator. These are the same + /// arguments in the same order as the arguments to [`from_parts_in`]. + /// + /// After calling this function, the caller is responsible for the + /// memory previously managed by the `Vec`. The only way to do + /// this is to convert the `NonNull` pointer, length, and capacity back + /// into a `Vec` with the [`from_parts_in`] function, allowing + /// the destructor to perform the cleanup. + /// + /// [`from_parts_in`]: Vec::from_parts_in + /// + /// # Examples + /// + /// ``` + /// #![feature(allocator_api, vec_into_raw_parts, box_vec_non_null)] + /// + /// use std::alloc::System; + /// + /// let mut v: Vec = Vec::new_in(System); + /// v.push(-1); + /// v.push(0); + /// v.push(1); + /// + /// let (ptr, len, cap, alloc) = v.into_parts_with_alloc(); + /// + /// let rebuilt = unsafe { + /// // We can now make changes to the components, such as + /// // transmuting the raw pointer to a compatible type. + /// let ptr = ptr.cast::(); + /// + /// Vec::from_parts_in(ptr, len, cap, alloc) + /// }; + /// assert_eq!(rebuilt, [4294967295, 0, 1]); + /// ``` + #[must_use = "losing the pointer will leak memory"] + #[unstable(feature = "allocator_api", issue = "32838")] + // #[unstable(feature = "box_vec_non_null", reason = "new API", issue = "130364")] + // #[unstable(feature = "vec_into_raw_parts", reason = "new API", issue = "65816")] + pub fn into_parts_with_alloc(self) -> (NonNull, usize, usize, A) { + let (ptr, len, capacity, alloc) = self.into_raw_parts_with_alloc(); + // SAFETY: A `Vec` always has a non-null pointer. + (unsafe { NonNull::new_unchecked(ptr) }, len, capacity, alloc) + } + /// Returns the total number of elements the vector can hold without /// reallocating. /// @@ -946,7 +1242,8 @@ impl Vec { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - pub fn capacity(&self) -> usize { + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] + pub const fn capacity(&self) -> usize { self.buf.capacity() } @@ -969,6 +1266,7 @@ impl Vec { /// ``` #[cfg(not(no_global_oom_handling))] #[stable(feature = "rust1", since = "1.0.0")] + #[track_caller] pub fn reserve(&mut self, additional: usize) { self.buf.reserve(self.len, additional); } @@ -999,6 +1297,7 @@ impl Vec { /// ``` #[cfg(not(no_global_oom_handling))] #[stable(feature = "rust1", since = "1.0.0")] + #[track_caller] pub fn reserve_exact(&mut self, additional: usize) { self.buf.reserve_exact(self.len, additional); } @@ -1102,6 +1401,7 @@ impl Vec { /// ``` #[cfg(not(no_global_oom_handling))] #[stable(feature = "rust1", since = "1.0.0")] + #[track_caller] #[inline] pub fn shrink_to_fit(&mut self) { // The capacity is never less than the length, and there's nothing to do when @@ -1132,6 +1432,7 @@ impl Vec { /// ``` #[cfg(not(no_global_oom_handling))] #[stable(feature = "shrink_to", since = "1.56.0")] + #[track_caller] pub fn shrink_to(&mut self, min_capacity: usize) { if self.capacity() > min_capacity { self.buf.shrink_to_fit(cmp::max(self.len, min_capacity)); @@ -1165,6 +1466,7 @@ impl Vec { /// ``` #[cfg(not(no_global_oom_handling))] #[stable(feature = "rust1", since = "1.0.0")] + #[track_caller] pub fn into_boxed_slice(mut self) -> Box<[T], A> { unsafe { self.shrink_to_fit(); @@ -1253,8 +1555,23 @@ impl Vec { /// ``` #[inline] #[stable(feature = "vec_as_slice", since = "1.7.0")] - pub fn as_slice(&self) -> &[T] { - self + #[cfg_attr(not(test), rustc_diagnostic_item = "vec_as_slice")] + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] + pub const fn as_slice(&self) -> &[T] { + // SAFETY: `slice::from_raw_parts` requires pointee is a contiguous, aligned buffer of size + // `len` containing properly-initialized `T`s. Data must not be mutated for the returned + // lifetime. Further, `len * mem::size_of::` <= `ISIZE::MAX`, and allocation does not + // "wrap" through overflowing memory addresses. + // + // * Vec API guarantees that self.buf: + // * contains only properly-initialized items within 0..len + // * is aligned, contiguous, and valid for `len` reads + // * obeys size and address-wrapping constraints + // + // * We only construct `&mut` references to `self.buf` through `&mut self` methods; borrow- + // check ensures that it is not possible to mutably alias `self.buf` within the + // returned lifetime. + unsafe { slice::from_raw_parts(self.as_ptr(), self.len) } } /// Extracts a mutable slice of the entire vector. @@ -1270,8 +1587,23 @@ impl Vec { /// ``` #[inline] #[stable(feature = "vec_as_slice", since = "1.7.0")] - pub fn as_mut_slice(&mut self) -> &mut [T] { - self + #[cfg_attr(not(test), rustc_diagnostic_item = "vec_as_mut_slice")] + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] + pub const fn as_mut_slice(&mut self) -> &mut [T] { + // SAFETY: `slice::from_raw_parts_mut` requires pointee is a contiguous, aligned buffer of + // size `len` containing properly-initialized `T`s. Data must not be accessed through any + // other pointer for the returned lifetime. Further, `len * mem::size_of::` <= + // `ISIZE::MAX` and allocation does not "wrap" through overflowing memory addresses. + // + // * Vec API guarantees that self.buf: + // * contains only properly-initialized items within 0..len + // * is aligned, contiguous, and valid for `len` reads + // * obeys size and address-wrapping constraints + // + // * We only construct references to `self.buf` through `&self` and `&mut self` methods; + // borrow-check ensures that it is not possible to construct a reference to `self.buf` + // within the returned lifetime. + unsafe { slice::from_raw_parts_mut(self.as_mut_ptr(), self.len) } } /// Returns a raw pointer to the vector's buffer, or a dangling raw pointer @@ -1288,7 +1620,8 @@ impl Vec { /// /// This method guarantees that for the purpose of the aliasing model, this method /// does not materialize a reference to the underlying slice, and thus the returned pointer - /// will remain valid when mixed with other calls to [`as_ptr`] and [`as_mut_ptr`]. + /// will remain valid when mixed with other calls to [`as_ptr`], [`as_mut_ptr`], + /// and [`as_non_null`]. /// Note that calling other methods that materialize mutable references to the slice, /// or mutable references to specific elements you are planning on accessing through this pointer, /// as well as writing to those elements, may still invalidate this pointer. @@ -1325,10 +1658,12 @@ impl Vec { /// /// [`as_mut_ptr`]: Vec::as_mut_ptr /// [`as_ptr`]: Vec::as_ptr + /// [`as_non_null`]: Vec::as_non_null #[stable(feature = "vec_as_ptr", since = "1.37.0")] + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] #[rustc_never_returns_null_ptr] #[inline] - pub fn as_ptr(&self) -> *const T { + pub const fn as_ptr(&self) -> *const T { // We shadow the slice method of the same name to avoid going through // `deref`, which creates an intermediate reference. self.buf.ptr() @@ -1344,7 +1679,8 @@ impl Vec { /// /// This method guarantees that for the purpose of the aliasing model, this method /// does not materialize a reference to the underlying slice, and thus the returned pointer - /// will remain valid when mixed with other calls to [`as_ptr`] and [`as_mut_ptr`]. + /// will remain valid when mixed with other calls to [`as_ptr`], [`as_mut_ptr`], + /// and [`as_non_null`]. /// Note that calling other methods that materialize references to the slice, /// or references to specific elements you are planning on accessing through this pointer, /// may still invalidate this pointer. @@ -1384,15 +1720,80 @@ impl Vec { /// /// [`as_mut_ptr`]: Vec::as_mut_ptr /// [`as_ptr`]: Vec::as_ptr + /// [`as_non_null`]: Vec::as_non_null #[stable(feature = "vec_as_ptr", since = "1.37.0")] + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] #[rustc_never_returns_null_ptr] #[inline] - pub fn as_mut_ptr(&mut self) -> *mut T { + pub const fn as_mut_ptr(&mut self) -> *mut T { // We shadow the slice method of the same name to avoid going through // `deref_mut`, which creates an intermediate reference. self.buf.ptr() } + /// Returns a `NonNull` pointer to the vector's buffer, or a dangling + /// `NonNull` pointer valid for zero sized reads if the vector didn't allocate. + /// + /// The caller must ensure that the vector outlives the pointer this + /// function returns, or else it will end up dangling. + /// Modifying the vector may cause its buffer to be reallocated, + /// which would also make any pointers to it invalid. + /// + /// This method guarantees that for the purpose of the aliasing model, this method + /// does not materialize a reference to the underlying slice, and thus the returned pointer + /// will remain valid when mixed with other calls to [`as_ptr`], [`as_mut_ptr`], + /// and [`as_non_null`]. + /// Note that calling other methods that materialize references to the slice, + /// or references to specific elements you are planning on accessing through this pointer, + /// may still invalidate this pointer. + /// See the second example below for how this guarantee can be used. + /// + /// # Examples + /// + /// ``` + /// #![feature(box_vec_non_null)] + /// + /// // Allocate vector big enough for 4 elements. + /// let size = 4; + /// let mut x: Vec = Vec::with_capacity(size); + /// let x_ptr = x.as_non_null(); + /// + /// // Initialize elements via raw pointer writes, then set length. + /// unsafe { + /// for i in 0..size { + /// x_ptr.add(i).write(i as i32); + /// } + /// x.set_len(size); + /// } + /// assert_eq!(&*x, &[0, 1, 2, 3]); + /// ``` + /// + /// Due to the aliasing guarantee, the following code is legal: + /// + /// ```rust + /// #![feature(box_vec_non_null)] + /// + /// unsafe { + /// let mut v = vec![0]; + /// let ptr1 = v.as_non_null(); + /// ptr1.write(1); + /// let ptr2 = v.as_non_null(); + /// ptr2.write(2); + /// // Notably, the write to `ptr2` did *not* invalidate `ptr1`: + /// ptr1.write(3); + /// } + /// ``` + /// + /// [`as_mut_ptr`]: Vec::as_mut_ptr + /// [`as_ptr`]: Vec::as_ptr + /// [`as_non_null`]: Vec::as_non_null + #[unstable(feature = "box_vec_non_null", reason = "new API", issue = "130364")] + #[inline] + pub fn as_non_null(&mut self) -> NonNull { + // SAFETY: A `Vec` always has a non-null pointer. + unsafe { NonNull::new_unchecked(self.as_mut_ptr()) } + } + /// Returns a reference to the underlying allocator. #[unstable(feature = "allocator_api", issue = "32838")] #[inline] @@ -1564,6 +1965,7 @@ impl Vec { /// the insertion index is 0. #[cfg(not(no_global_oom_handling))] #[stable(feature = "rust1", since = "1.0.0")] + #[track_caller] pub fn insert(&mut self, index: usize, element: T) { #[cold] #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))] @@ -2003,6 +2405,7 @@ impl Vec { #[inline] #[stable(feature = "rust1", since = "1.0.0")] #[rustc_confusables("push_back", "put", "append")] + #[track_caller] pub fn push(&mut self, value: T) { // Inform codegen that the length does not change across grow_one(). let len = self.len; @@ -2088,6 +2491,7 @@ impl Vec { /// Takes *O*(1) time. #[inline] #[stable(feature = "rust1", since = "1.0.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "vec_pop")] pub fn pop(&mut self) -> Option { if self.len == 0 { None @@ -2143,6 +2547,7 @@ impl Vec { #[cfg(not(no_global_oom_handling))] #[inline] #[stable(feature = "append", since = "1.4.0")] + #[track_caller] pub fn append(&mut self, other: &mut Self) { unsafe { self.append_elements(other.as_slice() as _); @@ -2153,6 +2558,7 @@ impl Vec { /// Appends elements to `self` from other buffer. #[cfg(not(no_global_oom_handling))] #[inline] + #[track_caller] unsafe fn append_elements(&mut self, other: *const [T]) { let count = unsafe { (*other).len() }; self.reserve(count); @@ -2264,8 +2670,9 @@ impl Vec { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] #[rustc_confusables("length", "size")] - pub fn len(&self) -> usize { + pub const fn len(&self) -> usize { self.len } @@ -2281,7 +2688,9 @@ impl Vec { /// assert!(!v.is_empty()); /// ``` #[stable(feature = "rust1", since = "1.0.0")] - pub fn is_empty(&self) -> bool { + #[cfg_attr(not(test), rustc_diagnostic_item = "vec_is_empty")] + #[rustc_const_unstable(feature = "const_vec_string_slice", issue = "129041")] + pub const fn is_empty(&self) -> bool { self.len() == 0 } @@ -2313,6 +2722,7 @@ impl Vec { #[inline] #[must_use = "use `.truncate()` if you don't need the other half"] #[stable(feature = "split_off", since = "1.4.0")] + #[track_caller] pub fn split_off(&mut self, at: usize) -> Self where A: Clone, @@ -2370,6 +2780,7 @@ impl Vec { /// ``` #[cfg(not(no_global_oom_handling))] #[stable(feature = "vec_resize_with", since = "1.33.0")] + #[track_caller] pub fn resize_with(&mut self, new_len: usize, f: F) where F: FnMut() -> T, @@ -2575,6 +2986,7 @@ impl Vec { /// ``` #[cfg(not(no_global_oom_handling))] #[stable(feature = "vec_resize", since = "1.5.0")] + #[track_caller] pub fn resize(&mut self, new_len: usize, value: T) { let len = self.len(); @@ -2606,6 +3018,7 @@ impl Vec { /// [`extend`]: Vec::extend #[cfg(not(no_global_oom_handling))] #[stable(feature = "vec_extend_from_slice", since = "1.6.0")] + #[track_caller] pub fn extend_from_slice(&mut self, other: &[T]) { self.spec_extend(other.iter()) } @@ -2633,6 +3046,7 @@ impl Vec { /// ``` #[cfg(not(no_global_oom_handling))] #[stable(feature = "vec_extend_from_within", since = "1.53.0")] + #[track_caller] pub fn extend_from_within(&mut self, src: R) where R: RangeBounds, @@ -2693,6 +3107,7 @@ impl Vec<[T; N], A> { impl Vec { #[cfg(not(no_global_oom_handling))] + #[track_caller] /// Extend the vector by `n` clones of value. fn extend_with(&mut self, n: usize, value: T) { self.reserve(n); @@ -2752,6 +3167,8 @@ impl Vec { #[doc(hidden)] #[cfg(not(no_global_oom_handling))] #[stable(feature = "rust1", since = "1.0.0")] +#[cfg_attr(not(test), rustc_diagnostic_item = "vec_from_elem")] +#[track_caller] pub fn from_elem(elem: T, n: usize) -> Vec { ::from_elem(elem, n, Global) } @@ -2759,6 +3176,7 @@ pub fn from_elem(elem: T, n: usize) -> Vec { #[doc(hidden)] #[cfg(not(no_global_oom_handling))] #[unstable(feature = "allocator_api", issue = "32838")] +#[track_caller] pub fn from_elem_in(elem: T, n: usize, alloc: A) -> Vec { ::from_elem(elem, n, alloc) } @@ -2831,7 +3249,7 @@ impl ops::Deref for Vec { #[inline] fn deref(&self) -> &[T] { - unsafe { slice::from_raw_parts(self.as_ptr(), self.len) } + self.as_slice() } } @@ -2839,7 +3257,7 @@ impl ops::Deref for Vec { impl ops::DerefMut for Vec { #[inline] fn deref_mut(&mut self) -> &mut [T] { - unsafe { slice::from_raw_parts_mut(self.as_mut_ptr(), self.len) } + self.as_mut_slice() } } @@ -2850,6 +3268,7 @@ unsafe impl ops::DerefPure for Vec {} #[stable(feature = "rust1", since = "1.0.0")] impl Clone for Vec { #[cfg(not(test))] + #[track_caller] fn clone(&self) -> Self { let alloc = self.allocator().clone(); <[T]>::to_vec_in(&**self, alloc) @@ -2887,6 +3306,7 @@ impl Clone for Vec { /// // And no reallocation occurred /// assert_eq!(yp, y.as_ptr()); /// ``` + #[track_caller] fn clone_from(&mut self, source: &Self) { crate::slice::SpecCloneIntoVec::clone_into(source.as_slice(), self); } @@ -2985,6 +3405,7 @@ impl, A: Allocator> IndexMut for Vec { #[stable(feature = "rust1", since = "1.0.0")] impl FromIterator for Vec { #[inline] + #[track_caller] fn from_iter>(iter: I) -> Vec { >::from_iter(iter.into_iter()) } @@ -3053,16 +3474,19 @@ impl<'a, T, A: Allocator> IntoIterator for &'a mut Vec { #[stable(feature = "rust1", since = "1.0.0")] impl Extend for Vec { #[inline] + #[track_caller] fn extend>(&mut self, iter: I) { >::spec_extend(self, iter.into_iter()) } #[inline] + #[track_caller] fn extend_one(&mut self, item: T) { self.push(item); } #[inline] + #[track_caller] fn extend_reserve(&mut self, additional: usize) { self.reserve(additional); } @@ -3082,6 +3506,7 @@ impl Vec { // leaf method to which various SpecFrom/SpecExtend implementations delegate when // they have no further optimizations to apply #[cfg(not(no_global_oom_handling))] + #[track_caller] fn extend_desugared>(&mut self, mut iterator: I) { // This is the case for a general iterator. // @@ -3109,6 +3534,7 @@ impl Vec { // specific extend for `TrustedLen` iterators, called both by the specializations // and internal places where resolving specialization makes compilation slower #[cfg(not(no_global_oom_handling))] + #[track_caller] fn extend_trusted(&mut self, iterator: impl iter::TrustedLen) { let (low, high) = iterator.size_hint(); if let Some(additional) = high { @@ -3259,16 +3685,19 @@ impl Vec { #[cfg(not(no_global_oom_handling))] #[stable(feature = "extend_ref", since = "1.2.0")] impl<'a, T: Copy + 'a, A: Allocator> Extend<&'a T> for Vec { + #[track_caller] fn extend>(&mut self, iter: I) { self.spec_extend(iter.into_iter()) } #[inline] + #[track_caller] fn extend_one(&mut self, &item: &'a T) { self.push(item); } #[inline] + #[track_caller] fn extend_reserve(&mut self, additional: usize) { self.reserve(additional); } @@ -3379,6 +3808,7 @@ impl From<&[T]> for Vec { /// assert_eq!(Vec::from(&[1, 2, 3][..]), vec![1, 2, 3]); /// ``` #[cfg(not(test))] + #[track_caller] fn from(s: &[T]) -> Vec { s.to_vec() } @@ -3399,6 +3829,7 @@ impl From<&mut [T]> for Vec { /// assert_eq!(Vec::from(&mut [1, 2, 3][..]), vec![1, 2, 3]); /// ``` #[cfg(not(test))] + #[track_caller] fn from(s: &mut [T]) -> Vec { s.to_vec() } @@ -3418,6 +3849,7 @@ impl From<&[T; N]> for Vec { /// ``` /// assert_eq!(Vec::from(&[1, 2, 3]), vec![1, 2, 3]); /// ``` + #[track_caller] fn from(s: &[T; N]) -> Vec { Self::from(s.as_slice()) } @@ -3433,6 +3865,7 @@ impl From<&mut [T; N]> for Vec { /// ``` /// assert_eq!(Vec::from(&mut [1, 2, 3]), vec![1, 2, 3]); /// ``` + #[track_caller] fn from(s: &mut [T; N]) -> Vec { Self::from(s.as_mut_slice()) } @@ -3449,6 +3882,7 @@ impl From<[T; N]> for Vec { /// assert_eq!(Vec::from([1, 2, 3]), vec![1, 2, 3]); /// ``` #[cfg(not(test))] + #[track_caller] fn from(s: [T; N]) -> Vec { <[T]>::into_vec(Box::new(s)) } @@ -3478,6 +3912,7 @@ where /// let b: Cow<'_, [i32]> = Cow::Borrowed(&[1, 2, 3]); /// assert_eq!(Vec::from(o), Vec::from(b)); /// ``` + #[track_caller] fn from(s: Cow<'a, [T]>) -> Vec { s.into_owned() } @@ -3526,6 +3961,7 @@ impl From> for Box<[T], A> { /// /// assert_eq!(Box::from(vec), vec![1, 2, 3].into_boxed_slice()); /// ``` + #[track_caller] fn from(v: Vec) -> Self { v.into_boxed_slice() } @@ -3541,6 +3977,7 @@ impl From<&str> for Vec { /// ``` /// assert_eq!(Vec::from("123"), vec![b'1', b'2', b'3']); /// ``` + #[track_caller] fn from(s: &str) -> Vec { From::from(s.as_bytes()) } diff --git a/library/alloc/src/vec/spec_extend.rs b/library/alloc/src/vec/spec_extend.rs index 7085bceef5baa..b98db669059f9 100644 --- a/library/alloc/src/vec/spec_extend.rs +++ b/library/alloc/src/vec/spec_extend.rs @@ -6,6 +6,7 @@ use crate::alloc::Allocator; // Specialization trait used for Vec::extend pub(super) trait SpecExtend { + #[track_caller] fn spec_extend(&mut self, iter: I); } @@ -13,6 +14,7 @@ impl SpecExtend for Vec where I: Iterator, { + #[track_caller] default fn spec_extend(&mut self, iter: I) { self.extend_desugared(iter) } @@ -22,12 +24,14 @@ impl SpecExtend for Vec where I: TrustedLen, { + #[track_caller] default fn spec_extend(&mut self, iterator: I) { self.extend_trusted(iterator) } } impl SpecExtend> for Vec { + #[track_caller] fn spec_extend(&mut self, mut iterator: IntoIter) { unsafe { self.append_elements(iterator.as_slice() as _); @@ -41,6 +45,7 @@ where I: Iterator, T: Clone, { + #[track_caller] default fn spec_extend(&mut self, iterator: I) { self.spec_extend(iterator.cloned()) } @@ -50,6 +55,7 @@ impl<'a, T: 'a, A: Allocator> SpecExtend<&'a T, slice::Iter<'a, T>> for Vec) { let slice = iterator.as_slice(); unsafe { self.append_elements(slice) }; diff --git a/library/alloc/src/vec/spec_from_elem.rs b/library/alloc/src/vec/spec_from_elem.rs index 96d701e15d487..6c7b4d89f2da7 100644 --- a/library/alloc/src/vec/spec_from_elem.rs +++ b/library/alloc/src/vec/spec_from_elem.rs @@ -10,6 +10,7 @@ pub(super) trait SpecFromElem: Sized { } impl SpecFromElem for T { + #[track_caller] default fn from_elem(elem: Self, n: usize, alloc: A) -> Vec { let mut v = Vec::with_capacity_in(n, alloc); v.extend_with(n, elem); @@ -19,6 +20,7 @@ impl SpecFromElem for T { impl SpecFromElem for T { #[inline] + #[track_caller] default fn from_elem(elem: T, n: usize, alloc: A) -> Vec { if elem.is_zero() { return Vec { buf: RawVec::with_capacity_zeroed_in(n, alloc), len: n }; @@ -31,6 +33,7 @@ impl SpecFromElem for T { impl SpecFromElem for i8 { #[inline] + #[track_caller] fn from_elem(elem: i8, n: usize, alloc: A) -> Vec { if elem == 0 { return Vec { buf: RawVec::with_capacity_zeroed_in(n, alloc), len: n }; @@ -46,6 +49,7 @@ impl SpecFromElem for i8 { impl SpecFromElem for u8 { #[inline] + #[track_caller] fn from_elem(elem: u8, n: usize, alloc: A) -> Vec { if elem == 0 { return Vec { buf: RawVec::with_capacity_zeroed_in(n, alloc), len: n }; diff --git a/library/alloc/src/vec/spec_from_iter.rs b/library/alloc/src/vec/spec_from_iter.rs index 6646ae7bccb7a..ad7688e1c59f0 100644 --- a/library/alloc/src/vec/spec_from_iter.rs +++ b/library/alloc/src/vec/spec_from_iter.rs @@ -29,12 +29,14 @@ impl SpecFromIter for Vec where I: Iterator, { + #[track_caller] default fn from_iter(iterator: I) -> Self { SpecFromIterNested::from_iter(iterator) } } impl SpecFromIter> for Vec { + #[track_caller] fn from_iter(iterator: IntoIter) -> Self { // A common case is passing a vector into a function which immediately // re-collects into a vector. We can short circuit this if the IntoIter @@ -51,7 +53,7 @@ impl SpecFromIter> for Vec { if has_advanced { ptr::copy(it.ptr.as_ptr(), it.buf.as_ptr(), it.len()); } - return Vec::from_nonnull(it.buf, it.len(), it.cap); + return Vec::from_parts(it.buf, it.len(), it.cap); } } diff --git a/library/alloc/src/vec/spec_from_iter_nested.rs b/library/alloc/src/vec/spec_from_iter_nested.rs index 77f7761d22f95..22eed238798cf 100644 --- a/library/alloc/src/vec/spec_from_iter_nested.rs +++ b/library/alloc/src/vec/spec_from_iter_nested.rs @@ -15,6 +15,7 @@ impl SpecFromIterNested for Vec where I: Iterator, { + #[track_caller] default fn from_iter(mut iterator: I) -> Self { // Unroll the first iteration, as the vector is going to be // expanded on this iteration in every case when the iterable is not @@ -47,6 +48,7 @@ impl SpecFromIterNested for Vec where I: TrustedLen, { + #[track_caller] fn from_iter(iterator: I) -> Self { let mut vector = match iterator.size_hint() { (_, Some(upper)) => Vec::with_capacity(upper), diff --git a/library/alloc/src/vec/splice.rs b/library/alloc/src/vec/splice.rs index 9e36377c148d2..ca5cb17f8bfda 100644 --- a/library/alloc/src/vec/splice.rs +++ b/library/alloc/src/vec/splice.rs @@ -52,6 +52,7 @@ impl ExactSizeIterator for Splice<'_, I, A> {} #[stable(feature = "vec_splice", since = "1.21.0")] impl Drop for Splice<'_, I, A> { + #[track_caller] fn drop(&mut self) { self.drain.by_ref().for_each(drop); // At this point draining is done and the only remaining tasks are splicing @@ -123,6 +124,7 @@ impl Drain<'_, T, A> { } /// Makes room for inserting more elements before the tail. + #[track_caller] unsafe fn move_tail(&mut self, additional: usize) { let vec = unsafe { self.vec.as_mut() }; let len = self.tail_start + self.tail_len; diff --git a/library/alloc/tests/arc.rs b/library/alloc/tests/arc.rs index dc27c578b57ef..a259c0131ecdf 100644 --- a/library/alloc/tests/arc.rs +++ b/library/alloc/tests/arc.rs @@ -1,5 +1,5 @@ use std::any::Any; -use std::cell::RefCell; +use std::cell::{Cell, RefCell}; use std::iter::TrustedLen; use std::mem; use std::sync::{Arc, Weak}; @@ -89,7 +89,7 @@ fn eq() { // The test code below is identical to that in `rc.rs`. // For better maintainability we therefore define this type alias. -type Rc = Arc; +type Rc = Arc; const SHARED_ITER_MAX: u16 = 100; @@ -210,6 +210,42 @@ fn weak_may_dangle() { // borrow might be used here, when `val` is dropped and runs the `Drop` code for type `std::sync::Weak` } +/// Test that a panic from a destructor does not leak the allocation. +#[test] +#[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] +fn panic_no_leak() { + use std::alloc::{AllocError, Allocator, Global, Layout}; + use std::panic::{AssertUnwindSafe, catch_unwind}; + use std::ptr::NonNull; + + struct AllocCount(Cell); + unsafe impl Allocator for AllocCount { + fn allocate(&self, layout: Layout) -> Result, AllocError> { + self.0.set(self.0.get() + 1); + Global.allocate(layout) + } + unsafe fn deallocate(&self, ptr: NonNull, layout: Layout) { + self.0.set(self.0.get() - 1); + unsafe { Global.deallocate(ptr, layout) } + } + } + + struct PanicOnDrop; + impl Drop for PanicOnDrop { + fn drop(&mut self) { + panic!("PanicOnDrop"); + } + } + + let alloc = AllocCount(Cell::new(0)); + let rc = Rc::new_in(PanicOnDrop, &alloc); + assert_eq!(alloc.0.get(), 1); + + let panic_message = catch_unwind(AssertUnwindSafe(|| drop(rc))).unwrap_err(); + assert_eq!(*panic_message.downcast_ref::<&'static str>().unwrap(), "PanicOnDrop"); + assert_eq!(alloc.0.get(), 0); +} + /// This is similar to the doc-test for `Arc::make_mut()`, but on an unsized type (slice). #[test] fn make_mut_unsized() { diff --git a/library/alloc/tests/boxed.rs b/library/alloc/tests/boxed.rs index bfc31a626fadd..6a8ba5c92fb30 100644 --- a/library/alloc/tests/boxed.rs +++ b/library/alloc/tests/boxed.rs @@ -4,6 +4,7 @@ use core::mem::MaybeUninit; use core::ptr::NonNull; #[test] +#[cfg_attr(not(bootstrap), expect(dangling_pointers_from_temporaries))] fn uninitialized_zero_size_box() { assert_eq!( &*Box::<()>::new_uninit() as *const _, @@ -59,6 +60,44 @@ fn box_deref_lval() { assert_eq!(x.get(), 1000); } +/// Test that a panic from a destructor does not leak the allocation. +#[test] +#[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] +fn panic_no_leak() { + use std::alloc::{AllocError, Allocator, Global, Layout}; + use std::panic::{AssertUnwindSafe, catch_unwind}; + use std::ptr::NonNull; + + struct AllocCount(Cell); + unsafe impl Allocator for AllocCount { + fn allocate(&self, layout: Layout) -> Result, AllocError> { + self.0.set(self.0.get() + 1); + Global.allocate(layout) + } + unsafe fn deallocate(&self, ptr: NonNull, layout: Layout) { + self.0.set(self.0.get() - 1); + unsafe { Global.deallocate(ptr, layout) } + } + } + + struct PanicOnDrop { + _data: u8, + } + impl Drop for PanicOnDrop { + fn drop(&mut self) { + panic!("PanicOnDrop"); + } + } + + let alloc = AllocCount(Cell::new(0)); + let b = Box::new_in(PanicOnDrop { _data: 42 }, &alloc); + assert_eq!(alloc.0.get(), 1); + + let panic_message = catch_unwind(AssertUnwindSafe(|| drop(b))).unwrap_err(); + assert_eq!(*panic_message.downcast_ref::<&'static str>().unwrap(), "PanicOnDrop"); + assert_eq!(alloc.0.get(), 0); +} + #[allow(unused)] pub struct ConstAllocator; diff --git a/library/alloc/tests/fmt.rs b/library/alloc/tests/fmt.rs index ce24a40f4c051..c13074c53b73d 100644 --- a/library/alloc/tests/fmt.rs +++ b/library/alloc/tests/fmt.rs @@ -1,4 +1,6 @@ #![deny(warnings)] +// FIXME(static_mut_refs): Do not allow `static_mut_refs` lint +#![allow(static_mut_refs)] use std::cell::RefCell; use std::fmt::{self, Write}; diff --git a/library/alloc/tests/lib.rs b/library/alloc/tests/lib.rs index c5c6a122cfec8..699a8e6776e6d 100644 --- a/library/alloc/tests/lib.rs +++ b/library/alloc/tests/lib.rs @@ -4,11 +4,7 @@ #![feature(assert_matches)] #![feature(btree_extract_if)] #![feature(cow_is_borrowed)] -#![feature(const_cow_is_borrowed)] #![feature(const_heap)] -#![feature(const_mut_refs)] -#![feature(const_slice_from_raw_parts_mut)] -#![feature(const_ptr_write)] #![feature(const_try)] #![feature(core_intrinsics)] #![feature(extract_if)] @@ -28,6 +24,7 @@ #![feature(iter_next_chunk)] #![feature(round_char_boundary)] #![feature(slice_partition_dedup)] +#![feature(string_from_utf8_lossy_owned)] #![feature(string_remove_matches)] #![feature(const_btree_len)] #![feature(const_trait_impl)] @@ -35,11 +32,13 @@ #![feature(panic_update_hook)] #![feature(pointer_is_aligned_to)] #![feature(thin_box)] -#![feature(strict_provenance)] +#![cfg_attr(bootstrap, feature(strict_provenance))] +#![cfg_attr(not(bootstrap), feature(strict_provenance_lints))] #![feature(drain_keep_rest)] #![feature(local_waker)] #![feature(vec_pop_if)] #![feature(unique_rc_arc)] +#![feature(macro_metavar_expr_concat)] #![allow(internal_features)] #![deny(fuzzy_provenance_casts)] #![deny(unsafe_op_in_unsafe_fn)] @@ -59,6 +58,7 @@ mod heap; mod linked_list; mod rc; mod slice; +mod sort; mod str; mod string; mod task; diff --git a/library/alloc/tests/rc.rs b/library/alloc/tests/rc.rs index 29dbdcf225eb5..451765d724283 100644 --- a/library/alloc/tests/rc.rs +++ b/library/alloc/tests/rc.rs @@ -1,5 +1,5 @@ use std::any::Any; -use std::cell::RefCell; +use std::cell::{Cell, RefCell}; use std::iter::TrustedLen; use std::mem; use std::rc::{Rc, Weak}; @@ -206,6 +206,42 @@ fn weak_may_dangle() { // borrow might be used here, when `val` is dropped and runs the `Drop` code for type `std::rc::Weak` } +/// Test that a panic from a destructor does not leak the allocation. +#[test] +#[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] +fn panic_no_leak() { + use std::alloc::{AllocError, Allocator, Global, Layout}; + use std::panic::{AssertUnwindSafe, catch_unwind}; + use std::ptr::NonNull; + + struct AllocCount(Cell); + unsafe impl Allocator for AllocCount { + fn allocate(&self, layout: Layout) -> Result, AllocError> { + self.0.set(self.0.get() + 1); + Global.allocate(layout) + } + unsafe fn deallocate(&self, ptr: NonNull, layout: Layout) { + self.0.set(self.0.get() - 1); + unsafe { Global.deallocate(ptr, layout) } + } + } + + struct PanicOnDrop; + impl Drop for PanicOnDrop { + fn drop(&mut self) { + panic!("PanicOnDrop"); + } + } + + let alloc = AllocCount(Cell::new(0)); + let rc = Rc::new_in(PanicOnDrop, &alloc); + assert_eq!(alloc.0.get(), 1); + + let panic_message = catch_unwind(AssertUnwindSafe(|| drop(rc))).unwrap_err(); + assert_eq!(*panic_message.downcast_ref::<&'static str>().unwrap(), "PanicOnDrop"); + assert_eq!(alloc.0.get(), 0); +} + #[allow(unused)] mod pin_coerce_unsized { use alloc::rc::{Rc, UniqueRc}; diff --git a/library/alloc/tests/slice.rs b/library/alloc/tests/slice.rs index df5a8af16fd70..9625e3d2b5e08 100644 --- a/library/alloc/tests/slice.rs +++ b/library/alloc/tests/slice.rs @@ -1417,8 +1417,8 @@ fn test_box_slice_clone() { #[cfg_attr(target_os = "emscripten", ignore)] #[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] fn test_box_slice_clone_panics() { - use std::sync::atomic::{AtomicUsize, Ordering}; use std::sync::Arc; + use std::sync::atomic::{AtomicUsize, Ordering}; struct Canary { count: Arc, diff --git a/library/alloc/tests/sort/ffi_types.rs b/library/alloc/tests/sort/ffi_types.rs new file mode 100644 index 0000000000000..11515ea476971 --- /dev/null +++ b/library/alloc/tests/sort/ffi_types.rs @@ -0,0 +1,82 @@ +use std::cmp::Ordering; + +// Very large stack value. +#[repr(C)] +#[derive(PartialEq, Eq, Debug, Clone)] +pub struct FFIOneKibiByte { + values: [i64; 128], +} + +impl FFIOneKibiByte { + pub fn new(val: i32) -> Self { + let mut values = [0i64; 128]; + let mut val_i64 = val as i64; + + for elem in &mut values { + *elem = val_i64; + val_i64 = std::hint::black_box(val_i64 + 1); + } + Self { values } + } + + fn as_i64(&self) -> i64 { + self.values[11] + self.values[55] + self.values[77] + } +} + +impl PartialOrd for FFIOneKibiByte { + fn partial_cmp(&self, other: &Self) -> Option { + Some(self.cmp(other)) + } +} + +impl Ord for FFIOneKibiByte { + fn cmp(&self, other: &Self) -> Ordering { + self.as_i64().cmp(&other.as_i64()) + } +} + +// 16 byte stack value, with more expensive comparison. +#[repr(C)] +#[derive(PartialEq, Debug, Clone, Copy)] +pub struct F128 { + x: f64, + y: f64, +} + +impl F128 { + pub fn new(val: i32) -> Self { + let val_f = (val as f64) + (i32::MAX as f64) + 10.0; + + let x = val_f + 0.1; + let y = val_f.log(4.1); + + assert!(y < x); + assert!(x.is_normal() && y.is_normal()); + + Self { x, y } + } +} + +// This is kind of hacky, but we know we only have normal comparable floats in there. +impl Eq for F128 {} + +impl PartialOrd for F128 { + fn partial_cmp(&self, other: &Self) -> Option { + Some(self.cmp(other)) + } +} + +// Goal is similar code-gen between Rust and C++ +// - Rust https://godbolt.org/z/3YM3xenPP +// - C++ https://godbolt.org/z/178M6j1zz +impl Ord for F128 { + fn cmp(&self, other: &Self) -> Ordering { + // Simulate expensive comparison function. + let this_div = self.x / self.y; + let other_div = other.x / other.y; + + // SAFETY: We checked in the ctor that both are normal. + unsafe { this_div.partial_cmp(&other_div).unwrap_unchecked() } + } +} diff --git a/library/alloc/tests/sort/known_good_stable_sort.rs b/library/alloc/tests/sort/known_good_stable_sort.rs new file mode 100644 index 0000000000000..f8615435fc2a7 --- /dev/null +++ b/library/alloc/tests/sort/known_good_stable_sort.rs @@ -0,0 +1,192 @@ +// This module implements a known good stable sort implementation that helps provide better error +// messages when the correctness tests fail, we can't use the stdlib sort functions because we are +// testing them for correctness. +// +// Based on https://github.com/voultapher/tiny-sort-rs. + +use alloc::alloc::{Layout, alloc, dealloc}; +use std::{mem, ptr}; + +/// Sort `v` preserving initial order of equal elements. +/// +/// - Guaranteed O(N * log(N)) worst case perf +/// - No adaptiveness +/// - Branch miss-prediction not affected by outcome of comparison function +/// - Uses `v.len()` auxiliary memory. +/// +/// If `T: Ord` does not implement a total order the resulting order is +/// unspecified. All original elements will remain in `v` and any possible modifications via +/// interior mutability will be observable. Same is true if `T: Ord` panics. +/// +/// Panics if allocating the auxiliary memory fails. +#[inline(always)] +pub fn sort(v: &mut [T]) { + stable_sort(v, |a, b| a.lt(b)) +} + +#[inline(always)] +fn stable_sort bool>(v: &mut [T], mut is_less: F) { + if mem::size_of::() == 0 { + return; + } + + let len = v.len(); + + // Inline the check for len < 2. This happens a lot, instrumenting the Rust compiler suggests + // len < 2 accounts for 94% of its calls to `slice::sort`. + if len < 2 { + return; + } + + // SAFETY: We checked that len is > 0 and that T is not a ZST. + unsafe { + mergesort_main(v, &mut is_less); + } +} + +/// The core logic should not be inlined. +/// +/// SAFETY: The caller has to ensure that len is > 0 and that T is not a ZST. +#[inline(never)] +unsafe fn mergesort_main bool>(v: &mut [T], is_less: &mut F) { + // While it would be nice to have a merge implementation that only requires N / 2 auxiliary + // memory. Doing so would make the merge implementation significantly more complex and + + // SAFETY: See function safety description. + let buf = unsafe { BufGuard::new(v.len()) }; + + // SAFETY: `scratch` has space for `v.len()` writes. And does not alias `v`. + unsafe { + mergesort_core(v, buf.buf_ptr.as_ptr(), is_less); + } +} + +/// Tiny recursive top-down merge sort optimized for binary size. It has no adaptiveness whatsoever, +/// no run detection, etc. +/// +/// Buffer as pointed to by `scratch` must have space for `v.len()` writes. And must not alias `v`. +#[inline(always)] +unsafe fn mergesort_core bool>( + v: &mut [T], + scratch_ptr: *mut T, + is_less: &mut F, +) { + let len = v.len(); + + if len > 2 { + // SAFETY: `mid` is guaranteed in-bounds. And caller has to ensure that `scratch_ptr` can + // hold `v.len()` values. + unsafe { + let mid = len / 2; + // Sort the left half recursively. + mergesort_core(v.get_unchecked_mut(..mid), scratch_ptr, is_less); + // Sort the right half recursively. + mergesort_core(v.get_unchecked_mut(mid..), scratch_ptr, is_less); + // Combine the two halves. + merge(v, scratch_ptr, is_less, mid); + } + } else if len == 2 { + if is_less(&v[1], &v[0]) { + v.swap(0, 1); + } + } +} + +/// Branchless merge function. +/// +/// SAFETY: The caller must ensure that `scratch_ptr` is valid for `v.len()` writes. And that mid is +/// in-bounds. +#[inline(always)] +unsafe fn merge(v: &mut [T], scratch_ptr: *mut T, is_less: &mut F, mid: usize) +where + F: FnMut(&T, &T) -> bool, +{ + let len = v.len(); + debug_assert!(mid > 0 && mid < len); + + let len = v.len(); + + // Indexes to track the positions while merging. + let mut l = 0; + let mut r = mid; + + // SAFETY: No matter what the result of is_less is we check that l and r remain in-bounds and if + // is_less panics the original elements remain in `v`. + unsafe { + let arr_ptr = v.as_ptr(); + + for i in 0..len { + let left_ptr = arr_ptr.add(l); + let right_ptr = arr_ptr.add(r); + + let is_lt = !is_less(&*right_ptr, &*left_ptr); + let copy_ptr = if is_lt { left_ptr } else { right_ptr }; + ptr::copy_nonoverlapping(copy_ptr, scratch_ptr.add(i), 1); + + l += is_lt as usize; + r += !is_lt as usize; + + // As long as neither side is exhausted merge left and right elements. + if ((l == mid) as u8 + (r == len) as u8) != 0 { + break; + } + } + + // The left or right side is exhausted, drain the right side in one go. + let copy_ptr = if l == mid { arr_ptr.add(r) } else { arr_ptr.add(l) }; + let i = l + (r - mid); + ptr::copy_nonoverlapping(copy_ptr, scratch_ptr.add(i), len - i); + + // Now that scratch_ptr holds the full merged content, write it back on-top of v. + ptr::copy_nonoverlapping(scratch_ptr, v.as_mut_ptr(), len); + } +} + +// SAFETY: The caller has to ensure that Option is Some, UB otherwise. +unsafe fn unwrap_unchecked(opt_val: Option) -> T { + match opt_val { + Some(val) => val, + None => { + // SAFETY: See function safety description. + unsafe { + core::hint::unreachable_unchecked(); + } + } + } +} + +// Extremely basic versions of Vec. +// Their use is super limited and by having the code here, it allows reuse between the sort +// implementations. +struct BufGuard { + buf_ptr: ptr::NonNull, + capacity: usize, +} + +impl BufGuard { + // SAFETY: The caller has to ensure that len is not 0 and that T is not a ZST. + unsafe fn new(len: usize) -> Self { + debug_assert!(len > 0 && mem::size_of::() > 0); + + // SAFETY: See function safety description. + let layout = unsafe { unwrap_unchecked(Layout::array::(len).ok()) }; + + // SAFETY: We checked that T is not a ZST. + let buf_ptr = unsafe { alloc(layout) as *mut T }; + + if buf_ptr.is_null() { + panic!("allocation failure"); + } + + Self { buf_ptr: ptr::NonNull::new(buf_ptr).unwrap(), capacity: len } + } +} + +impl Drop for BufGuard { + fn drop(&mut self) { + // SAFETY: We checked that T is not a ZST. + unsafe { + dealloc(self.buf_ptr.as_ptr() as *mut u8, Layout::array::(self.capacity).unwrap()); + } + } +} diff --git a/library/alloc/tests/sort/mod.rs b/library/alloc/tests/sort/mod.rs new file mode 100644 index 0000000000000..0e2494ca9d34e --- /dev/null +++ b/library/alloc/tests/sort/mod.rs @@ -0,0 +1,17 @@ +pub trait Sort { + fn name() -> String; + + fn sort(v: &mut [T]) + where + T: Ord; + + fn sort_by(v: &mut [T], compare: F) + where + F: FnMut(&T, &T) -> std::cmp::Ordering; +} + +mod ffi_types; +mod known_good_stable_sort; +mod patterns; +mod tests; +mod zipf; diff --git a/library/alloc/tests/sort/patterns.rs b/library/alloc/tests/sort/patterns.rs new file mode 100644 index 0000000000000..e5d31d868b251 --- /dev/null +++ b/library/alloc/tests/sort/patterns.rs @@ -0,0 +1,211 @@ +use std::env; +use std::hash::Hash; +use std::str::FromStr; +use std::sync::OnceLock; + +use rand::prelude::*; +use rand_xorshift::XorShiftRng; + +use crate::sort::zipf::ZipfDistribution; + +/// Provides a set of patterns useful for testing and benchmarking sorting algorithms. +/// Currently limited to i32 values. + +// --- Public --- + +pub fn random(len: usize) -> Vec { + // . + // : . : : + // :.:::.:: + + random_vec(len) +} + +pub fn random_uniform(len: usize, range: R) -> Vec +where + R: Into> + Hash, +{ + // :.:.:.:: + + let mut rng: XorShiftRng = rand::SeedableRng::seed_from_u64(get_or_init_rand_seed()); + + // Abstracting over ranges in Rust :( + let dist: rand::distributions::Uniform = range.into(); + (0..len).map(|_| dist.sample(&mut rng)).collect() +} + +pub fn random_zipf(len: usize, exponent: f64) -> Vec { + // https://en.wikipedia.org/wiki/Zipf's_law + + let mut rng: XorShiftRng = rand::SeedableRng::seed_from_u64(get_or_init_rand_seed()); + + // Abstracting over ranges in Rust :( + let dist = ZipfDistribution::new(len, exponent).unwrap(); + (0..len).map(|_| dist.sample(&mut rng) as i32).collect() +} + +pub fn random_sorted(len: usize, sorted_percent: f64) -> Vec { + // .: + // .:::. : + // .::::::.:: + // [----][--] + // ^ ^ + // | | + // sorted | + // unsorted + + // Simulate pre-existing sorted slice, where len - sorted_percent are the new unsorted values + // and part of the overall distribution. + let mut v = random_vec(len); + let sorted_len = ((len as f64) * (sorted_percent / 100.0)).round() as usize; + + v[0..sorted_len].sort_unstable(); + + v +} + +pub fn all_equal(len: usize) -> Vec { + // ...... + // :::::: + + (0..len).map(|_| 66).collect::>() +} + +pub fn ascending(len: usize) -> Vec { + // .: + // .::: + // .::::: + + (0..len as i32).collect::>() +} + +pub fn descending(len: usize) -> Vec { + // :. + // :::. + // :::::. + + (0..len as i32).rev().collect::>() +} + +pub fn saw_mixed(len: usize, saw_count: usize) -> Vec { + // :. :. .::. .: + // :::.:::..::::::..::: + + if len == 0 { + return Vec::new(); + } + + let mut vals = random_vec(len); + let chunks_size = len / saw_count.max(1); + let saw_directions = random_uniform((len / chunks_size) + 1, 0..=1); + + for (i, chunk) in vals.chunks_mut(chunks_size).enumerate() { + if saw_directions[i] == 0 { + chunk.sort_unstable(); + } else if saw_directions[i] == 1 { + chunk.sort_unstable_by_key(|&e| std::cmp::Reverse(e)); + } else { + unreachable!(); + } + } + + vals +} + +pub fn saw_mixed_range(len: usize, range: std::ops::Range) -> Vec { + // :. + // :. :::. .::. .: + // :::.:::::..::::::..:.::: + + // ascending and descending randomly picked, with length in `range`. + + if len == 0 { + return Vec::new(); + } + + let mut vals = random_vec(len); + + let max_chunks = len / range.start; + let saw_directions = random_uniform(max_chunks + 1, 0..=1); + let chunk_sizes = random_uniform(max_chunks + 1, (range.start as i32)..(range.end as i32)); + + let mut i = 0; + let mut l = 0; + while l < len { + let chunk_size = chunk_sizes[i] as usize; + let chunk_end = std::cmp::min(l + chunk_size, len); + let chunk = &mut vals[l..chunk_end]; + + if saw_directions[i] == 0 { + chunk.sort_unstable(); + } else if saw_directions[i] == 1 { + chunk.sort_unstable_by_key(|&e| std::cmp::Reverse(e)); + } else { + unreachable!(); + } + + i += 1; + l += chunk_size; + } + + vals +} + +pub fn pipe_organ(len: usize) -> Vec { + // .:. + // .:::::. + + let mut vals = random_vec(len); + + let first_half = &mut vals[0..(len / 2)]; + first_half.sort_unstable(); + + let second_half = &mut vals[(len / 2)..len]; + second_half.sort_unstable_by_key(|&e| std::cmp::Reverse(e)); + + vals +} + +pub fn get_or_init_rand_seed() -> u64 { + *SEED_VALUE.get_or_init(|| { + env::var("OVERRIDE_SEED") + .ok() + .map(|seed| u64::from_str(&seed).unwrap()) + .unwrap_or_else(rand_root_seed) + }) +} + +// --- Private --- + +static SEED_VALUE: OnceLock = OnceLock::new(); + +#[cfg(not(miri))] +fn rand_root_seed() -> u64 { + // Other test code hashes `panic::Location::caller()` and constructs a seed from that, in these + // tests we want to have a fuzzer like exploration of the test space, if we used the same caller + // based construction we would always test the same. + // + // Instead we use the seconds since UNIX epoch / 10, given CI log output this value should be + // reasonably easy to re-construct. + + use std::time::{SystemTime, UNIX_EPOCH}; + + let epoch_seconds = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_secs(); + + epoch_seconds / 10 +} + +#[cfg(miri)] +fn rand_root_seed() -> u64 { + // Miri is usually run with isolation with gives us repeatability but also permutations based on + // other code that runs before. + use core::hash::{BuildHasher, Hash, Hasher}; + let mut hasher = std::hash::RandomState::new().build_hasher(); + core::panic::Location::caller().hash(&mut hasher); + hasher.finish() +} + +fn random_vec(len: usize) -> Vec { + let mut rng: XorShiftRng = rand::SeedableRng::seed_from_u64(get_or_init_rand_seed()); + (0..len).map(|_| rng.gen::()).collect() +} diff --git a/library/alloc/tests/sort/tests.rs b/library/alloc/tests/sort/tests.rs new file mode 100644 index 0000000000000..14e6013f965d8 --- /dev/null +++ b/library/alloc/tests/sort/tests.rs @@ -0,0 +1,1233 @@ +use std::cell::Cell; +use std::cmp::Ordering; +use std::fmt::Debug; +use std::panic::{self, AssertUnwindSafe}; +use std::rc::Rc; +use std::{env, fs}; + +use crate::sort::ffi_types::{F128, FFIOneKibiByte}; +use crate::sort::{Sort, known_good_stable_sort, patterns}; + +#[cfg(miri)] +const TEST_LENGTHS: &[usize] = &[2, 3, 4, 7, 10, 15, 20, 24, 33, 50, 100, 171, 300]; + +#[cfg(not(miri))] +const TEST_LENGTHS: &[usize] = &[ + 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 16, 17, 20, 24, 30, 32, 33, 35, 50, 100, 200, 500, 1_000, + 2_048, 5_000, 10_000, 100_000, 1_100_000, +]; + +fn check_is_sorted(v: &mut [T]) { + let seed = patterns::get_or_init_rand_seed(); + + let is_small_test = v.len() <= 100; + let v_orig = v.to_vec(); + + ::sort(v); + + assert_eq!(v.len(), v_orig.len()); + + for window in v.windows(2) { + if window[0] > window[1] { + let mut known_good_sorted_vec = v_orig.clone(); + known_good_stable_sort::sort(known_good_sorted_vec.as_mut_slice()); + + if is_small_test { + eprintln!("Orginal: {:?}", v_orig); + eprintln!("Expected: {:?}", known_good_sorted_vec); + eprintln!("Got: {:?}", v); + } else { + if env::var("WRITE_LARGE_FAILURE").is_ok() { + // Large arrays output them as files. + let original_name = format!("original_{}.txt", seed); + let std_name = format!("known_good_sorted_{}.txt", seed); + let testsort_name = format!("{}_sorted_{}.txt", S::name(), seed); + + fs::write(&original_name, format!("{:?}", v_orig)).unwrap(); + fs::write(&std_name, format!("{:?}", known_good_sorted_vec)).unwrap(); + fs::write(&testsort_name, format!("{:?}", v)).unwrap(); + + eprintln!( + "Failed comparison, see files {original_name}, {std_name}, and {testsort_name}" + ); + } else { + eprintln!( + "Failed comparison, re-run with WRITE_LARGE_FAILURE env var set, to get output." + ); + } + } + + panic!("Test assertion failed!") + } + } +} + +fn test_is_sorted( + test_len: usize, + map_fn: impl Fn(i32) -> T, + pattern_fn: impl Fn(usize) -> Vec, +) { + let mut test_data: Vec = pattern_fn(test_len).into_iter().map(map_fn).collect(); + check_is_sorted::(test_data.as_mut_slice()); +} + +trait DynTrait: Debug { + fn get_val(&self) -> i32; +} + +#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord)] +struct DynValA { + value: i32, +} + +#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord)] +struct DynValB { + value: u64, +} + +impl DynTrait for DynValA { + fn get_val(&self) -> i32 { + self.value + } +} +impl DynTrait for DynValB { + fn get_val(&self) -> i32 { + let bytes = self.value.to_ne_bytes(); + i32::from_ne_bytes([bytes[0], bytes[1], bytes[6], bytes[7]]) + } +} + +impl PartialOrd for dyn DynTrait { + fn partial_cmp(&self, other: &Self) -> Option { + Some(self.cmp(other)) + } +} + +impl Ord for dyn DynTrait { + fn cmp(&self, other: &Self) -> Ordering { + self.get_val().cmp(&other.get_val()) + } +} + +impl PartialEq for dyn DynTrait { + fn eq(&self, other: &Self) -> bool { + self.get_val() == other.get_val() + } +} + +impl Eq for dyn DynTrait {} + +fn shift_i32_to_u32(val: i32) -> u32 { + (val as i64 + (i32::MAX as i64 + 1)) as u32 +} + +fn reverse_shift_i32_to_u32(val: u32) -> i32 { + (val as i64 - (i32::MAX as i64 + 1)) as i32 +} + +fn extend_i32_to_u64(val: i32) -> u64 { + // Extends the value into the 64 bit range, + // while preserving input order. + (shift_i32_to_u32(val) as u64) * i32::MAX as u64 +} + +fn extend_i32_to_u128(val: i32) -> u128 { + // Extends the value into the 64 bit range, + // while preserving input order. + (shift_i32_to_u32(val) as u128) * i64::MAX as u128 +} + +fn dyn_trait_from_i32(val: i32) -> Rc { + if val % 2 == 0 { + Rc::new(DynValA { value: val }) + } else { + Rc::new(DynValB { value: extend_i32_to_u64(val) }) + } +} + +fn i32_from_i32(val: i32) -> i32 { + val +} + +fn i32_from_i32_ref(val: &i32) -> i32 { + *val +} + +fn string_from_i32(val: i32) -> String { + format!("{:010}", shift_i32_to_u32(val)) +} + +fn i32_from_string(val: &String) -> i32 { + reverse_shift_i32_to_u32(val.parse::().unwrap()) +} + +fn cell_i32_from_i32(val: i32) -> Cell { + Cell::new(val) +} + +fn i32_from_cell_i32(val: &Cell) -> i32 { + val.get() +} + +fn calc_comps_required(v: &mut [T], mut cmp_fn: impl FnMut(&T, &T) -> Ordering) -> u32 { + let mut comp_counter = 0u32; + + ::sort_by(v, |a, b| { + comp_counter += 1; + + cmp_fn(a, b) + }); + + comp_counter +} + +#[derive(PartialEq, Eq, Debug, Clone)] +#[repr(C)] +struct CompCount { + val: i32, + comp_count: Cell, +} + +impl CompCount { + fn new(val: i32) -> Self { + Self { val, comp_count: Cell::new(0) } + } +} + +/// Generates $base_name_pattern_name_impl functions calling the test_fns for all test_len. +macro_rules! gen_sort_test_fns { + ( + $base_name:ident, + $test_fn:expr, + $test_lengths:expr, + [$(($pattern_name:ident, $pattern_fn:expr)),* $(,)?] $(,)? + ) => { + $(fn ${concat($base_name, _, $pattern_name, _impl)}() { + for test_len in $test_lengths { + $test_fn(*test_len, $pattern_fn); + } + })* + }; +} + +/// Generates $base_name_pattern_name_impl functions calling the test_fns for all test_len, +/// with a default set of patterns that can be extended by the caller. +macro_rules! gen_sort_test_fns_with_default_patterns { + ( + $base_name:ident, + $test_fn:expr, + $test_lengths:expr, + [$(($pattern_name:ident, $pattern_fn:expr)),* $(,)?] $(,)? + ) => { + gen_sort_test_fns!( + $base_name, + $test_fn, + $test_lengths, + [ + (random, patterns::random), + (random_z1, |len| patterns::random_zipf(len, 1.0)), + (random_d2, |len| patterns::random_uniform(len, 0..2)), + (random_d20, |len| patterns::random_uniform(len, 0..16)), + (random_s95, |len| patterns::random_sorted(len, 95.0)), + (ascending, patterns::ascending), + (descending, patterns::descending), + (saw_mixed, |len| patterns::saw_mixed( + len, + ((len as f64).log2().round()) as usize + )), + $(($pattern_name, $pattern_fn),)* + ] + ); + }; +} + +/// Generates $base_name_type_pattern_name_impl functions calling the test_fns for all test_len for +/// three types that cover the core specialization differences in the sort implementations, with a +/// default set of patterns that can be extended by the caller. +macro_rules! gen_sort_test_fns_with_default_patterns_3_ty { + ( + $base_name:ident, + $test_fn:ident, + [$(($pattern_name:ident, $pattern_fn:expr)),* $(,)?] $(,)? + ) => { + gen_sort_test_fns_with_default_patterns!( + ${concat($base_name, _i32)}, + |len, pattern_fn| $test_fn::(len, i32_from_i32, i32_from_i32_ref, pattern_fn), + &TEST_LENGTHS[..TEST_LENGTHS.len() - 2], + [$(($pattern_name, $pattern_fn),)*], + ); + + gen_sort_test_fns_with_default_patterns!( + ${concat($base_name, _cell_i32)}, + |len, pattern_fn| $test_fn::, S>(len, cell_i32_from_i32, i32_from_cell_i32, pattern_fn), + &TEST_LENGTHS[..TEST_LENGTHS.len() - 3], + [$(($pattern_name, $pattern_fn),)*], + ); + + gen_sort_test_fns_with_default_patterns!( + ${concat($base_name, _string)}, + |len, pattern_fn| $test_fn::(len, string_from_i32, i32_from_string, pattern_fn), + &TEST_LENGTHS[..TEST_LENGTHS.len() - 3], + [$(($pattern_name, $pattern_fn),)*], + ); + }; +} + +// --- TESTS --- + +pub fn basic_impl() { + check_is_sorted::(&mut []); + check_is_sorted::<(), S>(&mut []); + check_is_sorted::<(), S>(&mut [()]); + check_is_sorted::<(), S>(&mut [(), ()]); + check_is_sorted::<(), S>(&mut [(), (), ()]); + check_is_sorted::(&mut []); + check_is_sorted::(&mut [77]); + check_is_sorted::(&mut [2, 3]); + check_is_sorted::(&mut [2, 3, 6]); + check_is_sorted::(&mut [2, 3, 99, 6]); + check_is_sorted::(&mut [2, 7709, 400, 90932]); + check_is_sorted::(&mut [15, -1, 3, -1, -3, -1, 7]); +} + +fn fixed_seed_impl() { + let fixed_seed_a = patterns::get_or_init_rand_seed(); + let fixed_seed_b = patterns::get_or_init_rand_seed(); + + assert_eq!(fixed_seed_a, fixed_seed_b); +} + +fn fixed_seed_rand_vec_prefix_impl() { + let vec_rand_len_5 = patterns::random(5); + let vec_rand_len_7 = patterns::random(7); + + assert_eq!(vec_rand_len_5, vec_rand_len_7[..5]); +} + +fn int_edge_impl() { + // Ensure that the sort can handle integer edge cases. + check_is_sorted::(&mut [i32::MIN, i32::MAX]); + check_is_sorted::(&mut [i32::MAX, i32::MIN]); + check_is_sorted::(&mut [i32::MIN, 3]); + check_is_sorted::(&mut [i32::MIN, -3]); + check_is_sorted::(&mut [i32::MIN, -3, i32::MAX]); + check_is_sorted::(&mut [i32::MIN, -3, i32::MAX, i32::MIN, 5]); + check_is_sorted::(&mut [i32::MAX, 3, i32::MIN, 5, i32::MIN, -3, 60, 200, 50, 7, 10]); + + check_is_sorted::(&mut [u64::MIN, u64::MAX]); + check_is_sorted::(&mut [u64::MAX, u64::MIN]); + check_is_sorted::(&mut [u64::MIN, 3]); + check_is_sorted::(&mut [u64::MIN, u64::MAX - 3]); + check_is_sorted::(&mut [u64::MIN, u64::MAX - 3, u64::MAX]); + check_is_sorted::(&mut [u64::MIN, u64::MAX - 3, u64::MAX, u64::MIN, 5]); + check_is_sorted::(&mut [ + u64::MAX, + 3, + u64::MIN, + 5, + u64::MIN, + u64::MAX - 3, + 60, + 200, + 50, + 7, + 10, + ]); + + let mut large = patterns::random(TEST_LENGTHS[TEST_LENGTHS.len() - 2]); + large.push(i32::MAX); + large.push(i32::MIN); + large.push(i32::MAX); + check_is_sorted::(&mut large); +} + +fn sort_vs_sort_by_impl() { + // Ensure that sort and sort_by produce the same result. + let mut input_normal = [800, 3, -801, 5, -801, -3, 60, 200, 50, 7, 10]; + let expected = [-801, -801, -3, 3, 5, 7, 10, 50, 60, 200, 800]; + + let mut input_sort_by = input_normal.to_vec(); + + ::sort(&mut input_normal); + ::sort_by(&mut input_sort_by, |a, b| a.cmp(b)); + + assert_eq!(input_normal, expected); + assert_eq!(input_sort_by, expected); +} + +gen_sort_test_fns_with_default_patterns!( + correct_i32, + |len, pattern_fn| test_is_sorted::(len, |val| val, pattern_fn), + TEST_LENGTHS, + [ + (random_d4, |len| patterns::random_uniform(len, 0..4)), + (random_d8, |len| patterns::random_uniform(len, 0..8)), + (random_d311, |len| patterns::random_uniform(len, 0..311)), + (random_d1024, |len| patterns::random_uniform(len, 0..1024)), + (random_z1_03, |len| patterns::random_zipf(len, 1.03)), + (random_z2, |len| patterns::random_zipf(len, 2.0)), + (random_s50, |len| patterns::random_sorted(len, 50.0)), + (narrow, |len| patterns::random_uniform( + len, + 0..=(((len as f64).log2().round()) as i32) * 100 + )), + (all_equal, patterns::all_equal), + (saw_mixed_range, |len| patterns::saw_mixed_range(len, 20..50)), + (pipe_organ, patterns::pipe_organ), + ] +); + +gen_sort_test_fns_with_default_patterns!( + correct_u64, + |len, pattern_fn| test_is_sorted::(len, extend_i32_to_u64, pattern_fn), + TEST_LENGTHS, + [] +); + +gen_sort_test_fns_with_default_patterns!( + correct_u128, + |len, pattern_fn| test_is_sorted::(len, extend_i32_to_u128, pattern_fn), + &TEST_LENGTHS[..TEST_LENGTHS.len() - 2], + [] +); + +gen_sort_test_fns_with_default_patterns!( + correct_cell_i32, + |len, pattern_fn| test_is_sorted::, S>(len, Cell::new, pattern_fn), + &TEST_LENGTHS[..TEST_LENGTHS.len() - 2], + [] +); + +gen_sort_test_fns_with_default_patterns!( + correct_string, + |len, pattern_fn| test_is_sorted::( + len, + |val| format!("{:010}", shift_i32_to_u32(val)), + pattern_fn + ), + &TEST_LENGTHS[..TEST_LENGTHS.len() - 2], + [] +); + +gen_sort_test_fns_with_default_patterns!( + correct_f128, + |len, pattern_fn| test_is_sorted::(len, F128::new, pattern_fn), + &TEST_LENGTHS[..TEST_LENGTHS.len() - 2], + [] +); + +gen_sort_test_fns_with_default_patterns!( + correct_1k, + |len, pattern_fn| test_is_sorted::(len, FFIOneKibiByte::new, pattern_fn), + &TEST_LENGTHS[..TEST_LENGTHS.len() - 2], + [] +); + +// Dyn values are fat pointers, something the implementation might have overlooked. +gen_sort_test_fns_with_default_patterns!( + correct_dyn_val, + |len, pattern_fn| test_is_sorted::, S>(len, dyn_trait_from_i32, pattern_fn), + &TEST_LENGTHS[..TEST_LENGTHS.len() - 2], + [] +); + +fn stability_legacy_impl() { + // This non pattern variant has proven to catch some bugs the pattern version of this function + // doesn't catch, so it remains in conjunction with the other one. + + if ::name().contains("unstable") { + // It would be great to mark the test as skipped, but that isn't possible as of now. + return; + } + + let large_range = if cfg!(miri) { 100..110 } else { 3000..3010 }; + let rounds = if cfg!(miri) { 1 } else { 10 }; + + let rand_vals = patterns::random_uniform(5_000, 0..=9); + let mut rand_idx = 0; + + for len in (2..55).chain(large_range) { + for _ in 0..rounds { + let mut counts = [0; 10]; + + // create a vector like [(6, 1), (5, 1), (6, 2), ...], + // where the first item of each tuple is random, but + // the second item represents which occurrence of that + // number this element is, i.e., the second elements + // will occur in sorted order. + let orig: Vec<_> = (0..len) + .map(|_| { + let n = rand_vals[rand_idx]; + rand_idx += 1; + if rand_idx >= rand_vals.len() { + rand_idx = 0; + } + + counts[n as usize] += 1; + i32_tup_as_u64((n, counts[n as usize])) + }) + .collect(); + + let mut v = orig.clone(); + // Only sort on the first element, so an unstable sort + // may mix up the counts. + ::sort_by(&mut v, |a_packed, b_packed| { + let a = i32_tup_from_u64(*a_packed).0; + let b = i32_tup_from_u64(*b_packed).0; + + a.cmp(&b) + }); + + // This comparison includes the count (the second item + // of the tuple), so elements with equal first items + // will need to be ordered with increasing + // counts... i.e., exactly asserting that this sort is + // stable. + assert!(v.windows(2).all(|w| i32_tup_from_u64(w[0]) <= i32_tup_from_u64(w[1]))); + } + } + + // For cpp_sorts that only support u64 we can pack the two i32 inside a u64. + fn i32_tup_as_u64(val: (i32, i32)) -> u64 { + let a_bytes = val.0.to_le_bytes(); + let b_bytes = val.1.to_le_bytes(); + + u64::from_le_bytes([a_bytes, b_bytes].concat().try_into().unwrap()) + } + + fn i32_tup_from_u64(val: u64) -> (i32, i32) { + let bytes = val.to_le_bytes(); + + let a = i32::from_le_bytes(bytes[0..4].try_into().unwrap()); + let b = i32::from_le_bytes(bytes[4..8].try_into().unwrap()); + + (a, b) + } +} + +fn stability_with_patterns( + len: usize, + type_into_fn: impl Fn(i32) -> T, + _type_from_fn: impl Fn(&T) -> i32, + pattern_fn: fn(usize) -> Vec, +) { + if ::name().contains("unstable") { + // It would be great to mark the test as skipped, but that isn't possible as of now. + return; + } + + let pattern = pattern_fn(len); + + let mut counts = [0i32; 128]; + + // create a vector like [(6, 1), (5, 1), (6, 2), ...], + // where the first item of each tuple is random, but + // the second item represents which occurrence of that + // number this element is, i.e., the second elements + // will occur in sorted order. + let orig: Vec<_> = pattern + .iter() + .map(|val| { + let n = val.saturating_abs() % counts.len() as i32; + counts[n as usize] += 1; + (type_into_fn(n), counts[n as usize]) + }) + .collect(); + + let mut v = orig.clone(); + // Only sort on the first element, so an unstable sort + // may mix up the counts. + ::sort(&mut v); + + // This comparison includes the count (the second item + // of the tuple), so elements with equal first items + // will need to be ordered with increasing + // counts... i.e., exactly asserting that this sort is + // stable. + assert!(v.windows(2).all(|w| w[0] <= w[1])); +} + +gen_sort_test_fns_with_default_patterns_3_ty!(stability, stability_with_patterns, []); + +fn observable_is_less(len: usize, pattern_fn: fn(usize) -> Vec) { + // This test, tests that every is_less is actually observable. Ie. this can go wrong if a hole + // is created using temporary memory and, the whole is used as comparison but not copied back. + // + // If this is not upheld a custom type + comparison function could yield UB in otherwise safe + // code. Eg T == Mutex>> which replaces the pointer with none in the comparison + // function, which would not be observed in the original slice and would lead to a double free. + + let pattern = pattern_fn(len); + let mut test_input = pattern.into_iter().map(|val| CompCount::new(val)).collect::>(); + + let mut comp_count_global = 0; + + ::sort_by(&mut test_input, |a, b| { + a.comp_count.replace(a.comp_count.get() + 1); + b.comp_count.replace(b.comp_count.get() + 1); + comp_count_global += 1; + + a.val.cmp(&b.val) + }); + + let total_inner: u64 = test_input.iter().map(|c| c.comp_count.get() as u64).sum(); + + assert_eq!(total_inner, comp_count_global * 2); +} + +gen_sort_test_fns_with_default_patterns!( + observable_is_less, + observable_is_less::, + &TEST_LENGTHS[..TEST_LENGTHS.len() - 2], + [] +); + +fn panic_retain_orig_set( + len: usize, + type_into_fn: impl Fn(i32) -> T + Copy, + type_from_fn: impl Fn(&T) -> i32, + pattern_fn: fn(usize) -> Vec, +) { + let mut test_data: Vec = pattern_fn(len).into_iter().map(type_into_fn).collect(); + + let sum_before: i64 = test_data.iter().map(|x| type_from_fn(x) as i64).sum(); + + // Calculate a specific comparison that should panic. + // Ensure that it can be any of the possible comparisons and that it always panics. + let required_comps = calc_comps_required::(&mut test_data.clone(), |a, b| a.cmp(b)); + let panic_threshold = patterns::random_uniform(1, 1..=required_comps as i32)[0] as usize - 1; + + let mut comp_counter = 0; + + let res = panic::catch_unwind(AssertUnwindSafe(|| { + ::sort_by(&mut test_data, |a, b| { + if comp_counter == panic_threshold { + // Make the panic dependent on the test len and some random factor. We want to + // make sure that panicking may also happen when comparing elements a second + // time. + panic!(); + } + comp_counter += 1; + + a.cmp(b) + }); + })); + + assert!(res.is_err()); + + // If the sum before and after don't match, it means the set of elements hasn't remained the + // same. + let sum_after: i64 = test_data.iter().map(|x| type_from_fn(x) as i64).sum(); + assert_eq!(sum_before, sum_after); +} + +gen_sort_test_fns_with_default_patterns_3_ty!(panic_retain_orig_set, panic_retain_orig_set, []); + +fn panic_observable_is_less(len: usize, pattern_fn: fn(usize) -> Vec) { + // This test, tests that every is_less is actually observable. Ie. this can go wrong if a hole + // is created using temporary memory and, the whole is used as comparison but not copied back. + // This property must also hold if the user provided comparison panics. + // + // If this is not upheld a custom type + comparison function could yield UB in otherwise safe + // code. Eg T == Mutex>> which replaces the pointer with none in the comparison + // function, which would not be observed in the original slice and would lead to a double free. + + let mut test_input = + pattern_fn(len).into_iter().map(|val| CompCount::new(val)).collect::>(); + + let sum_before: i64 = test_input.iter().map(|x| x.val as i64).sum(); + + // Calculate a specific comparison that should panic. + // Ensure that it can be any of the possible comparisons and that it always panics. + let required_comps = + calc_comps_required::(&mut test_input.clone(), |a, b| a.val.cmp(&b.val)); + + let panic_threshold = patterns::random_uniform(1, 1..=required_comps as i32)[0] as u64 - 1; + + let mut comp_count_global = 0; + + let res = panic::catch_unwind(AssertUnwindSafe(|| { + ::sort_by(&mut test_input, |a, b| { + if comp_count_global == panic_threshold { + // Make the panic dependent on the test len and some random factor. We want to + // make sure that panicking may also happen when comparing elements a second + // time. + panic!(); + } + + a.comp_count.replace(a.comp_count.get() + 1); + b.comp_count.replace(b.comp_count.get() + 1); + comp_count_global += 1; + + a.val.cmp(&b.val) + }); + })); + + assert!(res.is_err()); + + let total_inner: u64 = test_input.iter().map(|c| c.comp_count.get() as u64).sum(); + + assert_eq!(total_inner, comp_count_global * 2); + + // If the sum before and after don't match, it means the set of elements hasn't remained the + // same. + let sum_after: i64 = test_input.iter().map(|x| x.val as i64).sum(); + assert_eq!(sum_before, sum_after); +} + +gen_sort_test_fns_with_default_patterns!( + panic_observable_is_less, + panic_observable_is_less::, + &TEST_LENGTHS[..TEST_LENGTHS.len() - 2], + [] +); + +fn deterministic( + len: usize, + type_into_fn: impl Fn(i32) -> T + Copy, + type_from_fn: impl Fn(&T) -> i32, + pattern_fn: fn(usize) -> Vec, +) { + // A property similar to stability is deterministic output order. If the entire value is used as + // the comparison key a lack of determinism has no effect. But if only a part of the value is + // used as comparison key, a lack of determinism can manifest itself in the order of values + // considered equal by the comparison predicate. + // + // This test only tests that results are deterministic across runs, it does not test determinism + // on different platforms and with different toolchains. + + let mut test_input = + pattern_fn(len).into_iter().map(|val| type_into_fn(val)).collect::>(); + + let mut test_input_clone = test_input.clone(); + + let comparison_fn = |a: &T, b: &T| { + let a_i32 = type_from_fn(a); + let b_i32 = type_from_fn(b); + + let a_i32_key_space_reduced = a_i32 % 10_000; + let b_i32_key_space_reduced = b_i32 % 10_000; + + a_i32_key_space_reduced.cmp(&b_i32_key_space_reduced) + }; + + ::sort_by(&mut test_input, comparison_fn); + ::sort_by(&mut test_input_clone, comparison_fn); + + assert_eq!(test_input, test_input_clone); +} + +gen_sort_test_fns_with_default_patterns_3_ty!(deterministic, deterministic, []); + +fn self_cmp( + len: usize, + type_into_fn: impl Fn(i32) -> T + Copy, + _type_from_fn: impl Fn(&T) -> i32, + pattern_fn: fn(usize) -> Vec, +) { + // It's possible for comparisons to run into problems if the values of `a` and `b` passed into + // the comparison function are the same reference. So this tests that they never are. + + let mut test_input = + pattern_fn(len).into_iter().map(|val| type_into_fn(val)).collect::>(); + + let comparison_fn = |a: &T, b: &T| { + assert_ne!(a as *const T as usize, b as *const T as usize); + a.cmp(b) + }; + + ::sort_by(&mut test_input, comparison_fn); + + // Check that the output is actually sorted and wasn't stopped by the assert. + for window in test_input.windows(2) { + assert!(window[0] <= window[1]); + } +} + +gen_sort_test_fns_with_default_patterns_3_ty!(self_cmp, self_cmp, []); + +fn violate_ord_retain_orig_set( + len: usize, + type_into_fn: impl Fn(i32) -> T + Copy, + type_from_fn: impl Fn(&T) -> i32, + pattern_fn: fn(usize) -> Vec, +) { + // A user may implement Ord incorrectly for a type or violate it by calling sort_by with a + // comparison function that violates Ord with the orderings it returns. Even under such + // circumstances the input must retain its original set of elements. + + // Ord implies a strict total order see https://en.wikipedia.org/wiki/Total_order. + + // Generating random numbers with miri is quite expensive. + let random_orderings_len = if cfg!(miri) { 200 } else { 10_000 }; + + // Make sure we get a good distribution of random orderings, that are repeatable with the seed. + // Just using random_uniform with the same len and range will always yield the same value. + let random_orderings = patterns::random_uniform(random_orderings_len, 0..2); + + let get_random_0_1_or_2 = |random_idx: &mut usize| { + let ridx = *random_idx; + *random_idx += 1; + if ridx + 1 == random_orderings.len() { + *random_idx = 0; + } + + random_orderings[ridx] as usize + }; + + let mut random_idx_a = 0; + let mut random_idx_b = 0; + let mut random_idx_c = 0; + + let mut last_element_a = -1; + let mut last_element_b = -1; + + let mut rand_counter_b = 0; + let mut rand_counter_c = 0; + + let mut streak_counter_a = 0; + let mut streak_counter_b = 0; + + // Examples, a = 3, b = 5, c = 9. + // Correct Ord -> 10010 | is_less(a, b) is_less(a, a) is_less(b, a) is_less(a, c) is_less(c, a) + let mut invalid_ord_comp_functions: Vec Ordering>> = vec![ + Box::new(|_a, _b| -> Ordering { + // random + // Eg. is_less(3, 5) == true, is_less(3, 5) == false + + let idx = get_random_0_1_or_2(&mut random_idx_a); + [Ordering::Less, Ordering::Equal, Ordering::Greater][idx] + }), + Box::new(|_a, _b| -> Ordering { + // everything is less -> 11111 + Ordering::Less + }), + Box::new(|_a, _b| -> Ordering { + // everything is equal -> 00000 + Ordering::Equal + }), + Box::new(|_a, _b| -> Ordering { + // everything is greater -> 00000 + // Eg. is_less(3, 5) == false, is_less(5, 3) == false, is_less(3, 3) == false + Ordering::Greater + }), + Box::new(|a, b| -> Ordering { + // equal means less else greater -> 01000 + if a == b { Ordering::Less } else { Ordering::Greater } + }), + Box::new(|a, b| -> Ordering { + // Transitive breaker. remember last element -> 10001 + let lea = last_element_a; + let leb = last_element_b; + + let a_as_i32 = type_from_fn(a); + let b_as_i32 = type_from_fn(b); + + last_element_a = a_as_i32; + last_element_b = b_as_i32; + + if a_as_i32 == lea && b_as_i32 != leb { b.cmp(a) } else { a.cmp(b) } + }), + Box::new(|a, b| -> Ordering { + // Sampled random 1% of comparisons are reversed. + rand_counter_b += get_random_0_1_or_2(&mut random_idx_b); + if rand_counter_b >= 100 { + rand_counter_b = 0; + b.cmp(a) + } else { + a.cmp(b) + } + }), + Box::new(|a, b| -> Ordering { + // Sampled random 33% of comparisons are reversed. + rand_counter_c += get_random_0_1_or_2(&mut random_idx_c); + if rand_counter_c >= 3 { + rand_counter_c = 0; + b.cmp(a) + } else { + a.cmp(b) + } + }), + Box::new(|a, b| -> Ordering { + // STREAK_LEN comparisons yield a.cmp(b) then STREAK_LEN comparisons less. This can + // discover bugs that neither, random Ord, or just Less or Greater can find. Because it + // can push a pointer further than expected. Random Ord will average out how far a + // comparison based pointer travels. Just Less or Greater will be caught by pattern + // analysis and never enter interesting code. + const STREAK_LEN: usize = 50; + + streak_counter_a += 1; + if streak_counter_a <= STREAK_LEN { + a.cmp(b) + } else { + if streak_counter_a == STREAK_LEN * 2 { + streak_counter_a = 0; + } + Ordering::Less + } + }), + Box::new(|a, b| -> Ordering { + // See above. + const STREAK_LEN: usize = 50; + + streak_counter_b += 1; + if streak_counter_b <= STREAK_LEN { + a.cmp(b) + } else { + if streak_counter_b == STREAK_LEN * 2 { + streak_counter_b = 0; + } + Ordering::Greater + } + }), + ]; + + for comp_func in &mut invalid_ord_comp_functions { + let mut test_data: Vec = pattern_fn(len).into_iter().map(type_into_fn).collect(); + let sum_before: i64 = test_data.iter().map(|x| type_from_fn(x) as i64).sum(); + + // It's ok to panic on Ord violation or to complete. + // In both cases the original elements must still be present. + let _ = panic::catch_unwind(AssertUnwindSafe(|| { + ::sort_by(&mut test_data, &mut *comp_func); + })); + + // If the sum before and after don't match, it means the set of elements hasn't remained the + // same. + let sum_after: i64 = test_data.iter().map(|x| type_from_fn(x) as i64).sum(); + assert_eq!(sum_before, sum_after); + + if cfg!(miri) { + // This test is prohibitively expensive in miri, so only run one of the comparison + // functions. This test is not expected to yield direct UB, but rather surface potential + // UB by showing that the sum is different now. + break; + } + } +} + +gen_sort_test_fns_with_default_patterns_3_ty!( + violate_ord_retain_orig_set, + violate_ord_retain_orig_set, + [] +); + +macro_rules! instantiate_sort_test_inner { + ($sort_impl:ty, miri_yes, $test_fn_name:ident) => { + #[test] + fn $test_fn_name() { + $crate::sort::tests::$test_fn_name::<$sort_impl>(); + } + }; + ($sort_impl:ty, miri_no, $test_fn_name:ident) => { + #[test] + #[cfg_attr(miri, ignore)] + fn $test_fn_name() { + $crate::sort::tests::$test_fn_name::<$sort_impl>(); + } + }; +} + +// Using this construct allows us to get warnings for unused test functions. +macro_rules! define_instantiate_sort_tests { + ($([$miri_use:ident, $test_fn_name:ident]),*,) => { + $(pub fn $test_fn_name() { + ${concat($test_fn_name, _impl)}::(); + })* + + + macro_rules! instantiate_sort_tests_gen { + ($sort_impl:ty) => { + $( + instantiate_sort_test_inner!( + $sort_impl, + $miri_use, + $test_fn_name + ); + )* + } + } + }; +} + +// Some tests are not tested with miri to avoid prohibitively long test times. This leaves coverage +// holes, but the way they are selected should make for relatively small holes. Many properties that +// can lead to UB are tested directly, for example that the original set of elements is retained +// even when a panic occurs or Ord is implemented incorrectly. +define_instantiate_sort_tests!( + [miri_yes, basic], + [miri_yes, fixed_seed], + [miri_yes, fixed_seed_rand_vec_prefix], + [miri_yes, int_edge], + [miri_yes, sort_vs_sort_by], + [miri_yes, correct_i32_random], + [miri_yes, correct_i32_random_z1], + [miri_yes, correct_i32_random_d2], + [miri_yes, correct_i32_random_d20], + [miri_yes, correct_i32_random_s95], + [miri_yes, correct_i32_ascending], + [miri_yes, correct_i32_descending], + [miri_yes, correct_i32_saw_mixed], + [miri_no, correct_i32_random_d4], + [miri_no, correct_i32_random_d8], + [miri_no, correct_i32_random_d311], + [miri_no, correct_i32_random_d1024], + [miri_no, correct_i32_random_z1_03], + [miri_no, correct_i32_random_z2], + [miri_no, correct_i32_random_s50], + [miri_no, correct_i32_narrow], + [miri_no, correct_i32_all_equal], + [miri_no, correct_i32_saw_mixed_range], + [miri_yes, correct_i32_pipe_organ], + [miri_no, correct_u64_random], + [miri_yes, correct_u64_random_z1], + [miri_no, correct_u64_random_d2], + [miri_no, correct_u64_random_d20], + [miri_no, correct_u64_random_s95], + [miri_no, correct_u64_ascending], + [miri_no, correct_u64_descending], + [miri_no, correct_u64_saw_mixed], + [miri_no, correct_u128_random], + [miri_yes, correct_u128_random_z1], + [miri_no, correct_u128_random_d2], + [miri_no, correct_u128_random_d20], + [miri_no, correct_u128_random_s95], + [miri_no, correct_u128_ascending], + [miri_no, correct_u128_descending], + [miri_no, correct_u128_saw_mixed], + [miri_no, correct_cell_i32_random], + [miri_yes, correct_cell_i32_random_z1], + [miri_no, correct_cell_i32_random_d2], + [miri_no, correct_cell_i32_random_d20], + [miri_no, correct_cell_i32_random_s95], + [miri_no, correct_cell_i32_ascending], + [miri_no, correct_cell_i32_descending], + [miri_no, correct_cell_i32_saw_mixed], + [miri_no, correct_string_random], + [miri_yes, correct_string_random_z1], + [miri_no, correct_string_random_d2], + [miri_no, correct_string_random_d20], + [miri_no, correct_string_random_s95], + [miri_no, correct_string_ascending], + [miri_no, correct_string_descending], + [miri_no, correct_string_saw_mixed], + [miri_no, correct_f128_random], + [miri_yes, correct_f128_random_z1], + [miri_no, correct_f128_random_d2], + [miri_no, correct_f128_random_d20], + [miri_no, correct_f128_random_s95], + [miri_no, correct_f128_ascending], + [miri_no, correct_f128_descending], + [miri_no, correct_f128_saw_mixed], + [miri_no, correct_1k_random], + [miri_yes, correct_1k_random_z1], + [miri_no, correct_1k_random_d2], + [miri_no, correct_1k_random_d20], + [miri_no, correct_1k_random_s95], + [miri_no, correct_1k_ascending], + [miri_no, correct_1k_descending], + [miri_no, correct_1k_saw_mixed], + [miri_no, correct_dyn_val_random], + [miri_yes, correct_dyn_val_random_z1], + [miri_no, correct_dyn_val_random_d2], + [miri_no, correct_dyn_val_random_d20], + [miri_no, correct_dyn_val_random_s95], + [miri_no, correct_dyn_val_ascending], + [miri_no, correct_dyn_val_descending], + [miri_no, correct_dyn_val_saw_mixed], + [miri_no, stability_legacy], + [miri_no, stability_i32_random], + [miri_yes, stability_i32_random_z1], + [miri_no, stability_i32_random_d2], + [miri_no, stability_i32_random_d20], + [miri_no, stability_i32_random_s95], + [miri_no, stability_i32_ascending], + [miri_no, stability_i32_descending], + [miri_no, stability_i32_saw_mixed], + [miri_no, stability_cell_i32_random], + [miri_yes, stability_cell_i32_random_z1], + [miri_no, stability_cell_i32_random_d2], + [miri_no, stability_cell_i32_random_d20], + [miri_no, stability_cell_i32_random_s95], + [miri_no, stability_cell_i32_ascending], + [miri_no, stability_cell_i32_descending], + [miri_no, stability_cell_i32_saw_mixed], + [miri_no, stability_string_random], + [miri_yes, stability_string_random_z1], + [miri_no, stability_string_random_d2], + [miri_no, stability_string_random_d20], + [miri_no, stability_string_random_s95], + [miri_no, stability_string_ascending], + [miri_no, stability_string_descending], + [miri_no, stability_string_saw_mixed], + [miri_no, observable_is_less_random], + [miri_yes, observable_is_less_random_z1], + [miri_no, observable_is_less_random_d2], + [miri_no, observable_is_less_random_d20], + [miri_no, observable_is_less_random_s95], + [miri_no, observable_is_less_ascending], + [miri_no, observable_is_less_descending], + [miri_no, observable_is_less_saw_mixed], + [miri_no, panic_retain_orig_set_i32_random], + [miri_yes, panic_retain_orig_set_i32_random_z1], + [miri_no, panic_retain_orig_set_i32_random_d2], + [miri_no, panic_retain_orig_set_i32_random_d20], + [miri_no, panic_retain_orig_set_i32_random_s95], + [miri_no, panic_retain_orig_set_i32_ascending], + [miri_no, panic_retain_orig_set_i32_descending], + [miri_no, panic_retain_orig_set_i32_saw_mixed], + [miri_no, panic_retain_orig_set_cell_i32_random], + [miri_yes, panic_retain_orig_set_cell_i32_random_z1], + [miri_no, panic_retain_orig_set_cell_i32_random_d2], + [miri_no, panic_retain_orig_set_cell_i32_random_d20], + [miri_no, panic_retain_orig_set_cell_i32_random_s95], + [miri_no, panic_retain_orig_set_cell_i32_ascending], + [miri_no, panic_retain_orig_set_cell_i32_descending], + [miri_no, panic_retain_orig_set_cell_i32_saw_mixed], + [miri_no, panic_retain_orig_set_string_random], + [miri_yes, panic_retain_orig_set_string_random_z1], + [miri_no, panic_retain_orig_set_string_random_d2], + [miri_no, panic_retain_orig_set_string_random_d20], + [miri_no, panic_retain_orig_set_string_random_s95], + [miri_no, panic_retain_orig_set_string_ascending], + [miri_no, panic_retain_orig_set_string_descending], + [miri_no, panic_retain_orig_set_string_saw_mixed], + [miri_no, panic_observable_is_less_random], + [miri_yes, panic_observable_is_less_random_z1], + [miri_no, panic_observable_is_less_random_d2], + [miri_no, panic_observable_is_less_random_d20], + [miri_no, panic_observable_is_less_random_s95], + [miri_no, panic_observable_is_less_ascending], + [miri_no, panic_observable_is_less_descending], + [miri_no, panic_observable_is_less_saw_mixed], + [miri_no, deterministic_i32_random], + [miri_yes, deterministic_i32_random_z1], + [miri_no, deterministic_i32_random_d2], + [miri_no, deterministic_i32_random_d20], + [miri_no, deterministic_i32_random_s95], + [miri_no, deterministic_i32_ascending], + [miri_no, deterministic_i32_descending], + [miri_no, deterministic_i32_saw_mixed], + [miri_no, deterministic_cell_i32_random], + [miri_yes, deterministic_cell_i32_random_z1], + [miri_no, deterministic_cell_i32_random_d2], + [miri_no, deterministic_cell_i32_random_d20], + [miri_no, deterministic_cell_i32_random_s95], + [miri_no, deterministic_cell_i32_ascending], + [miri_no, deterministic_cell_i32_descending], + [miri_no, deterministic_cell_i32_saw_mixed], + [miri_no, deterministic_string_random], + [miri_yes, deterministic_string_random_z1], + [miri_no, deterministic_string_random_d2], + [miri_no, deterministic_string_random_d20], + [miri_no, deterministic_string_random_s95], + [miri_no, deterministic_string_ascending], + [miri_no, deterministic_string_descending], + [miri_no, deterministic_string_saw_mixed], + [miri_no, self_cmp_i32_random], + [miri_yes, self_cmp_i32_random_z1], + [miri_no, self_cmp_i32_random_d2], + [miri_no, self_cmp_i32_random_d20], + [miri_no, self_cmp_i32_random_s95], + [miri_no, self_cmp_i32_ascending], + [miri_no, self_cmp_i32_descending], + [miri_no, self_cmp_i32_saw_mixed], + [miri_no, self_cmp_cell_i32_random], + [miri_yes, self_cmp_cell_i32_random_z1], + [miri_no, self_cmp_cell_i32_random_d2], + [miri_no, self_cmp_cell_i32_random_d20], + [miri_no, self_cmp_cell_i32_random_s95], + [miri_no, self_cmp_cell_i32_ascending], + [miri_no, self_cmp_cell_i32_descending], + [miri_no, self_cmp_cell_i32_saw_mixed], + [miri_no, self_cmp_string_random], + [miri_yes, self_cmp_string_random_z1], + [miri_no, self_cmp_string_random_d2], + [miri_no, self_cmp_string_random_d20], + [miri_no, self_cmp_string_random_s95], + [miri_no, self_cmp_string_ascending], + [miri_no, self_cmp_string_descending], + [miri_no, self_cmp_string_saw_mixed], + [miri_no, violate_ord_retain_orig_set_i32_random], + [miri_yes, violate_ord_retain_orig_set_i32_random_z1], + [miri_no, violate_ord_retain_orig_set_i32_random_d2], + [miri_no, violate_ord_retain_orig_set_i32_random_d20], + [miri_no, violate_ord_retain_orig_set_i32_random_s95], + [miri_no, violate_ord_retain_orig_set_i32_ascending], + [miri_no, violate_ord_retain_orig_set_i32_descending], + [miri_no, violate_ord_retain_orig_set_i32_saw_mixed], + [miri_no, violate_ord_retain_orig_set_cell_i32_random], + [miri_yes, violate_ord_retain_orig_set_cell_i32_random_z1], + [miri_no, violate_ord_retain_orig_set_cell_i32_random_d2], + [miri_no, violate_ord_retain_orig_set_cell_i32_random_d20], + [miri_no, violate_ord_retain_orig_set_cell_i32_random_s95], + [miri_no, violate_ord_retain_orig_set_cell_i32_ascending], + [miri_no, violate_ord_retain_orig_set_cell_i32_descending], + [miri_no, violate_ord_retain_orig_set_cell_i32_saw_mixed], + [miri_no, violate_ord_retain_orig_set_string_random], + [miri_yes, violate_ord_retain_orig_set_string_random_z1], + [miri_no, violate_ord_retain_orig_set_string_random_d2], + [miri_no, violate_ord_retain_orig_set_string_random_d20], + [miri_no, violate_ord_retain_orig_set_string_random_s95], + [miri_no, violate_ord_retain_orig_set_string_ascending], + [miri_no, violate_ord_retain_orig_set_string_descending], + [miri_no, violate_ord_retain_orig_set_string_saw_mixed], +); + +macro_rules! instantiate_sort_tests { + ($sort_impl:ty) => { + instantiate_sort_tests_gen!($sort_impl); + }; +} + +mod unstable { + struct SortImpl {} + + impl crate::sort::Sort for SortImpl { + fn name() -> String { + "rust_std_unstable".into() + } + + fn sort(v: &mut [T]) + where + T: Ord, + { + v.sort_unstable(); + } + + fn sort_by(v: &mut [T], mut compare: F) + where + F: FnMut(&T, &T) -> std::cmp::Ordering, + { + v.sort_unstable_by(|a, b| compare(a, b)); + } + } + + instantiate_sort_tests!(SortImpl); +} + +mod stable { + struct SortImpl {} + + impl crate::sort::Sort for SortImpl { + fn name() -> String { + "rust_std_stable".into() + } + + fn sort(v: &mut [T]) + where + T: Ord, + { + v.sort(); + } + + fn sort_by(v: &mut [T], mut compare: F) + where + F: FnMut(&T, &T) -> std::cmp::Ordering, + { + v.sort_by(|a, b| compare(a, b)); + } + } + + instantiate_sort_tests!(SortImpl); +} diff --git a/library/alloc/tests/sort/zipf.rs b/library/alloc/tests/sort/zipf.rs new file mode 100644 index 0000000000000..cc774ee5c43bf --- /dev/null +++ b/library/alloc/tests/sort/zipf.rs @@ -0,0 +1,208 @@ +// This module implements a Zipfian distribution generator. +// +// Based on https://github.com/jonhoo/rust-zipf. + +use rand::Rng; + +/// Random number generator that generates Zipf-distributed random numbers using rejection +/// inversion. +#[derive(Clone, Copy)] +pub struct ZipfDistribution { + /// Number of elements + num_elements: f64, + /// Exponent parameter of the distribution + exponent: f64, + /// `hIntegral(1.5) - 1}` + h_integral_x1: f64, + /// `hIntegral(num_elements + 0.5)}` + h_integral_num_elements: f64, + /// `2 - hIntegralInverse(hIntegral(2.5) - h(2)}` + s: f64, +} + +impl ZipfDistribution { + /// Creates a new [Zipf-distributed](https://en.wikipedia.org/wiki/Zipf's_law) + /// random number generator. + /// + /// Note that both the number of elements and the exponent must be greater than 0. + pub fn new(num_elements: usize, exponent: f64) -> Result { + if num_elements == 0 { + return Err(()); + } + if exponent <= 0f64 { + return Err(()); + } + + let z = ZipfDistribution { + num_elements: num_elements as f64, + exponent, + h_integral_x1: ZipfDistribution::h_integral(1.5, exponent) - 1f64, + h_integral_num_elements: ZipfDistribution::h_integral( + num_elements as f64 + 0.5, + exponent, + ), + s: 2f64 + - ZipfDistribution::h_integral_inv( + ZipfDistribution::h_integral(2.5, exponent) + - ZipfDistribution::h(2f64, exponent), + exponent, + ), + }; + + // populate cache + + Ok(z) + } +} + +impl ZipfDistribution { + fn next(&self, rng: &mut R) -> usize { + // The paper describes an algorithm for exponents larger than 1 (Algorithm ZRI). + // + // The original method uses + // H(x) = (v + x)^(1 - q) / (1 - q) + // as the integral of the hat function. + // + // This function is undefined for q = 1, which is the reason for the limitation of the + // exponent. + // + // If instead the integral function + // H(x) = ((v + x)^(1 - q) - 1) / (1 - q) + // is used, for which a meaningful limit exists for q = 1, the method works for all + // positive exponents. + // + // The following implementation uses v = 0 and generates integral number in the range [1, + // num_elements]. This is different to the original method where v is defined to + // be positive and numbers are taken from [0, i_max]. This explains why the implementation + // looks slightly different. + + let hnum = self.h_integral_num_elements; + + loop { + use std::cmp; + let u: f64 = hnum + rng.gen::() * (self.h_integral_x1 - hnum); + // u is uniformly distributed in (h_integral_x1, h_integral_num_elements] + + let x: f64 = ZipfDistribution::h_integral_inv(u, self.exponent); + + // Limit k to the range [1, num_elements] if it would be outside + // due to numerical inaccuracies. + let k64 = x.max(1.0).min(self.num_elements); + // float -> integer rounds towards zero, so we add 0.5 + // to prevent bias towards k == 1 + let k = cmp::max(1, (k64 + 0.5) as usize); + + // Here, the distribution of k is given by: + // + // P(k = 1) = C * (hIntegral(1.5) - h_integral_x1) = C + // P(k = m) = C * (hIntegral(m + 1/2) - hIntegral(m - 1/2)) for m >= 2 + // + // where C = 1 / (h_integral_num_elements - h_integral_x1) + if k64 - x <= self.s + || u >= ZipfDistribution::h_integral(k64 + 0.5, self.exponent) + - ZipfDistribution::h(k64, self.exponent) + { + // Case k = 1: + // + // The right inequality is always true, because replacing k by 1 gives + // u >= hIntegral(1.5) - h(1) = h_integral_x1 and u is taken from + // (h_integral_x1, h_integral_num_elements]. + // + // Therefore, the acceptance rate for k = 1 is P(accepted | k = 1) = 1 + // and the probability that 1 is returned as random value is + // P(k = 1 and accepted) = P(accepted | k = 1) * P(k = 1) = C = C / 1^exponent + // + // Case k >= 2: + // + // The left inequality (k - x <= s) is just a short cut + // to avoid the more expensive evaluation of the right inequality + // (u >= hIntegral(k + 0.5) - h(k)) in many cases. + // + // If the left inequality is true, the right inequality is also true: + // Theorem 2 in the paper is valid for all positive exponents, because + // the requirements h'(x) = -exponent/x^(exponent + 1) < 0 and + // (-1/hInverse'(x))'' = (1+1/exponent) * x^(1/exponent-1) >= 0 + // are both fulfilled. + // Therefore, f(x) = x - hIntegralInverse(hIntegral(x + 0.5) - h(x)) + // is a non-decreasing function. If k - x <= s holds, + // k - x <= s + f(k) - f(2) is obviously also true which is equivalent to + // -x <= -hIntegralInverse(hIntegral(k + 0.5) - h(k)), + // -hIntegralInverse(u) <= -hIntegralInverse(hIntegral(k + 0.5) - h(k)), + // and finally u >= hIntegral(k + 0.5) - h(k). + // + // Hence, the right inequality determines the acceptance rate: + // P(accepted | k = m) = h(m) / (hIntegrated(m+1/2) - hIntegrated(m-1/2)) + // The probability that m is returned is given by + // P(k = m and accepted) = P(accepted | k = m) * P(k = m) + // = C * h(m) = C / m^exponent. + // + // In both cases the probabilities are proportional to the probability mass + // function of the Zipf distribution. + + return k; + } + } + } +} + +impl rand::distributions::Distribution for ZipfDistribution { + fn sample(&self, rng: &mut R) -> usize { + self.next(rng) + } +} + +use std::fmt; +impl fmt::Debug for ZipfDistribution { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> Result<(), fmt::Error> { + f.debug_struct("ZipfDistribution") + .field("e", &self.exponent) + .field("n", &self.num_elements) + .finish() + } +} + +impl ZipfDistribution { + /// Computes `H(x)`, defined as + /// + /// - `(x^(1 - exponent) - 1) / (1 - exponent)`, if `exponent != 1` + /// - `log(x)`, if `exponent == 1` + /// + /// `H(x)` is an integral function of `h(x)`, the derivative of `H(x)` is `h(x)`. + fn h_integral(x: f64, exponent: f64) -> f64 { + let log_x = x.ln(); + helper2((1f64 - exponent) * log_x) * log_x + } + + /// Computes `h(x) = 1 / x^exponent` + fn h(x: f64, exponent: f64) -> f64 { + (-exponent * x.ln()).exp() + } + + /// The inverse function of `H(x)`. + /// Returns the `y` for which `H(y) = x`. + fn h_integral_inv(x: f64, exponent: f64) -> f64 { + let mut t: f64 = x * (1f64 - exponent); + if t < -1f64 { + // Limit value to the range [-1, +inf). + // t could be smaller than -1 in some rare cases due to numerical errors. + t = -1f64; + } + (helper1(t) * x).exp() + } +} + +/// Helper function that calculates `log(1 + x) / x`. +/// A Taylor series expansion is used, if x is close to 0. +fn helper1(x: f64) -> f64 { + if x.abs() > 1e-8 { x.ln_1p() / x } else { 1f64 - x * (0.5 - x * (1.0 / 3.0 - 0.25 * x)) } +} + +/// Helper function to calculate `(exp(x) - 1) / x`. +/// A Taylor series expansion is used, if x is close to 0. +fn helper2(x: f64) -> f64 { + if x.abs() > 1e-8 { + x.exp_m1() / x + } else { + 1f64 + x * 0.5 * (1f64 + x * 1.0 / 3.0 * (1f64 + 0.25 * x)) + } +} diff --git a/library/alloc/tests/str.rs b/library/alloc/tests/str.rs index a6b1fe5b97945..6f930ab08535c 100644 --- a/library/alloc/tests/str.rs +++ b/library/alloc/tests/str.rs @@ -1524,14 +1524,18 @@ fn test_lines() { t("bare\r", &["bare\r"]); t("bare\rcr", &["bare\rcr"]); t("Text\n\r", &["Text", "\r"]); - t( - "\nMäry häd ä little lämb\n\r\nLittle lämb\n", - &["", "Märy häd ä little lämb", "", "Little lämb"], - ); - t( - "\r\nMäry häd ä little lämb\n\nLittle lämb", - &["", "Märy häd ä little lämb", "", "Little lämb"], - ); + t("\nMäry häd ä little lämb\n\r\nLittle lämb\n", &[ + "", + "Märy häd ä little lämb", + "", + "Little lämb", + ]); + t("\r\nMäry häd ä little lämb\n\nLittle lämb", &[ + "", + "Märy häd ä little lämb", + "", + "Little lämb", + ]); } #[test] @@ -1850,7 +1854,10 @@ fn to_lowercase() { assert_eq!("ΑΣ''Α".to_lowercase(), "ασ''α"); // https://github.com/rust-lang/rust/issues/124714 + // input lengths around the boundary of the chunk size used by the ascii prefix optimization + assert_eq!("abcdefghijklmnoΣ".to_lowercase(), "abcdefghijklmnoς"); assert_eq!("abcdefghijklmnopΣ".to_lowercase(), "abcdefghijklmnopς"); + assert_eq!("abcdefghijklmnopqΣ".to_lowercase(), "abcdefghijklmnopqς"); // a really long string that has it's lowercase form // even longer. this tests that implementations don't assume @@ -1971,88 +1978,73 @@ mod pattern { assert_eq!(v, right); } - make_test!( - str_searcher_ascii_haystack, - "bb", - "abbcbbd", - [Reject(0, 1), Match(1, 3), Reject(3, 4), Match(4, 6), Reject(6, 7),] - ); - make_test!( - str_searcher_ascii_haystack_seq, - "bb", - "abbcbbbbd", - [Reject(0, 1), Match(1, 3), Reject(3, 4), Match(4, 6), Match(6, 8), Reject(8, 9),] - ); - make_test!( - str_searcher_empty_needle_ascii_haystack, - "", - "abbcbbd", - [ - Match(0, 0), - Reject(0, 1), - Match(1, 1), - Reject(1, 2), - Match(2, 2), - Reject(2, 3), - Match(3, 3), - Reject(3, 4), - Match(4, 4), - Reject(4, 5), - Match(5, 5), - Reject(5, 6), - Match(6, 6), - Reject(6, 7), - Match(7, 7), - ] - ); - make_test!( - str_searcher_multibyte_haystack, - " ", - "├──", - [Reject(0, 3), Reject(3, 6), Reject(6, 9),] - ); - make_test!( - str_searcher_empty_needle_multibyte_haystack, - "", - "├──", - [ - Match(0, 0), - Reject(0, 3), - Match(3, 3), - Reject(3, 6), - Match(6, 6), - Reject(6, 9), - Match(9, 9), - ] - ); + make_test!(str_searcher_ascii_haystack, "bb", "abbcbbd", [ + Reject(0, 1), + Match(1, 3), + Reject(3, 4), + Match(4, 6), + Reject(6, 7), + ]); + make_test!(str_searcher_ascii_haystack_seq, "bb", "abbcbbbbd", [ + Reject(0, 1), + Match(1, 3), + Reject(3, 4), + Match(4, 6), + Match(6, 8), + Reject(8, 9), + ]); + make_test!(str_searcher_empty_needle_ascii_haystack, "", "abbcbbd", [ + Match(0, 0), + Reject(0, 1), + Match(1, 1), + Reject(1, 2), + Match(2, 2), + Reject(2, 3), + Match(3, 3), + Reject(3, 4), + Match(4, 4), + Reject(4, 5), + Match(5, 5), + Reject(5, 6), + Match(6, 6), + Reject(6, 7), + Match(7, 7), + ]); + make_test!(str_searcher_multibyte_haystack, " ", "├──", [ + Reject(0, 3), + Reject(3, 6), + Reject(6, 9), + ]); + make_test!(str_searcher_empty_needle_multibyte_haystack, "", "├──", [ + Match(0, 0), + Reject(0, 3), + Match(3, 3), + Reject(3, 6), + Match(6, 6), + Reject(6, 9), + Match(9, 9), + ]); make_test!(str_searcher_empty_needle_empty_haystack, "", "", [Match(0, 0),]); make_test!(str_searcher_nonempty_needle_empty_haystack, "├", "", []); - make_test!( - char_searcher_ascii_haystack, - 'b', - "abbcbbd", - [ - Reject(0, 1), - Match(1, 2), - Match(2, 3), - Reject(3, 4), - Match(4, 5), - Match(5, 6), - Reject(6, 7), - ] - ); - make_test!( - char_searcher_multibyte_haystack, - ' ', - "├──", - [Reject(0, 3), Reject(3, 6), Reject(6, 9),] - ); - make_test!( - char_searcher_short_haystack, - '\u{1F4A9}', - "* \t", - [Reject(0, 1), Reject(1, 2), Reject(2, 3),] - ); + make_test!(char_searcher_ascii_haystack, 'b', "abbcbbd", [ + Reject(0, 1), + Match(1, 2), + Match(2, 3), + Reject(3, 4), + Match(4, 5), + Match(5, 6), + Reject(6, 7), + ]); + make_test!(char_searcher_multibyte_haystack, ' ', "├──", [ + Reject(0, 3), + Reject(3, 6), + Reject(6, 9), + ]); + make_test!(char_searcher_short_haystack, '\u{1F4A9}', "* \t", [ + Reject(0, 1), + Reject(1, 2), + Reject(2, 3), + ]); // See #85462 #[test] diff --git a/library/alloc/tests/string.rs b/library/alloc/tests/string.rs index dc03c4860e84b..1c8bff1564db2 100644 --- a/library/alloc/tests/string.rs +++ b/library/alloc/tests/string.rs @@ -114,31 +114,59 @@ fn test_from_utf8_lossy() { ); } +#[test] +fn test_fromutf8error_into_lossy() { + fn func(input: &[u8]) -> String { + String::from_utf8(input.to_owned()).unwrap_or_else(|e| e.into_utf8_lossy()) + } + + let xs = b"hello"; + let ys = "hello".to_owned(); + assert_eq!(func(xs), ys); + + let xs = "ศไทย中华Việt Nam".as_bytes(); + let ys = "ศไทย中华Việt Nam".to_owned(); + assert_eq!(func(xs), ys); + + let xs = b"Hello\xC2 There\xFF Goodbye"; + assert_eq!(func(xs), "Hello\u{FFFD} There\u{FFFD} Goodbye".to_owned()); + + let xs = b"Hello\xC0\x80 There\xE6\x83 Goodbye"; + assert_eq!(func(xs), "Hello\u{FFFD}\u{FFFD} There\u{FFFD} Goodbye".to_owned()); + + let xs = b"\xF5foo\xF5\x80bar"; + assert_eq!(func(xs), "\u{FFFD}foo\u{FFFD}\u{FFFD}bar".to_owned()); + + let xs = b"\xF1foo\xF1\x80bar\xF1\x80\x80baz"; + assert_eq!(func(xs), "\u{FFFD}foo\u{FFFD}bar\u{FFFD}baz".to_owned()); + + let xs = b"\xF4foo\xF4\x80bar\xF4\xBFbaz"; + assert_eq!(func(xs), "\u{FFFD}foo\u{FFFD}bar\u{FFFD}\u{FFFD}baz".to_owned()); + + let xs = b"\xF0\x80\x80\x80foo\xF0\x90\x80\x80bar"; + assert_eq!(func(xs), "\u{FFFD}\u{FFFD}\u{FFFD}\u{FFFD}foo\u{10000}bar".to_owned()); + + // surrogates + let xs = b"\xED\xA0\x80foo\xED\xBF\xBFbar"; + assert_eq!(func(xs), "\u{FFFD}\u{FFFD}\u{FFFD}foo\u{FFFD}\u{FFFD}\u{FFFD}bar".to_owned()); +} + #[test] fn test_from_utf16() { let pairs = [ - ( - String::from("𐍅𐌿𐌻𐍆𐌹𐌻𐌰\n"), - vec![ - 0xd800, 0xdf45, 0xd800, 0xdf3f, 0xd800, 0xdf3b, 0xd800, 0xdf46, 0xd800, 0xdf39, - 0xd800, 0xdf3b, 0xd800, 0xdf30, 0x000a, - ], - ), - ( - String::from("𐐒𐑉𐐮𐑀𐐲𐑋 𐐏𐐲𐑍\n"), - vec![ - 0xd801, 0xdc12, 0xd801, 0xdc49, 0xd801, 0xdc2e, 0xd801, 0xdc40, 0xd801, 0xdc32, - 0xd801, 0xdc4b, 0x0020, 0xd801, 0xdc0f, 0xd801, 0xdc32, 0xd801, 0xdc4d, 0x000a, - ], - ), - ( - String::from("𐌀𐌖𐌋𐌄𐌑𐌉·𐌌𐌄𐌕𐌄𐌋𐌉𐌑\n"), - vec![ - 0xd800, 0xdf00, 0xd800, 0xdf16, 0xd800, 0xdf0b, 0xd800, 0xdf04, 0xd800, 0xdf11, - 0xd800, 0xdf09, 0x00b7, 0xd800, 0xdf0c, 0xd800, 0xdf04, 0xd800, 0xdf15, 0xd800, - 0xdf04, 0xd800, 0xdf0b, 0xd800, 0xdf09, 0xd800, 0xdf11, 0x000a, - ], - ), + (String::from("𐍅𐌿𐌻𐍆𐌹𐌻𐌰\n"), vec![ + 0xd800, 0xdf45, 0xd800, 0xdf3f, 0xd800, 0xdf3b, 0xd800, 0xdf46, 0xd800, 0xdf39, 0xd800, + 0xdf3b, 0xd800, 0xdf30, 0x000a, + ]), + (String::from("𐐒𐑉𐐮𐑀𐐲𐑋 𐐏𐐲𐑍\n"), vec![ + 0xd801, 0xdc12, 0xd801, 0xdc49, 0xd801, 0xdc2e, 0xd801, 0xdc40, 0xd801, 0xdc32, 0xd801, + 0xdc4b, 0x0020, 0xd801, 0xdc0f, 0xd801, 0xdc32, 0xd801, 0xdc4d, 0x000a, + ]), + (String::from("𐌀𐌖𐌋𐌄𐌑𐌉·𐌌𐌄𐌕𐌄𐌋𐌉𐌑\n"), vec![ + 0xd800, 0xdf00, 0xd800, 0xdf16, 0xd800, 0xdf0b, 0xd800, 0xdf04, 0xd800, 0xdf11, 0xd800, + 0xdf09, 0x00b7, 0xd800, 0xdf0c, 0xd800, 0xdf04, 0xd800, 0xdf15, 0xd800, 0xdf04, 0xd800, + 0xdf0b, 0xd800, 0xdf09, 0xd800, 0xdf11, 0x000a, + ]), ( String::from("𐒋𐒘𐒈𐒑𐒛𐒒 𐒕𐒓 𐒈𐒚𐒍 𐒏𐒜𐒒𐒖𐒆 𐒕𐒆\n"), vec![ diff --git a/library/alloc/tests/vec.rs b/library/alloc/tests/vec.rs index 3722fb06a6a8a..0f27fdff3e182 100644 --- a/library/alloc/tests/vec.rs +++ b/library/alloc/tests/vec.rs @@ -1,3 +1,6 @@ +// FIXME(static_mut_refs): Do not allow `static_mut_refs` lint +#![allow(static_mut_refs)] + use core::alloc::{Allocator, Layout}; use core::num::NonZero; use core::ptr::NonNull; @@ -11,7 +14,7 @@ use std::fmt::Debug; use std::iter::InPlaceIterable; use std::mem::{size_of, swap}; use std::ops::Bound::*; -use std::panic::{catch_unwind, AssertUnwindSafe}; +use std::panic::{AssertUnwindSafe, catch_unwind}; use std::rc::Rc; use std::sync::atomic::{AtomicU32, Ordering}; use std::vec::{Drain, IntoIter}; @@ -1284,6 +1287,8 @@ fn test_from_iter_specialization_panic_during_iteration_drops() { #[test] #[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] +// FIXME(static_mut_refs): Do not allow `static_mut_refs` lint +#[allow(static_mut_refs)] fn test_from_iter_specialization_panic_during_drop_doesnt_leak() { static mut DROP_COUNTER_OLD: [usize; 5] = [0; 5]; static mut DROP_COUNTER_NEW: [usize; 2] = [0; 2]; diff --git a/library/alloc/tests/vec_deque.rs b/library/alloc/tests/vec_deque.rs index f32ba8d5aa461..4b8d3c735f72b 100644 --- a/library/alloc/tests/vec_deque.rs +++ b/library/alloc/tests/vec_deque.rs @@ -1,11 +1,14 @@ +// FIXME(static_mut_refs): Do not allow `static_mut_refs` lint +#![allow(static_mut_refs)] + use core::num::NonZero; use std::assert_matches::assert_matches; -use std::collections::vec_deque::Drain; use std::collections::TryReserveErrorKind::*; use std::collections::VecDeque; +use std::collections::vec_deque::Drain; use std::fmt::Debug; use std::ops::Bound::*; -use std::panic::{catch_unwind, AssertUnwindSafe}; +use std::panic::{AssertUnwindSafe, catch_unwind}; use Taggy::*; use Taggypar::*; diff --git a/library/alloc/tests/vec_deque_alloc_error.rs b/library/alloc/tests/vec_deque_alloc_error.rs index c41d8266eb457..21a9118a05bd6 100644 --- a/library/alloc/tests/vec_deque_alloc_error.rs +++ b/library/alloc/tests/vec_deque_alloc_error.rs @@ -1,8 +1,8 @@ #![feature(alloc_error_hook, allocator_api)] -use std::alloc::{set_alloc_error_hook, AllocError, Allocator, Layout, System}; +use std::alloc::{AllocError, Allocator, Layout, System, set_alloc_error_hook}; use std::collections::VecDeque; -use std::panic::{catch_unwind, AssertUnwindSafe}; +use std::panic::{AssertUnwindSafe, catch_unwind}; use std::ptr::NonNull; #[test] diff --git a/library/contracts/safety/src/kani.rs b/library/contracts/safety/src/kani.rs index a8b4ab360dc8e..0f36ed2d68530 100644 --- a/library/contracts/safety/src/kani.rs +++ b/library/contracts/safety/src/kani.rs @@ -1,6 +1,6 @@ -use proc_macro::{TokenStream}; -use quote::{quote, format_ident}; -use syn::{ItemFn, parse_macro_input}; +use proc_macro::TokenStream; +use quote::{format_ident, quote}; +use syn::{parse_macro_input, ItemFn, Stmt}; pub(crate) fn requires(attr: TokenStream, item: TokenStream) -> TokenStream { rewrite_attr(attr, item, "requires") @@ -10,6 +10,21 @@ pub(crate) fn ensures(attr: TokenStream, item: TokenStream) -> TokenStream { rewrite_attr(attr, item, "ensures") } +pub(crate) fn loop_invariant(attr: TokenStream, stmt: TokenStream) -> TokenStream { + rewrite_stmt_attr(attr, stmt, "loop_invariant") +} + +fn rewrite_stmt_attr(attr: TokenStream, stmt_stream: TokenStream, name: &str) -> TokenStream { + let args = proc_macro2::TokenStream::from(attr); + let stmt = parse_macro_input!(stmt_stream as Stmt); + let attribute = format_ident!("{}", name); + quote!( + #[kani_core::#attribute(#args)] + #stmt + ) + .into() +} + fn rewrite_attr(attr: TokenStream, item: TokenStream, name: &str) -> TokenStream { let args = proc_macro2::TokenStream::from(attr); let fn_item = parse_macro_input!(item as ItemFn); @@ -17,5 +32,6 @@ fn rewrite_attr(attr: TokenStream, item: TokenStream, name: &str) -> TokenStream quote!( #[kani_core::#attribute(#args)] #fn_item - ).into() + ) + .into() } diff --git a/library/contracts/safety/src/lib.rs b/library/contracts/safety/src/lib.rs index 9fe852a622de3..eba2e91b44e85 100644 --- a/library/contracts/safety/src/lib.rs +++ b/library/contracts/safety/src/lib.rs @@ -3,6 +3,11 @@ use proc_macro::TokenStream; use proc_macro_error::proc_macro_error; +use quote::{format_ident, quote, quote_spanned}; +use syn::{ + parse_macro_input, parse_quote, spanned::Spanned, Data, DataEnum, DeriveInput, Fields, + GenericParam, Generics, Ident, Index, ItemStruct, +}; #[cfg(kani_host)] #[path = "kani.rs"] @@ -12,6 +17,135 @@ mod tool; #[path = "runtime.rs"] mod tool; +/// Expands the `#[invariant(...)]` attribute macro. +/// The macro expands to an implementation of the `is_safe` method for the `Invariant` trait. +/// This attribute is only supported for structs. +/// +/// # Example +/// +/// ```ignore +/// #[invariant(self.width == self.height)] +/// struct Square { +/// width: u32, +/// height: u32, +/// } +/// ``` +/// +/// expands to: +/// ```ignore +/// impl core::ub_checks::Invariant for Square { +/// fn is_safe(&self) -> bool { +/// self.width == self.height +/// } +/// } +/// ``` +/// For more information on the Invariant trait, see its documentation in core::ub_checks. +#[proc_macro_error] +#[proc_macro_attribute] +pub fn invariant(attr: TokenStream, item: TokenStream) -> TokenStream { + let safe_body = proc_macro2::TokenStream::from(attr); + let item = parse_macro_input!(item as ItemStruct); + let item_name = &item.ident; + let (impl_generics, ty_generics, where_clause) = item.generics.split_for_impl(); + + let expanded = quote! { + #item + #[unstable(feature="invariant", issue="none")] + impl #impl_generics core::ub_checks::Invariant for #item_name #ty_generics #where_clause { + fn is_safe(&self) -> bool { + #safe_body + } + } + }; + + proc_macro::TokenStream::from(expanded) +} + +/// Expands the derive macro for the Invariant trait. +/// The macro expands to an implementation of the `is_safe` method for the `Invariant` trait. +/// This macro is only supported for structs and enums. +/// +/// # Example +/// +/// ```ignore +/// #[derive(Invariant)] +/// struct Square { +/// width: u32, +/// height: u32, +/// } +/// ``` +/// +/// expands to: +/// ```ignore +/// impl core::ub_checks::Invariant for Square { +/// fn is_safe(&self) -> bool { +/// self.width.is_safe() && self.height.is_safe() +/// } +/// } +/// ``` +/// For enums, the body of `is_safe` matches on the variant and calls `is_safe` on its fields, +/// # Example +/// +/// ```ignore +/// #[derive(Invariant)] +/// enum MyEnum { +/// OptionOne(u32, u32), +/// OptionTwo(Square), +/// OptionThree +/// } +/// ``` +/// +/// expands to: +/// ```ignore +/// impl core::ub_checks::Invariant for MyEnum { +/// fn is_safe(&self) -> bool { +/// match self { +/// MyEnum::OptionOne(field1, field2) => field1.is_safe() && field2.is_safe(), +/// MyEnum::OptionTwo(field1) => field1.is_safe(), +/// MyEnum::OptionThree => true, +/// } +/// } +/// } +/// ``` +/// For more information on the Invariant trait, see its documentation in core::ub_checks. +#[proc_macro_error] +#[proc_macro_derive(Invariant)] +pub fn derive_invariant(item: TokenStream) -> TokenStream { + let derive_item = parse_macro_input!(item as DeriveInput); + let item_name = &derive_item.ident; + let safe_body = match derive_item.data { + Data::Struct(struct_data) => { + safe_body(&struct_data.fields) + }, + Data::Enum(enum_data) => { + let variant_checks = variant_checks(enum_data, item_name); + + quote! { + match self { + #(#variant_checks),* + } + } + }, + Data::Union(..) => unimplemented!("Attempted to derive Invariant on a union; Invariant can only be derived for structs and enums."), + }; + + // Add a bound `T: Invariant` to every type parameter T. + let generics = add_trait_bound_invariant(derive_item.generics); + // Generate an expression to sum up the heap size of each field. + let (impl_generics, ty_generics, where_clause) = generics.split_for_impl(); + + let expanded = quote! { + // The generated implementation. + #[unstable(feature="invariant", issue="none")] + impl #impl_generics core::ub_checks::Invariant for #item_name #ty_generics #where_clause { + fn is_safe(&self) -> bool { + #safe_body + } + } + }; + proc_macro::TokenStream::from(expanded) +} + #[proc_macro_error] #[proc_macro_attribute] pub fn requires(attr: TokenStream, item: TokenStream) -> TokenStream { @@ -23,3 +157,102 @@ pub fn requires(attr: TokenStream, item: TokenStream) -> TokenStream { pub fn ensures(attr: TokenStream, item: TokenStream) -> TokenStream { tool::ensures(attr, item) } + +#[proc_macro_error] +#[proc_macro_attribute] +pub fn loop_invariant(attr: TokenStream, stmt_stream: TokenStream) -> TokenStream { + tool::loop_invariant(attr, stmt_stream) +} + +/// Add a bound `T: Invariant` to every type parameter T. +fn add_trait_bound_invariant(mut generics: Generics) -> Generics { + generics.params.iter_mut().for_each(|param| { + if let GenericParam::Type(type_param) = param { + type_param + .bounds + .push(parse_quote!(core::ub_checks::Invariant)); + } + }); + generics +} + +/// Generate safety checks for each variant of an enum +fn variant_checks(enum_data: DataEnum, item_name: &Ident) -> Vec { + enum_data + .variants + .iter() + .map(|variant| { + let variant_name = &variant.ident; + match &variant.fields { + Fields::Unnamed(fields) => { + let field_names: Vec<_> = fields + .unnamed + .iter() + .enumerate() + .map(|(i, _)| format_ident!("field{}", i + 1)) + .collect(); + + let field_checks: Vec<_> = field_names + .iter() + .map(|field_name| { + quote! { #field_name.is_safe() } + }) + .collect(); + + quote! { + #item_name::#variant_name(#(#field_names),*) => #(#field_checks)&&* + } + } + Fields::Unit => { + quote! { + #item_name::#variant_name => true + } + } + Fields::Named(_) => unreachable!("Enums do not have named fields"), + } + }) + .collect() +} + +/// Generate the body for the `is_safe` method. +/// For each field of the type, enforce that it is safe. +fn safe_body(fields: &Fields) -> proc_macro2::TokenStream { + match fields { + Fields::Named(ref fields) => { + let field_safe_calls: Vec = fields + .named + .iter() + .map(|field| { + let name = &field.ident; + quote_spanned! {field.span()=> + self.#name.is_safe() + } + }) + .collect(); + if !field_safe_calls.is_empty() { + quote! { #( #field_safe_calls )&&* } + } else { + quote! { true } + } + } + Fields::Unnamed(ref fields) => { + let field_safe_calls: Vec = fields + .unnamed + .iter() + .enumerate() + .map(|(idx, field)| { + let field_idx = Index::from(idx); + quote_spanned! {field.span()=> + self.#field_idx.is_safe() + } + }) + .collect(); + if !field_safe_calls.is_empty() { + quote! { #( #field_safe_calls )&&* } + } else { + quote! { true } + } + } + Fields::Unit => quote! { true }, + } +} diff --git a/library/contracts/safety/src/runtime.rs b/library/contracts/safety/src/runtime.rs index 78e8b1dc354d2..3d1cb1d96c90e 100644 --- a/library/contracts/safety/src/runtime.rs +++ b/library/contracts/safety/src/runtime.rs @@ -7,9 +7,16 @@ pub(crate) fn requires(_attr: TokenStream, item: TokenStream) -> TokenStream { item } -/// For now, runtime requires is a no-op. +/// For now, runtime ensures is a no-op. /// /// TODO: At runtime the `ensures` should become an assert as well. pub(crate) fn ensures(_attr: TokenStream, item: TokenStream) -> TokenStream { item } + +/// For now, runtime loop_invariant is a no-op. +/// +/// TODO: At runtime the `loop_invariant` should become an assert as well. +pub(crate) fn loop_invariant(_attr: TokenStream, stmt_stream: TokenStream) -> TokenStream { + stmt_stream +} diff --git a/library/core/benches/any.rs b/library/core/benches/any.rs index f6be41bcdbfa7..6b150432f874d 100644 --- a/library/core/benches/any.rs +++ b/library/core/benches/any.rs @@ -1,6 +1,6 @@ use core::any::*; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; #[bench] fn bench_downcast_ref(b: &mut Bencher) { diff --git a/library/core/benches/array.rs b/library/core/benches/array.rs index b1a41a088c493..751f3235a5f34 100644 --- a/library/core/benches/array.rs +++ b/library/core/benches/array.rs @@ -1,4 +1,4 @@ -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; macro_rules! map_array { ($func_name:ident, $start_item: expr, $map_item: expr, $arr_size: expr) => { diff --git a/library/core/benches/ascii.rs b/library/core/benches/ascii.rs index 61bf8bbf411d5..3fe45aa360bf0 100644 --- a/library/core/benches/ascii.rs +++ b/library/core/benches/ascii.rs @@ -65,7 +65,7 @@ macro_rules! benches { use std::fmt::Write; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; const ASCII_CASE_MASK: u8 = 0b0010_0000; diff --git a/library/core/benches/ascii/is_ascii.rs b/library/core/benches/ascii/is_ascii.rs index 05f60a46f8be2..4b2920c5eb45f 100644 --- a/library/core/benches/ascii/is_ascii.rs +++ b/library/core/benches/ascii/is_ascii.rs @@ -1,4 +1,4 @@ -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; use super::{LONG, MEDIUM, SHORT}; diff --git a/library/core/benches/char/methods.rs b/library/core/benches/char/methods.rs index 5d4df1ac8bd58..ed71920a4fc2a 100644 --- a/library/core/benches/char/methods.rs +++ b/library/core/benches/char/methods.rs @@ -1,4 +1,4 @@ -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; const CHARS: [char; 9] = ['0', 'x', '2', '5', 'A', 'f', '7', '8', '9']; const RADIX: [u32; 5] = [2, 8, 10, 16, 32]; diff --git a/library/core/benches/fmt.rs b/library/core/benches/fmt.rs index 906d7ac3eef28..4baefd5578808 100644 --- a/library/core/benches/fmt.rs +++ b/library/core/benches/fmt.rs @@ -1,7 +1,7 @@ use std::fmt::{self, Write as FmtWrite}; use std::io::{self, Write as IoWrite}; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; #[bench] fn write_vec_value(bh: &mut Bencher) { diff --git a/library/core/benches/hash/sip.rs b/library/core/benches/hash/sip.rs index 8e8c07b6ee4c3..c6562d3c01187 100644 --- a/library/core/benches/hash/sip.rs +++ b/library/core/benches/hash/sip.rs @@ -2,7 +2,7 @@ use core::hash::*; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; fn hash_bytes(mut s: H, x: &[u8]) -> u64 { Hasher::write(&mut s, x); diff --git a/library/core/benches/iter.rs b/library/core/benches/iter.rs index 24469ba0c4262..e14f26b729032 100644 --- a/library/core/benches/iter.rs +++ b/library/core/benches/iter.rs @@ -4,7 +4,7 @@ use core::mem; use core::num::Wrapping; use core::ops::Range; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; #[bench] fn bench_rposition(b: &mut Bencher) { diff --git a/library/core/benches/lib.rs b/library/core/benches/lib.rs index 3f1c58bbd7204..32d15c386cb1b 100644 --- a/library/core/benches/lib.rs +++ b/library/core/benches/lib.rs @@ -8,7 +8,6 @@ #![feature(iter_array_chunks)] #![feature(iter_next_chunk)] #![feature(iter_advance_by)] -#![feature(isqrt)] extern crate test; diff --git a/library/core/benches/net/addr_parser.rs b/library/core/benches/net/addr_parser.rs index b9406a9779dc6..bbf2ea3eb9796 100644 --- a/library/core/benches/net/addr_parser.rs +++ b/library/core/benches/net/addr_parser.rs @@ -1,7 +1,7 @@ use core::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr, SocketAddrV4, SocketAddrV6}; use core::str::FromStr; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; const IPV4_STR: &str = "192.168.0.1"; const IPV4_STR_PORT: &str = "192.168.0.1:8080"; diff --git a/library/core/benches/num/dec2flt/mod.rs b/library/core/benches/num/dec2flt/mod.rs index fb4a786b27e3d..bad211f240ca1 100644 --- a/library/core/benches/num/dec2flt/mod.rs +++ b/library/core/benches/num/dec2flt/mod.rs @@ -1,4 +1,4 @@ -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; #[bench] fn bench_0(b: &mut Bencher) { diff --git a/library/core/benches/num/flt2dec/mod.rs b/library/core/benches/num/flt2dec/mod.rs index 6c7de5dcd2286..428d0bbbbfbfa 100644 --- a/library/core/benches/num/flt2dec/mod.rs +++ b/library/core/benches/num/flt2dec/mod.rs @@ -3,10 +3,10 @@ mod strategy { mod grisu; } -use core::num::flt2dec::{decode, DecodableFloat, Decoded, FullDecoded, MAX_SIG_DIGITS}; +use core::num::flt2dec::{DecodableFloat, Decoded, FullDecoded, MAX_SIG_DIGITS, decode}; use std::io::Write; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; pub fn decode_finite(v: T) -> Decoded { match decode(v).1 { diff --git a/library/core/benches/num/int_log/mod.rs b/library/core/benches/num/int_log/mod.rs index 3807cd5d76cfc..e5874ddf03b5b 100644 --- a/library/core/benches/num/int_log/mod.rs +++ b/library/core/benches/num/int_log/mod.rs @@ -1,5 +1,5 @@ use rand::Rng; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; macro_rules! int_log10_bench { ($t:ty, $predictable:ident, $random:ident, $random_small:ident) => { diff --git a/library/core/benches/num/int_pow/mod.rs b/library/core/benches/num/int_pow/mod.rs index 063d722bdd1b5..6cf9021358283 100644 --- a/library/core/benches/num/int_pow/mod.rs +++ b/library/core/benches/num/int_pow/mod.rs @@ -1,5 +1,5 @@ use rand::Rng; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; const ITERATIONS: usize = 128; // Uses an ITERATIONS * 20 Byte stack allocation type IntType = i128; // Hardest native type to multiply diff --git a/library/core/benches/num/int_sqrt/mod.rs b/library/core/benches/num/int_sqrt/mod.rs index 3c9d173e456a1..e47b92e866eff 100644 --- a/library/core/benches/num/int_sqrt/mod.rs +++ b/library/core/benches/num/int_sqrt/mod.rs @@ -1,5 +1,5 @@ use rand::Rng; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; macro_rules! int_sqrt_bench { ($t:ty, $predictable:ident, $random:ident, $random_small:ident, $random_uniform:ident) => { diff --git a/library/core/benches/num/mod.rs b/library/core/benches/num/mod.rs index 7ff7443cfa7fe..b36100e59a97a 100644 --- a/library/core/benches/num/mod.rs +++ b/library/core/benches/num/mod.rs @@ -6,7 +6,7 @@ mod int_sqrt; use std::str::FromStr; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; const ASCII_NUMBERS: [&str; 19] = [ "0", diff --git a/library/core/benches/pattern.rs b/library/core/benches/pattern.rs index 0d60b005feb32..b0f8b39c22e16 100644 --- a/library/core/benches/pattern.rs +++ b/library/core/benches/pattern.rs @@ -1,4 +1,4 @@ -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; #[bench] fn starts_with_char(b: &mut Bencher) { diff --git a/library/core/benches/slice.rs b/library/core/benches/slice.rs index 2741dbd53f14c..29a66b6219976 100644 --- a/library/core/benches/slice.rs +++ b/library/core/benches/slice.rs @@ -1,6 +1,6 @@ use core::ptr::NonNull; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; enum Cache { L1, diff --git a/library/core/benches/str.rs b/library/core/benches/str.rs index a8178f9c18752..2f7d9d56a70b7 100644 --- a/library/core/benches/str.rs +++ b/library/core/benches/str.rs @@ -1,6 +1,6 @@ use std::str; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; mod char_count; mod corpora; diff --git a/library/core/benches/str/char_count.rs b/library/core/benches/str/char_count.rs index b87ad0f6adf24..343e23dcf41c5 100644 --- a/library/core/benches/str/char_count.rs +++ b/library/core/benches/str/char_count.rs @@ -1,4 +1,4 @@ -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; use super::corpora::*; diff --git a/library/core/benches/str/debug.rs b/library/core/benches/str/debug.rs index 6dbf4e92c084b..e41d4fa110a63 100644 --- a/library/core/benches/str/debug.rs +++ b/library/core/benches/str/debug.rs @@ -5,7 +5,7 @@ use std::fmt::{self, Write}; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; #[derive(Default)] struct CountingWriter { diff --git a/library/core/benches/str/iter.rs b/library/core/benches/str/iter.rs index f6e73e48d8e7d..d2586cef25871 100644 --- a/library/core/benches/str/iter.rs +++ b/library/core/benches/str/iter.rs @@ -1,4 +1,4 @@ -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; use super::corpora; diff --git a/library/core/benches/tuple.rs b/library/core/benches/tuple.rs index d9ff9d0dd9378..dcda7c641aa21 100644 --- a/library/core/benches/tuple.rs +++ b/library/core/benches/tuple.rs @@ -1,5 +1,5 @@ use rand::prelude::*; -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; #[bench] fn bench_tuple_comparison(b: &mut Bencher) { diff --git a/library/core/src/alloc/global.rs b/library/core/src/alloc/global.rs index a6f799c4a7deb..8f48af24557d8 100644 --- a/library/core/src/alloc/global.rs +++ b/library/core/src/alloc/global.rs @@ -124,8 +124,8 @@ pub unsafe trait GlobalAlloc { /// /// # Safety /// - /// This function is unsafe because undefined behavior can result - /// if the caller does not ensure that `layout` has non-zero size. + /// `layout` must have non-zero size. Attempting to allocate for a zero-sized `layout` may + /// result in undefined behavior. /// /// (Extension subtraits might provide more specific bounds on /// behavior, e.g., guarantee a sentinel address or a null pointer @@ -156,14 +156,14 @@ pub unsafe trait GlobalAlloc { /// /// # Safety /// - /// This function is unsafe because undefined behavior can result - /// if the caller does not ensure all of the following: + /// The caller must ensure: /// - /// * `ptr` must denote a block of memory currently allocated via - /// this allocator, + /// * `ptr` is a block of memory currently allocated via this allocator and, /// - /// * `layout` must be the same layout that was used - /// to allocate that block of memory. + /// * `layout` is the same layout that was used to allocate that block of + /// memory. + /// + /// Otherwise undefined behavior can result. #[stable(feature = "global_alloc", since = "1.28.0")] unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout); @@ -172,7 +172,8 @@ pub unsafe trait GlobalAlloc { /// /// # Safety /// - /// This function is unsafe for the same reasons that `alloc` is. + /// The caller has to ensure that `layout` has non-zero size. Like `alloc` + /// zero sized `layout` can result in undefined behavior. /// However the allocated block of memory is guaranteed to be initialized. /// /// # Errors @@ -220,20 +221,21 @@ pub unsafe trait GlobalAlloc { /// /// # Safety /// - /// This function is unsafe because undefined behavior can result - /// if the caller does not ensure all of the following: + /// The caller must ensure that: /// - /// * `ptr` must be currently allocated via this allocator, + /// * `ptr` is allocated via this allocator, /// - /// * `layout` must be the same layout that was used + /// * `layout` is the same layout that was used /// to allocate that block of memory, /// - /// * `new_size` must be greater than zero. + /// * `new_size` is greater than zero. /// /// * `new_size`, when rounded up to the nearest multiple of `layout.align()`, - /// must not overflow `isize` (i.e., the rounded value must be less than or + /// does not overflow `isize` (i.e., the rounded value must be less than or /// equal to `isize::MAX`). /// + /// If these are not followed, undefined behavior can result. + /// /// (Extension subtraits might provide more specific bounds on /// behavior, e.g., guarantee a sentinel address or a null pointer /// in response to a zero-size allocation request.) diff --git a/library/core/src/alloc/layout.rs b/library/core/src/alloc/layout.rs index 666c5be7c29b5..808e2245045e0 100644 --- a/library/core/src/alloc/layout.rs +++ b/library/core/src/alloc/layout.rs @@ -4,13 +4,21 @@ // collections, resulting in having to optimize down excess IR multiple times. // Your performance intuition is useless. Run perf. -use safety::{ensures, requires}; +use safety::{ensures, Invariant, requires}; use crate::error::Error; +use crate::intrinsics::{unchecked_add, unchecked_mul, unchecked_sub}; +use crate::mem::SizedTypeProperties; use crate::ptr::{Alignment, NonNull}; -use crate::{assert_unsafe_precondition, cmp, fmt, mem}; +use crate::{assert_unsafe_precondition, fmt, mem}; #[cfg(kani)] use crate::kani; +#[cfg(kani)] +use crate::cmp; + +// Used only for contract verification. +#[allow(unused_imports)] +use crate::ub_checks::Invariant; // While this function is used in one place and its implementation // could be inlined, the previous attempts to do so made rustc @@ -39,6 +47,7 @@ const fn size_align() -> (usize, usize) { #[stable(feature = "alloc_layout", since = "1.28.0")] #[derive(Copy, Clone, Debug, PartialEq, Eq, Hash)] #[lang = "alloc_layout"] +#[derive(Invariant)] pub struct Layout { // size of the requested block of memory, measured in bytes. size: usize, @@ -106,7 +115,10 @@ impl Layout { // // Above implies that checking for summation overflow is both // necessary and sufficient. - isize::MAX as usize - (align.as_usize() - 1) + + // SAFETY: the maximum possible alignment is `isize::MAX + 1`, + // so the subtraction cannot overflow. + unsafe { unchecked_sub(isize::MAX as usize + 1, align.as_usize()) } } /// Internal helper constructor to skip revalidating alignment validity. @@ -132,6 +144,7 @@ impl Layout { #[inline] #[rustc_allow_const_fn_unstable(ptr_alignment_type)] #[requires(Layout::from_size_align(size, align).is_ok())] + #[ensures(|result| result.is_safe())] #[ensures(|result| result.size() == size)] #[ensures(|result| result.align() == align)] pub const unsafe fn from_size_align_unchecked(size: usize, align: usize) -> Self { @@ -165,7 +178,7 @@ impl Layout { #[must_use = "this returns the minimum alignment, \ without modifying the layout"] #[inline] - #[rustc_allow_const_fn_unstable(ptr_alignment_type)] + #[cfg_attr(bootstrap, rustc_allow_const_fn_unstable(ptr_alignment_type))] pub const fn align(&self) -> usize { self.align.as_usize() } @@ -228,7 +241,7 @@ impl Layout { /// [trait object]: ../../book/ch17-02-trait-objects.html /// [extern type]: ../../unstable-book/language-features/extern-types.html #[unstable(feature = "layout_for_ptr", issue = "69835")] - #[rustc_const_unstable(feature = "const_alloc_layout", issue = "67521")] + #[rustc_const_unstable(feature = "layout_for_ptr", issue = "69835")] #[must_use] // TODO: we should try to capture the above constraints on T in a `requires` clause, and the // metadata helpers from https://github.com/model-checking/verify-rust-std/pull/37 may be able @@ -248,7 +261,6 @@ impl Layout { /// sentinel value. Types that lazily allocate must track initialization by /// some other means. #[unstable(feature = "alloc_layout_extra", issue = "55724")] - #[rustc_const_unstable(feature = "alloc_layout_extra", issue = "55724")] #[must_use] #[inline] #[ensures(|result| result.is_aligned())] @@ -272,11 +284,17 @@ impl Layout { /// Returns an error if the combination of `self.size()` and the given /// `align` violates the conditions listed in [`Layout::from_size_align`]. #[stable(feature = "alloc_layout_manipulation", since = "1.44.0")] + #[rustc_const_unstable(feature = "const_alloc_layout", issue = "67521")] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[inline] #[ensures(|result| result.is_err() || result.as_ref().unwrap().align() >= align)] #[ensures(|result| result.is_err() || result.as_ref().unwrap().align().is_power_of_two())] - pub fn align_to(&self, align: usize) -> Result { - Layout::from_size_align(self.size(), cmp::max(self.align(), align)) + pub const fn align_to(&self, align: usize) -> Result { + if let Some(align) = Alignment::new(align) { + Layout::from_size_alignment(self.size, Alignment::max(self.align, align)) + } else { + Err(LayoutError) + } } /// Returns the amount of padding we must insert after `self` @@ -296,35 +314,47 @@ impl Layout { /// address for the whole allocated block of memory. One way to /// satisfy this constraint is to ensure `align <= self.align()`. #[unstable(feature = "alloc_layout_extra", issue = "55724")] - #[rustc_const_unstable(feature = "const_alloc_layout", issue = "67521")] #[must_use = "this returns the padding needed, \ without modifying the `Layout`"] #[inline] #[ensures(|result| *result <= align)] pub const fn padding_needed_for(&self, align: usize) -> usize { - let len = self.size(); + // FIXME: Can we just change the type on this to `Alignment`? + let Some(align) = Alignment::new(align) else { return usize::MAX }; + let len_rounded_up = self.size_rounded_up_to_custom_align(align); + // SAFETY: Cannot overflow because the rounded-up value is never less + unsafe { unchecked_sub(len_rounded_up, self.size) } + } + /// Returns the smallest multiple of `align` greater than or equal to `self.size()`. + /// + /// This can return at most `Alignment::MAX` (aka `isize::MAX + 1`) + /// because the original size is at most `isize::MAX`. + #[inline] + const fn size_rounded_up_to_custom_align(&self, align: Alignment) -> usize { + // SAFETY: // Rounded up value is: - // len_rounded_up = (len + align - 1) & !(align - 1); - // and then we return the padding difference: `len_rounded_up - len`. + // size_rounded_up = (size + align - 1) & !(align - 1); // - // We use modular arithmetic throughout: + // The arithmetic we do here can never overflow: // // 1. align is guaranteed to be > 0, so align - 1 is always // valid. // - // 2. `len + align - 1` can overflow by at most `align - 1`, - // so the &-mask with `!(align - 1)` will ensure that in the - // case of overflow, `len_rounded_up` will itself be 0. - // Thus the returned padding, when added to `len`, yields 0, - // which trivially satisfies the alignment `align`. + // 2. size is at most `isize::MAX`, so adding `align - 1` (which is at + // most `isize::MAX`) can never overflow a `usize`. // - // (Of course, attempts to allocate blocks of memory whose - // size and padding overflow in the above manner should cause - // the allocator to yield an error anyway.) - - let len_rounded_up = len.wrapping_add(align).wrapping_sub(1) & !align.wrapping_sub(1); - len_rounded_up.wrapping_sub(len) + // 3. masking by the alignment can remove at most `align - 1`, + // which is what we just added, thus the value we return is never + // less than the original `size`. + // + // (Size 0 Align MAX is already aligned, so stays the same, but things like + // Size 1 Align MAX or Size isize::MAX Align 2 round up to `isize::MAX + 1`.) + unsafe { + let align_m1 = unchecked_sub(align.as_usize(), 1); + let size_rounded_up = unchecked_add(self.size, align_m1) & !align_m1; + size_rounded_up + } } /// Creates a layout by rounding the size of this layout up to a multiple @@ -334,6 +364,7 @@ impl Layout { /// to the layout's current size. #[stable(feature = "alloc_layout_manipulation", since = "1.44.0")] #[rustc_const_unstable(feature = "const_alloc_layout", issue = "67521")] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[must_use = "this returns a new `Layout`, \ without modifying the original"] #[inline] @@ -342,12 +373,11 @@ impl Layout { #[ensures(|result| result.size() % result.align() == 0)] #[ensures(|result| self.size() + self.padding_needed_for(self.align()) == result.size())] pub const fn pad_to_align(&self) -> Layout { - let pad = self.padding_needed_for(self.align()); // This cannot overflow. Quoting from the invariant of Layout: // > `size`, when rounded up to the nearest multiple of `align`, // > must not overflow isize (i.e., the rounded value must be // > less than or equal to `isize::MAX`) - let new_size = self.size() + pad; + let new_size = self.size_rounded_up_to_custom_align(self.align); // SAFETY: padded size is guaranteed to not exceed `isize::MAX`. unsafe { Layout::from_size_align_unchecked(new_size, self.align()) } @@ -360,7 +390,26 @@ impl Layout { /// layout of the array and `offs` is the distance between the start /// of each element in the array. /// + /// (That distance between elements is sometimes known as "stride".) + /// /// On arithmetic overflow, returns `LayoutError`. + /// + /// # Examples + /// + /// ``` + /// #![feature(alloc_layout_extra)] + /// use std::alloc::Layout; + /// + /// // All rust types have a size that's a multiple of their alignment. + /// let normal = Layout::from_size_align(12, 4).unwrap(); + /// let repeated = normal.repeat(3).unwrap(); + /// assert_eq!(repeated, (Layout::from_size_align(36, 4).unwrap(), 12)); + /// + /// // But you can manually make layouts which don't meet that rule. + /// let padding_needed = Layout::from_size_align(6, 4).unwrap(); + /// let repeated = padding_needed.repeat(3).unwrap(); + /// assert_eq!(repeated, (Layout::from_size_align(24, 4).unwrap(), 8)); + /// ``` #[unstable(feature = "alloc_layout_extra", issue = "55724")] #[inline] // for Kani (v0.54.0), the below modulo operation is too costly to prove (running into the @@ -377,17 +426,13 @@ impl Layout { #[cfg_attr(kani, ensures(|result| result.is_err() || n == 0 || result.as_ref().unwrap().0.size() >= result.as_ref().unwrap().1))] - pub fn repeat(&self, n: usize) -> Result<(Self, usize), LayoutError> { - // This cannot overflow. Quoting from the invariant of Layout: - // > `size`, when rounded up to the nearest multiple of `align`, - // > must not overflow isize (i.e., the rounded value must be - // > less than or equal to `isize::MAX`) - let padded_size = self.size() + self.padding_needed_for(self.align()); - let alloc_size = padded_size.checked_mul(n).ok_or(LayoutError)?; - - // The safe constructor is called here to enforce the isize size limit. - let layout = Layout::from_size_alignment(alloc_size, self.align)?; - Ok((layout, padded_size)) + pub const fn repeat(&self, n: usize) -> Result<(Self, usize), LayoutError> { + let padded = self.pad_to_align(); + if let Ok(repeated) = padded.repeat_packed(n) { + Ok((repeated, padded.size())) + } else { + Err(LayoutError) + } } /// Creates a layout describing the record for `self` followed by @@ -436,21 +481,28 @@ impl Layout { /// # assert_eq!(repr_c(&[u64, u32, u16, u32]), Ok((s, vec![0, 8, 12, 16]))); /// ``` #[stable(feature = "alloc_layout_manipulation", since = "1.44.0")] + #[rustc_const_unstable(feature = "const_alloc_layout", issue = "67521")] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[inline] #[ensures(|result| result.is_err() || result.as_ref().unwrap().0.align() == cmp::max(self.align(), next.align()))] #[ensures(|result| result.is_err() || result.as_ref().unwrap().0.size() >= self.size() + next.size())] #[ensures(|result| result.is_err() || result.as_ref().unwrap().1 >= self.size())] #[ensures(|result| result.is_err() || result.as_ref().unwrap().1 <= result.as_ref().unwrap().0.size())] - pub fn extend(&self, next: Self) -> Result<(Self, usize), LayoutError> { - let new_align = cmp::max(self.align, next.align); - let pad = self.padding_needed_for(next.align()); - - let offset = self.size().checked_add(pad).ok_or(LayoutError)?; - let new_size = offset.checked_add(next.size()).ok_or(LayoutError)?; - - // The safe constructor is called here to enforce the isize size limit. - let layout = Layout::from_size_alignment(new_size, new_align)?; - Ok((layout, offset)) + pub const fn extend(&self, next: Self) -> Result<(Self, usize), LayoutError> { + let new_align = Alignment::max(self.align, next.align); + let offset = self.size_rounded_up_to_custom_align(next.align); + + // SAFETY: `offset` is at most `isize::MAX + 1` (such as from aligning + // to `Alignment::MAX`) and `next.size` is at most `isize::MAX` (from the + // `Layout` type invariant). Thus the largest possible `new_size` is + // `isize::MAX + 1 + isize::MAX`, which is `usize::MAX`, and cannot overflow. + let new_size = unsafe { unchecked_add(offset, next.size) }; + + if let Ok(layout) = Layout::from_size_alignment(new_size, new_align) { + Ok((layout, offset)) + } else { + Err(LayoutError) + } } /// Creates a layout describing the record for `n` instances of @@ -474,10 +526,13 @@ impl Layout { #[cfg_attr(kani, ensures(|result| result.is_err() || n == 0 || result.as_ref().unwrap().size() >= self.size()))] #[ensures(|result| result.is_err() || result.as_ref().unwrap().align() == self.align())] - pub fn repeat_packed(&self, n: usize) -> Result { - let size = self.size().checked_mul(n).ok_or(LayoutError)?; - // The safe constructor is called here to enforce the isize size limit. - Layout::from_size_alignment(size, self.align) + pub const fn repeat_packed(&self, n: usize) -> Result { + if let Some(size) = self.size.checked_mul(n) { + // The safe constructor is called here to enforce the isize size limit. + Layout::from_size_alignment(size, self.align) + } else { + Err(LayoutError) + } } /// Creates a layout describing the record for `self` followed by @@ -490,9 +545,11 @@ impl Layout { #[inline] #[ensures(|result| result.is_err() || result.as_ref().unwrap().size() == self.size() + next.size())] #[ensures(|result| result.is_err() || result.as_ref().unwrap().align() == self.align())] - pub fn extend_packed(&self, next: Self) -> Result { - let new_size = self.size().checked_add(next.size()).ok_or(LayoutError)?; - // The safe constructor is called here to enforce the isize size limit. + pub const fn extend_packed(&self, next: Self) -> Result { + // SAFETY: each `size` is at most `isize::MAX == usize::MAX/2`, so the + // sum is at most `usize::MAX/2*2 == usize::MAX - 1`, and cannot overflow. + let new_size = unsafe { unchecked_add(self.size, next.size) }; + // The safe constructor enforces that the new size isn't too big for the alignment Layout::from_size_alignment(new_size, self.align) } @@ -502,19 +559,18 @@ impl Layout { /// `isize::MAX`, returns `LayoutError`. #[stable(feature = "alloc_layout_manipulation", since = "1.44.0")] #[rustc_const_unstable(feature = "const_alloc_layout", issue = "67521")] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[inline] #[ensures(|result| result.is_err() || result.as_ref().unwrap().size() == n * mem::size_of::())] #[ensures(|result| result.is_err() || result.as_ref().unwrap().align() == mem::align_of::())] pub const fn array(n: usize) -> Result { // Reduce the amount of code we need to monomorphize per `T`. - return inner(mem::size_of::(), Alignment::of::(), n); + return inner(T::LAYOUT, n); #[inline] - const fn inner( - element_size: usize, - align: Alignment, - n: usize, - ) -> Result { + const fn inner(element_layout: Layout, n: usize) -> Result { + let Layout { size: element_size, align } = element_layout; + // We need to check two things about the size: // - That the total size won't overflow a `usize`, and // - That the total size still fits in an `isize`. @@ -529,7 +585,7 @@ impl Layout { // This is a useless hint inside this function, but after inlining this helps // deduplicate checks for whether the overall capacity is zero (e.g., in RawVec's // allocation path) before/after this multiplication. - let array_size = unsafe { element_size.unchecked_mul(n) }; + let array_size = unsafe { unchecked_mul(element_size, n) }; // SAFETY: We just checked above that the `array_size` will not // exceed `isize::MAX` even when rounded up to the alignment. @@ -547,7 +603,8 @@ impl Layout { )] pub type LayoutErr = LayoutError; -/// The parameters given to `Layout::from_size_align` +/// The `LayoutError` is returned when the parameters given +/// to `Layout::from_size_align` /// or some other `Layout` constructor /// do not satisfy its documented constraints. #[stable(feature = "alloc_layout_error", since = "1.50.0")] diff --git a/library/core/src/arch.rs b/library/core/src/arch.rs index 31d6bc36fc8b9..57f456c98b3c6 100644 --- a/library/core/src/arch.rs +++ b/library/core/src/arch.rs @@ -17,6 +17,19 @@ pub macro asm("assembly template", $(operands,)* $(options($(option),*))?) { /* compiler built-in */ } +/// Inline assembly used in combination with `#[naked]` functions. +/// +/// Refer to [Rust By Example] for a usage guide and the [reference] for +/// detailed information about the syntax and available options. +/// +/// [Rust By Example]: https://doc.rust-lang.org/nightly/rust-by-example/unsafe/asm.html +/// [reference]: https://doc.rust-lang.org/nightly/reference/inline-assembly.html +#[unstable(feature = "naked_functions", issue = "90957")] +#[rustc_builtin_macro] +pub macro naked_asm("assembly template", $(operands,)* $(options($(option),*))?) { + /* compiler built-in */ +} + /// Module-level inline assembly. /// /// Refer to [Rust By Example] for a usage guide and the [reference] for diff --git a/library/core/src/array/ascii.rs b/library/core/src/array/ascii.rs index 05797b042ee4a..e2faef855bc2c 100644 --- a/library/core/src/array/ascii.rs +++ b/library/core/src/array/ascii.rs @@ -9,7 +9,6 @@ impl [u8; N] { /// /// ``` /// #![feature(ascii_char)] - /// #![feature(const_option)] /// /// const HEX_DIGITS: [std::ascii::Char; 16] = /// *b"0123456789abcdef".as_ascii().unwrap(); diff --git a/library/core/src/array/mod.rs b/library/core/src/array/mod.rs index c63f261edabfa..4764d7f0b0fe0 100644 --- a/library/core/src/array/mod.rs +++ b/library/core/src/array/mod.rs @@ -2,7 +2,7 @@ //! //! *[See also the array primitive type](array).* -#![stable(feature = "core_array", since = "1.36.0")] +#![stable(feature = "core_array", since = "1.35.0")] use crate::borrow::{Borrow, BorrowMut}; use crate::cmp::Ordering; @@ -10,7 +10,7 @@ use crate::convert::Infallible; use crate::error::Error; use crate::fmt; use crate::hash::{self, Hash}; -use crate::iter::{repeat_n, UncheckedIterator}; +use crate::iter::{UncheckedIterator, repeat_n}; use crate::mem::{self, MaybeUninit}; use crate::ops::{ ChangeOutputType, ControlFlow, FromResidual, Index, IndexMut, NeverShortCircuit, Residual, Try, @@ -146,7 +146,7 @@ pub const fn from_ref(s: &T) -> &[T; 1] { /// Converts a mutable reference to `T` into a mutable reference to an array of length 1 (without copying). #[stable(feature = "array_from_ref", since = "1.53.0")] -#[rustc_const_unstable(feature = "const_array_from_ref", issue = "90206")] +#[rustc_const_stable(feature = "const_array_from_ref", since = "1.83.0")] pub const fn from_mut(s: &mut T) -> &mut [T; 1] { // SAFETY: Converting `&mut T` to `&mut [T; 1]` is sound. unsafe { &mut *(s as *mut T).cast::<[T; 1]>() } @@ -154,10 +154,11 @@ pub const fn from_mut(s: &mut T) -> &mut [T; 1] { /// The error type returned when a conversion from a slice to an array fails. #[stable(feature = "try_from", since = "1.34.0")] +#[rustc_allowed_through_unstable_modules] #[derive(Debug, Copy, Clone)] pub struct TryFromSliceError(()); -#[stable(feature = "core_array", since = "1.36.0")] +#[stable(feature = "core_array", since = "1.35.0")] impl fmt::Display for TryFromSliceError { #[inline] fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { diff --git a/library/core/src/ascii/ascii_char.rs b/library/core/src/ascii/ascii_char.rs index 29f4c041e166d..84267c38b5973 100644 --- a/library/core/src/ascii/ascii_char.rs +++ b/library/core/src/ascii/ascii_char.rs @@ -513,7 +513,7 @@ impl AsciiChar { pub const unsafe fn digit_unchecked(d: u8) -> Self { assert_unsafe_precondition!( check_language_ub, - "`AsciiChar::digit_unchecked` input cannot exceed 9.", + "`ascii::Char::digit_unchecked` input cannot exceed 9.", (d: u8 = d) => d < 10 ); diff --git a/library/core/src/async_iter/mod.rs b/library/core/src/async_iter/mod.rs index e1f1c9075823d..a5b03b7dd4f14 100644 --- a/library/core/src/async_iter/mod.rs +++ b/library/core/src/async_iter/mod.rs @@ -125,4 +125,4 @@ mod async_iter; mod from_iter; pub use async_iter::{AsyncIterator, IntoAsyncIterator}; -pub use from_iter::{from_iter, FromIter}; +pub use from_iter::{FromIter, from_iter}; diff --git a/library/core/src/bool.rs b/library/core/src/bool.rs index 03cdff9b13be1..58a870d2e0725 100644 --- a/library/core/src/bool.rs +++ b/library/core/src/bool.rs @@ -55,6 +55,7 @@ impl bool { /// assert_eq!(a, 1); /// ``` #[stable(feature = "lazy_bool_to_option", since = "1.50.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "bool_then")] #[inline] pub fn then T>(self, f: F) -> Option { if self { Some(f()) } else { None } diff --git a/library/core/src/cell.rs b/library/core/src/cell.rs index a3a471a57c7aa..7e6c042274df6 100644 --- a/library/core/src/cell.rs +++ b/library/core/src/cell.rs @@ -193,11 +193,11 @@ //! use std::marker::PhantomData; //! //! struct Rc { -//! ptr: NonNull>, -//! phantom: PhantomData>, +//! ptr: NonNull>, +//! phantom: PhantomData>, //! } //! -//! struct RcBox { +//! struct RcInner { //! strong: Cell, //! refcount: Cell, //! value: T, @@ -213,9 +213,9 @@ //! } //! } //! -//! trait RcBoxPtr { +//! trait RcInnerPtr { //! -//! fn inner(&self) -> &RcBox; +//! fn inner(&self) -> &RcInner; //! //! fn strong(&self) -> usize { //! self.inner().strong.get() @@ -230,8 +230,8 @@ //! } //! } //! -//! impl RcBoxPtr for Rc { -//! fn inner(&self) -> &RcBox { +//! impl RcInnerPtr for Rc { +//! fn inner(&self) -> &RcInner { //! unsafe { //! self.ptr.as_ref() //! } @@ -304,6 +304,7 @@ pub use once::OnceCell; /// ``` /// /// See the [module-level documentation](self) for more. +#[cfg_attr(not(test), rustc_diagnostic_item = "Cell")] #[stable(feature = "rust1", since = "1.0.0")] #[repr(transparent)] #[rustc_pub_transparent] @@ -494,8 +495,9 @@ impl Cell { /// ``` #[inline] #[stable(feature = "move_cell", since = "1.17.0")] + #[rustc_const_unstable(feature = "const_cell", issue = "131283")] #[rustc_confusables("swap")] - pub fn replace(&self, val: T) -> T { + pub const fn replace(&self, val: T) -> T { // SAFETY: This can cause data races if called from a separate thread, // but `Cell` is `!Sync` so this won't happen. mem::replace(unsafe { &mut *self.value.get() }, val) @@ -514,7 +516,8 @@ impl Cell { /// assert_eq!(five, 5); /// ``` #[stable(feature = "move_cell", since = "1.17.0")] - #[rustc_const_unstable(feature = "const_cell_into_inner", issue = "78729")] + #[rustc_const_stable(feature = "const_cell_into_inner", since = "1.83.0")] + #[rustc_allow_const_fn_unstable(const_precise_live_drops)] pub const fn into_inner(self) -> T { self.value.into_inner() } @@ -534,7 +537,8 @@ impl Cell { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - pub fn get(&self) -> T { + #[rustc_const_unstable(feature = "const_cell", issue = "131283")] + pub const fn get(&self) -> T { // SAFETY: This can cause data races if called from a separate thread, // but `Cell` is `!Sync` so this won't happen. unsafe { *self.value.get() } @@ -612,7 +616,8 @@ impl Cell { /// ``` #[inline] #[stable(feature = "cell_get_mut", since = "1.11.0")] - pub fn get_mut(&mut self) -> &mut T { + #[rustc_const_unstable(feature = "const_cell", issue = "131283")] + pub const fn get_mut(&mut self) -> &mut T { self.value.get_mut() } @@ -631,7 +636,8 @@ impl Cell { /// ``` #[inline] #[stable(feature = "as_cell", since = "1.37.0")] - pub fn from_mut(t: &mut T) -> &Cell { + #[rustc_const_unstable(feature = "const_cell", issue = "131283")] + pub const fn from_mut(t: &mut T) -> &Cell { // SAFETY: `&mut` ensures unique access. unsafe { &*(t as *mut T as *const Cell) } } @@ -661,7 +667,7 @@ impl Cell { impl, U> CoerceUnsized> for Cell {} // Allow types that wrap `Cell` to also implement `DispatchFromDyn` -// and become object safe method receivers. +// and become dyn-compatible method receivers. // Note that currently `Cell` itself cannot be a method receiver // because it does not implement Deref. // In other words: @@ -685,7 +691,8 @@ impl Cell<[T]> { /// assert_eq!(slice_cell.len(), 3); /// ``` #[stable(feature = "as_cell", since = "1.37.0")] - pub fn as_slice_of_cells(&self) -> &[Cell] { + #[rustc_const_unstable(feature = "const_cell", issue = "131283")] + pub const fn as_slice_of_cells(&self) -> &[Cell] { // SAFETY: `Cell` has the same memory layout as `T`. unsafe { &*(self as *const Cell<[T]> as *const [Cell]) } } @@ -705,7 +712,8 @@ impl Cell<[T; N]> { /// let array_cell: &[Cell; 3] = cell_array.as_array_of_cells(); /// ``` #[unstable(feature = "as_array_of_cells", issue = "88248")] - pub fn as_array_of_cells(&self) -> &[Cell; N] { + #[rustc_const_unstable(feature = "as_array_of_cells", issue = "88248")] + pub const fn as_array_of_cells(&self) -> &[Cell; N] { // SAFETY: `Cell` has the same memory layout as `T`. unsafe { &*(self as *const Cell<[T; N]> as *const [Cell; N]) } } @@ -857,7 +865,8 @@ impl RefCell { /// let five = c.into_inner(); /// ``` #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_cell_into_inner", issue = "78729")] + #[rustc_const_stable(feature = "const_cell_into_inner", since = "1.83.0")] + #[rustc_allow_const_fn_unstable(const_precise_live_drops)] #[inline] pub const fn into_inner(self) -> T { // Since this function takes `self` (the `RefCell`) by value, the @@ -1213,7 +1222,7 @@ impl RefCell { /// Unlike `RefCell::borrow`, this method is unsafe because it does not /// return a `Ref`, thus leaving the borrow flag untouched. Mutably /// borrowing the `RefCell` while the reference returned by this method - /// is alive is undefined behaviour. + /// is alive is undefined behavior. /// /// # Examples /// @@ -1577,10 +1586,10 @@ impl<'b, T: ?Sized> Ref<'b, T> { { let (a, b) = f(&*orig); let borrow = orig.borrow.clone(); - ( - Ref { value: NonNull::from(a), borrow }, - Ref { value: NonNull::from(b), borrow: orig.borrow }, - ) + (Ref { value: NonNull::from(a), borrow }, Ref { + value: NonNull::from(b), + borrow: orig.borrow, + }) } /// Converts into a reference to the underlying data. @@ -1745,10 +1754,11 @@ impl<'b, T: ?Sized> RefMut<'b, T> { { let borrow = orig.borrow.clone(); let (a, b) = f(&mut *orig); - ( - RefMut { value: NonNull::from(a), borrow, marker: PhantomData }, - RefMut { value: NonNull::from(b), borrow: orig.borrow, marker: PhantomData }, - ) + (RefMut { value: NonNull::from(a), borrow, marker: PhantomData }, RefMut { + value: NonNull::from(b), + borrow: orig.borrow, + marker: PhantomData, + }) } /// Converts into a mutable reference to the underlying data. @@ -1894,11 +1904,17 @@ impl fmt::Display for RefMut<'_, T> { /// uniqueness guarantee for mutable references is unaffected. There is *no* legal way to obtain /// aliasing `&mut`, not even with `UnsafeCell`. /// +/// `UnsafeCell` does nothing to avoid data races; they are still undefined behavior. If multiple +/// threads have access to the same `UnsafeCell`, they must follow the usual rules of the +/// [concurrent memory model]: conflicting non-synchronized accesses must be done via the APIs in +/// [`core::sync::atomic`]. +/// /// The `UnsafeCell` API itself is technically very simple: [`.get()`] gives you a raw pointer /// `*mut T` to its contents. It is up to _you_ as the abstraction designer to use that raw pointer /// correctly. /// /// [`.get()`]: `UnsafeCell::get` +/// [concurrent memory model]: ../sync/atomic/index.html#memory-model-for-atomic-accesses /// /// The precise Rust aliasing rules are somewhat in flux, but the main points are not contentious: /// @@ -1921,10 +1937,6 @@ impl fmt::Display for RefMut<'_, T> { /// live memory and the compiler is allowed to insert spurious reads if it can prove that this /// memory has not yet been deallocated. /// -/// - At all times, you must avoid data races. If multiple threads have access to -/// the same `UnsafeCell`, then any writes must have a proper happens-before relation to all other -/// accesses (or use atomics). -/// /// To assist with proper design, the following scenarios are explicitly declared legal /// for single-threaded code: /// @@ -2097,8 +2109,8 @@ impl UnsafeCell { /// ``` #[inline(always)] #[stable(feature = "rust1", since = "1.0.0")] - // When this is const stabilized, please remove `primitive_into_inner` below. - #[rustc_const_unstable(feature = "const_cell_into_inner", issue = "78729")] + #[rustc_const_stable(feature = "const_cell_into_inner", since = "1.83.0")] + #[rustc_allow_const_fn_unstable(const_precise_live_drops)] pub const fn into_inner(self) -> T { self.value } @@ -2170,7 +2182,7 @@ impl UnsafeCell { /// ``` #[inline(always)] #[stable(feature = "unsafe_cell_get_mut", since = "1.50.0")] - #[rustc_const_unstable(feature = "const_unsafecell_get_mut", issue = "88836")] + #[rustc_const_stable(feature = "const_unsafecell_get_mut", since = "1.83.0")] pub const fn get_mut(&mut self) -> &mut T { &mut self.value } @@ -2235,7 +2247,7 @@ impl From for UnsafeCell { impl, U> CoerceUnsized> for UnsafeCell {} // Allow types that wrap `UnsafeCell` to also implement `DispatchFromDyn` -// and become object safe method receivers. +// and become dyn-compatible method receivers. // Note that currently `UnsafeCell` itself cannot be a method receiver // because it does not implement Deref. // In other words: @@ -2244,47 +2256,6 @@ impl, U> CoerceUnsized> for UnsafeCell {} #[unstable(feature = "dispatch_from_dyn", issue = "none")] impl, U> DispatchFromDyn> for UnsafeCell {} -// Special cases of UnsafeCell::into_inner where T is a primitive. These are -// used by Atomic*::into_inner. -// -// The real UnsafeCell::into_inner cannot be used yet in a stable const function. -// That is blocked on a "precise drop analysis" unstable const feature. -// https://github.com/rust-lang/rust/issues/73255 -macro_rules! unsafe_cell_primitive_into_inner { - ($($primitive:ident $atomic:literal)*) => { - $( - #[cfg(target_has_atomic_load_store = $atomic)] - impl UnsafeCell<$primitive> { - pub(crate) const fn primitive_into_inner(self) -> $primitive { - self.value - } - } - )* - }; -} - -unsafe_cell_primitive_into_inner! { - i8 "8" - u8 "8" - i16 "16" - u16 "16" - i32 "32" - u32 "32" - i64 "64" - u64 "64" - i128 "128" - u128 "128" - isize "ptr" - usize "ptr" -} - -#[cfg(target_has_atomic_load_store = "ptr")] -impl UnsafeCell<*mut T> { - pub(crate) const fn primitive_into_inner(self) -> *mut T { - self.value - } -} - /// [`UnsafeCell`], but [`Sync`]. /// /// This is just an `UnsafeCell`, except it implements `Sync` @@ -2317,6 +2288,7 @@ impl SyncUnsafeCell { /// Unwraps the value, consuming the cell. #[inline] + #[rustc_const_unstable(feature = "sync_unsafe_cell", issue = "95439")] pub const fn into_inner(self) -> T { self.value.into_inner() } @@ -2378,7 +2350,7 @@ impl From for SyncUnsafeCell { impl, U> CoerceUnsized> for SyncUnsafeCell {} // Allow types that wrap `SyncUnsafeCell` to also implement `DispatchFromDyn` -// and become object safe method receivers. +// and become dyn-compatible method receivers. // Note that currently `SyncUnsafeCell` itself cannot be a method receiver // because it does not implement Deref. // In other words: diff --git a/library/core/src/cell/lazy.rs b/library/core/src/cell/lazy.rs index 6ec1d2a33bef5..5ac33516684d7 100644 --- a/library/core/src/cell/lazy.rs +++ b/library/core/src/cell/lazy.rs @@ -1,4 +1,5 @@ use super::UnsafeCell; +use crate::hint::unreachable_unchecked; use crate::ops::Deref; use crate::{fmt, mem}; @@ -78,11 +79,12 @@ impl T> LazyCell { /// assert_eq!(LazyCell::into_inner(lazy).ok(), Some("HELLO, WORLD!".to_string())); /// ``` #[unstable(feature = "lazy_cell_into_inner", issue = "125623")] - pub fn into_inner(this: Self) -> Result { + #[rustc_const_unstable(feature = "lazy_cell_into_inner", issue = "125623")] + pub const fn into_inner(this: Self) -> Result { match this.state.into_inner() { State::Init(data) => Ok(data), State::Uninit(f) => Err(f), - State::Poisoned => panic!("LazyCell instance has previously been poisoned"), + State::Poisoned => panic_poisoned(), } } @@ -114,7 +116,72 @@ impl T> LazyCell { State::Init(data) => data, // SAFETY: The state is uninitialized. State::Uninit(_) => unsafe { LazyCell::really_init(this) }, - State::Poisoned => panic!("LazyCell has previously been poisoned"), + State::Poisoned => panic_poisoned(), + } + } + + /// Forces the evaluation of this lazy value and returns a mutable reference to + /// the result. + /// + /// # Examples + /// + /// ``` + /// #![feature(lazy_get)] + /// use std::cell::LazyCell; + /// + /// let mut lazy = LazyCell::new(|| 92); + /// + /// let p = LazyCell::force_mut(&mut lazy); + /// assert_eq!(*p, 92); + /// *p = 44; + /// assert_eq!(*lazy, 44); + /// ``` + #[inline] + #[unstable(feature = "lazy_get", issue = "129333")] + pub fn force_mut(this: &mut LazyCell) -> &mut T { + #[cold] + /// # Safety + /// May only be called when the state is `Uninit`. + unsafe fn really_init_mut T>(state: &mut State) -> &mut T { + // INVARIANT: Always valid, but the value may not be dropped. + struct PoisonOnPanic(*mut State); + impl Drop for PoisonOnPanic { + #[inline] + fn drop(&mut self) { + // SAFETY: Invariant states it is valid, and we don't drop the old value. + unsafe { + self.0.write(State::Poisoned); + } + } + } + + let State::Uninit(f) = state else { + // `unreachable!()` here won't optimize out because the function is cold. + // SAFETY: Precondition. + unsafe { unreachable_unchecked() }; + }; + // SAFETY: We never drop the state after we read `f`, and we write a valid value back + // in any case, panic or success. `f` can't access the `LazyCell` because it is mutably + // borrowed. + let f = unsafe { core::ptr::read(f) }; + // INVARIANT: Initiated from mutable reference, don't drop because we read it. + let guard = PoisonOnPanic(state); + let data = f(); + // SAFETY: `PoisonOnPanic` invariant, and we don't drop the old value. + unsafe { + core::ptr::write(guard.0, State::Init(data)); + } + core::mem::forget(guard); + let State::Init(data) = state else { unreachable!() }; + data + } + + let state = this.state.get_mut(); + match state { + State::Init(data) => data, + // SAFETY: `state` is `Uninit`. + State::Uninit(_) => unsafe { really_init_mut(state) }, + State::Poisoned => panic_poisoned(), } } @@ -152,13 +219,55 @@ impl T> LazyCell { } impl LazyCell { + /// Returns a reference to the value if initialized, or `None` if not. + /// + /// # Examples + /// + /// ``` + /// #![feature(lazy_get)] + /// + /// use std::cell::LazyCell; + /// + /// let mut lazy = LazyCell::new(|| 92); + /// + /// assert_eq!(LazyCell::get_mut(&mut lazy), None); + /// let _ = LazyCell::force(&lazy); + /// *LazyCell::get_mut(&mut lazy).unwrap() = 44; + /// assert_eq!(*lazy, 44); + /// ``` #[inline] - fn get(&self) -> Option<&T> { + #[unstable(feature = "lazy_get", issue = "129333")] + pub fn get_mut(this: &mut LazyCell) -> Option<&mut T> { + let state = this.state.get_mut(); + match state { + State::Init(data) => Some(data), + _ => None, + } + } + + /// Returns a mutable reference to the value if initialized, or `None` if not. + /// + /// # Examples + /// + /// ``` + /// #![feature(lazy_get)] + /// + /// use std::cell::LazyCell; + /// + /// let lazy = LazyCell::new(|| 92); + /// + /// assert_eq!(LazyCell::get(&lazy), None); + /// let _ = LazyCell::force(&lazy); + /// assert_eq!(LazyCell::get(&lazy), Some(&92)); + /// ``` + #[inline] + #[unstable(feature = "lazy_get", issue = "129333")] + pub fn get(this: &LazyCell) -> Option<&T> { // SAFETY: // This is sound for the same reason as in `force`: once the state is // initialized, it will not be mutably accessed again, so this reference // will stay valid for the duration of the borrow to `self`. - let state = unsafe { &*self.state.get() }; + let state = unsafe { &*this.state.get() }; match state { State::Init(data) => Some(data), _ => None, @@ -188,10 +297,16 @@ impl Default for LazyCell { impl fmt::Debug for LazyCell { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { let mut d = f.debug_tuple("LazyCell"); - match self.get() { + match LazyCell::get(self) { Some(data) => d.field(data), None => d.field(&format_args!("")), }; d.finish() } } + +#[cold] +#[inline(never)] +const fn panic_poisoned() -> ! { + panic!("LazyCell instance has previously been poisoned") +} diff --git a/library/core/src/cell/once.rs b/library/core/src/cell/once.rs index 87df8a4e272e8..c14afe0f4761c 100644 --- a/library/core/src/cell/once.rs +++ b/library/core/src/cell/once.rs @@ -309,7 +309,8 @@ impl OnceCell { /// ``` #[inline] #[stable(feature = "once_cell", since = "1.70.0")] - #[rustc_const_unstable(feature = "const_cell_into_inner", issue = "78729")] + #[rustc_const_stable(feature = "const_cell_into_inner", since = "1.83.0")] + #[rustc_allow_const_fn_unstable(const_precise_live_drops)] pub const fn into_inner(self) -> Option { // Because `into_inner` takes `self` by value, the compiler statically verifies // that it is not currently borrowed. So it is safe to move out `Option`. diff --git a/library/core/src/char/convert.rs b/library/core/src/char/convert.rs index 698c85fcd1301..c4aa33afd6ad3 100644 --- a/library/core/src/char/convert.rs +++ b/library/core/src/char/convert.rs @@ -15,7 +15,7 @@ use crate::kani; #[must_use] #[inline] pub(super) const fn from_u32(i: u32) -> Option { - // FIXME: once Result::ok is const fn, use it here + // FIXME(const-hack): once Result::ok is const fn, use it here match char_try_from_u32(i) { Ok(c) => Some(c), Err(_) => None, diff --git a/library/core/src/char/methods.rs b/library/core/src/char/methods.rs index cbda18b5ac2eb..503499fa8f92f 100644 --- a/library/core/src/char/methods.rs +++ b/library/core/src/char/methods.rs @@ -1,6 +1,7 @@ //! impl char {} use super::*; +use crate::intrinsics::const_eval_select; use crate::slice; use crate::str::from_utf8_unchecked_mut; use crate::unicode::printable::is_printable; @@ -18,7 +19,6 @@ impl char { /// for you: /// /// ``` - /// #![feature(char_min)] /// let dist = u32::from(char::MAX) - u32::from(char::MIN); /// let size = (char::MIN..=char::MAX).count() as u32; /// assert!(size < dist); @@ -32,7 +32,6 @@ impl char { /// # Examples /// /// ``` - /// #![feature(char_min)] /// # fn something_which_returns_char() -> char { 'a' } /// let c: char = something_which_returns_char(); /// assert!(char::MIN <= c); @@ -40,7 +39,7 @@ impl char { /// let value_at_min = u32::from(char::MIN); /// assert_eq!(char::from_u32(value_at_min), Some('\0')); /// ``` - #[unstable(feature = "char_min", issue = "114298")] + #[stable(feature = "char_min", since = "1.83.0")] pub const MIN: char = '\0'; /// The highest valid code point a `char` can have, `'\u{10FFFF}'`. @@ -51,7 +50,6 @@ impl char { /// for you: /// /// ``` - /// #![feature(char_min)] /// let dist = u32::from(char::MAX) - u32::from(char::MIN); /// let size = (char::MIN..=char::MAX).count() as u32; /// assert!(size < dist); @@ -74,7 +72,7 @@ impl char { /// assert_eq!(char::from_u32(value_at_max + 1), None); /// ``` #[stable(feature = "assoc_char_consts", since = "1.52.0")] - pub const MAX: char = '\u{10ffff}'; + pub const MAX: char = '\u{10FFFF}'; /// `U+FFFD REPLACEMENT CHARACTER` (�) is used in Unicode to represent a /// decoding error. @@ -325,8 +323,9 @@ impl char { /// '1'.is_digit(37); /// ``` #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_char_is_digit", issue = "132241")] #[inline] - pub fn is_digit(self, radix: u32) -> bool { + pub const fn is_digit(self, radix: u32) -> bool { self.to_digit(radix).is_some() } @@ -389,7 +388,7 @@ impl char { // Force the 6th bit to be set to ensure ascii is lower case. digit = (self as u32 | 0b10_0000).wrapping_sub('a' as u32).saturating_add(10); } - // FIXME: once then_some is const fn, use it here + // FIXME(const-hack): once then_some is const fn, use it here if digit < radix { Some(digit) } else { None } } @@ -611,6 +610,7 @@ impl char { #[stable(feature = "rust1", since = "1.0.0")] #[rustc_const_stable(feature = "const_char_len_utf", since = "1.52.0")] #[inline] + #[must_use] pub const fn len_utf8(self) -> usize { len_utf8(self as u32) } @@ -642,9 +642,9 @@ impl char { #[stable(feature = "rust1", since = "1.0.0")] #[rustc_const_stable(feature = "const_char_len_utf", since = "1.52.0")] #[inline] + #[must_use] pub const fn len_utf16(self) -> usize { - let ch = self as u32; - if (ch & 0xFFFF) == ch { 1 } else { 2 } + len_utf16(self as u32) } /// Encodes this character as UTF-8 into the provided byte buffer, @@ -678,8 +678,9 @@ impl char { /// 'ß'.encode_utf8(&mut b); /// ``` #[stable(feature = "unicode_encode_char", since = "1.15.0")] + #[rustc_const_stable(feature = "const_char_encode_utf8", since = "1.83.0")] #[inline] - pub fn encode_utf8(self, dst: &mut [u8]) -> &mut str { + pub const fn encode_utf8(self, dst: &mut [u8]) -> &mut str { // SAFETY: `char` is not a surrogate, so this is valid UTF-8. unsafe { from_utf8_unchecked_mut(encode_utf8_raw(self as u32, dst)) } } @@ -713,8 +714,9 @@ impl char { /// '𝕊'.encode_utf16(&mut b); /// ``` #[stable(feature = "unicode_encode_char", since = "1.15.0")] + #[rustc_const_unstable(feature = "const_char_encode_utf16", issue = "130660")] #[inline] - pub fn encode_utf16(self, dst: &mut [u16]) -> &mut [u16] { + pub const fn encode_utf16(self, dst: &mut [u16]) -> &mut [u16] { encode_utf16_raw(self as u32, dst) } @@ -1283,8 +1285,9 @@ impl char { /// /// [`to_ascii_uppercase()`]: #method.to_ascii_uppercase #[stable(feature = "ascii_methods_on_intrinsics", since = "1.23.0")] + #[rustc_const_stable(feature = "const_make_ascii", since = "CURRENT_RUSTC_VERSION")] #[inline] - pub fn make_ascii_uppercase(&mut self) { + pub const fn make_ascii_uppercase(&mut self) { *self = self.to_ascii_uppercase(); } @@ -1308,8 +1311,9 @@ impl char { /// /// [`to_ascii_lowercase()`]: #method.to_ascii_lowercase #[stable(feature = "ascii_methods_on_intrinsics", since = "1.23.0")] + #[rustc_const_stable(feature = "const_make_ascii", since = "CURRENT_RUSTC_VERSION")] #[inline] - pub fn make_ascii_lowercase(&mut self) { + pub const fn make_ascii_lowercase(&mut self) { *self = self.to_ascii_lowercase(); } @@ -1740,19 +1744,23 @@ impl EscapeDebugExtArgs { } #[inline] +#[must_use] const fn len_utf8(code: u32) -> usize { - if code < MAX_ONE_B { - 1 - } else if code < MAX_TWO_B { - 2 - } else if code < MAX_THREE_B { - 3 - } else { - 4 + match code { + ..MAX_ONE_B => 1, + ..MAX_TWO_B => 2, + ..MAX_THREE_B => 3, + _ => 4, } } -/// Encodes a raw u32 value as UTF-8 into the provided byte buffer, +#[inline] +#[must_use] +const fn len_utf16(code: u32) -> usize { + if (code & 0xFFFF) == code { 1 } else { 2 } +} + +/// Encodes a raw `u32` value as UTF-8 into the provided byte buffer, /// and then returns the subslice of the buffer that contains the encoded character. /// /// Unlike `char::encode_utf8`, this method also handles codepoints in the surrogate range. @@ -1766,11 +1774,22 @@ const fn len_utf8(code: u32) -> usize { /// Panics if the buffer is not large enough. /// A buffer of length four is large enough to encode any `char`. #[unstable(feature = "char_internals", reason = "exposed only for libstd", issue = "none")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "const_char_encode_utf8", since = "1.83.0"))] #[doc(hidden)] #[inline] -pub fn encode_utf8_raw(code: u32, dst: &mut [u8]) -> &mut [u8] { +#[rustc_allow_const_fn_unstable(const_eval_select)] +pub const fn encode_utf8_raw(code: u32, dst: &mut [u8]) -> &mut [u8] { + const fn panic_at_const(_code: u32, _len: usize, _dst_len: usize) { + // Note that we cannot format in constant expressions. + panic!("encode_utf8: buffer does not have enough bytes to encode code point"); + } + fn panic_at_rt(code: u32, len: usize, dst_len: usize) { + panic!( + "encode_utf8: need {len} bytes to encode U+{code:04X} but buffer has just {dst_len}", + ); + } let len = len_utf8(code); - match (len, &mut dst[..]) { + match (len, &mut *dst) { (1, [a, ..]) => { *a = code as u8; } @@ -1789,17 +1808,14 @@ pub fn encode_utf8_raw(code: u32, dst: &mut [u8]) -> &mut [u8] { *c = (code >> 6 & 0x3F) as u8 | TAG_CONT; *d = (code & 0x3F) as u8 | TAG_CONT; } - _ => panic!( - "encode_utf8: need {} bytes to encode U+{:X}, but the buffer has {}", - len, - code, - dst.len(), - ), + // FIXME(const-hack): We would prefer to have streamlined panics when formatters become const-friendly. + _ => const_eval_select((code, len, dst.len()), panic_at_const, panic_at_rt), }; - &mut dst[..len] + // SAFETY: `<&mut [u8]>::as_mut_ptr` is guaranteed to return a valid pointer and `len` has been tested to be within bounds. + unsafe { slice::from_raw_parts_mut(dst.as_mut_ptr(), len) } } -/// Encodes a raw u32 value as UTF-16 into the provided `u16` buffer, +/// Encodes a raw `u32` value as UTF-16 into the provided `u16` buffer, /// and then returns the subslice of the buffer that contains the encoded character. /// /// Unlike `char::encode_utf16`, this method also handles codepoints in the surrogate range. @@ -1810,30 +1826,34 @@ pub fn encode_utf8_raw(code: u32, dst: &mut [u8]) -> &mut [u8] { /// Panics if the buffer is not large enough. /// A buffer of length 2 is large enough to encode any `char`. #[unstable(feature = "char_internals", reason = "exposed only for libstd", issue = "none")] +#[rustc_const_unstable(feature = "const_char_encode_utf16", issue = "130660")] #[doc(hidden)] #[inline] -pub fn encode_utf16_raw(mut code: u32, dst: &mut [u16]) -> &mut [u16] { - // SAFETY: each arm checks whether there are enough bits to write into - unsafe { - if (code & 0xFFFF) == code && !dst.is_empty() { - // The BMP falls through - *dst.get_unchecked_mut(0) = code as u16; - slice::from_raw_parts_mut(dst.as_mut_ptr(), 1) - } else if dst.len() >= 2 { - // Supplementary planes break into surrogates. +pub const fn encode_utf16_raw(mut code: u32, dst: &mut [u16]) -> &mut [u16] { + const fn panic_at_const(_code: u32, _len: usize, _dst_len: usize) { + // Note that we cannot format in constant expressions. + panic!("encode_utf16: buffer does not have enough bytes to encode code point"); + } + fn panic_at_rt(code: u32, len: usize, dst_len: usize) { + panic!( + "encode_utf16: need {len} bytes to encode U+{code:04X} but buffer has just {dst_len}", + ); + } + let len = len_utf16(code); + match (len, &mut *dst) { + (1, [a, ..]) => { + *a = code as u16; + } + (2, [a, b, ..]) => { code -= 0x1_0000; - *dst.get_unchecked_mut(0) = 0xD800 | ((code >> 10) as u16); - *dst.get_unchecked_mut(1) = 0xDC00 | ((code as u16) & 0x3FF); - slice::from_raw_parts_mut(dst.as_mut_ptr(), 2) - } else { - panic!( - "encode_utf16: need {} units to encode U+{:X}, but the buffer has {}", - char::from_u32_unchecked(code).len_utf16(), - code, - dst.len(), - ) + *a = (code >> 10) as u16 | 0xD800; + *b = (code & 0x3FF) as u16 | 0xDC00; } - } + // FIXME(const-hack): We would prefer to have streamlined panics when formatters become const-friendly. + _ => const_eval_select((code, len, dst.len()), panic_at_const, panic_at_rt), + }; + // SAFETY: `<&mut [u16]>::as_mut_ptr` is guaranteed to return a valid pointer and `len` has been tested to be within bounds. + unsafe { slice::from_raw_parts_mut(dst.as_mut_ptr(), len) } } #[cfg(kani)] diff --git a/library/core/src/char/mod.rs b/library/core/src/char/mod.rs index fa3c2075423bc..59fd7250e8f8e 100644 --- a/library/core/src/char/mod.rs +++ b/library/core/src/char/mod.rs @@ -18,7 +18,7 @@ //! functions that convert various types to `char`. #![allow(non_snake_case)] -#![stable(feature = "core_char", since = "1.2.0")] +#![stable(feature = "rust1", since = "1.0.0")] mod convert; mod decode; diff --git a/library/core/src/cmp.rs b/library/core/src/cmp.rs index a1ed993b7d9bf..5a3b9365cd220 100644 --- a/library/core/src/cmp.rs +++ b/library/core/src/cmp.rs @@ -275,49 +275,56 @@ pub macro PartialEq($item:item) { /// Trait for comparisons corresponding to [equivalence relations]( /// https://en.wikipedia.org/wiki/Equivalence_relation). /// -/// This means, that in addition to `a == b` and `a != b` being strict inverses, -/// the relation must be (for all `a`, `b` and `c`): +/// The primary difference to [`PartialEq`] is the additional requirement for reflexivity. A type +/// that implements [`PartialEq`] guarantees that for all `a`, `b` and `c`: /// -/// - reflexive: `a == a`; -/// - symmetric: `a == b` implies `b == a` (required by `PartialEq` as well); and -/// - transitive: `a == b` and `b == c` implies `a == c` (required by `PartialEq` as well). +/// - symmetric: `a == b` implies `b == a` and `a != b` implies `!(a == b)` +/// - transitive: `a == b` and `b == c` implies `a == c` /// -/// This property cannot be checked by the compiler, and therefore `Eq` implies -/// [`PartialEq`], and has no extra methods. +/// `Eq`, which builds on top of [`PartialEq`] also implies: +/// +/// - reflexive: `a == a` +/// +/// This property cannot be checked by the compiler, and therefore `Eq` is a trait without methods. /// /// Violating this property is a logic error. The behavior resulting from a logic error is not /// specified, but users of the trait must ensure that such logic errors do *not* result in /// undefined behavior. This means that `unsafe` code **must not** rely on the correctness of these /// methods. /// -/// Implement `Eq` in addition to `PartialEq` if it's guaranteed that -/// `PartialEq::eq(a, a)` always returns `true` (reflexivity), in addition to -/// the symmetric and transitive properties already required by `PartialEq`. +/// Floating point types such as [`f32`] and [`f64`] implement only [`PartialEq`] but *not* `Eq` +/// because `NaN` != `NaN`. /// /// ## Derivable /// -/// This trait can be used with `#[derive]`. When `derive`d, because `Eq` has -/// no extra methods, it is only informing the compiler that this is an -/// equivalence relation rather than a partial equivalence relation. Note that -/// the `derive` strategy requires all fields are `Eq`, which isn't +/// This trait can be used with `#[derive]`. When `derive`d, because `Eq` has no extra methods, it +/// is only informing the compiler that this is an equivalence relation rather than a partial +/// equivalence relation. Note that the `derive` strategy requires all fields are `Eq`, which isn't /// always desired. /// /// ## How can I implement `Eq`? /// -/// If you cannot use the `derive` strategy, specify that your type implements -/// `Eq`, which has no methods: +/// If you cannot use the `derive` strategy, specify that your type implements `Eq`, which has no +/// extra methods: /// /// ``` -/// enum BookFormat { Paperback, Hardback, Ebook } +/// enum BookFormat { +/// Paperback, +/// Hardback, +/// Ebook, +/// } +/// /// struct Book { /// isbn: i32, /// format: BookFormat, /// } +/// /// impl PartialEq for Book { /// fn eq(&self, other: &Self) -> bool { /// self.isbn == other.isbn /// } /// } +/// /// impl Eq for Book {} /// ``` #[doc(alias = "==")] @@ -325,11 +332,9 @@ pub macro PartialEq($item:item) { #[stable(feature = "rust1", since = "1.0.0")] #[rustc_diagnostic_item = "Eq"] pub trait Eq: PartialEq { - // this method is used solely by #[derive(Eq)] to assert - // that every component of a type implements `Eq` - // itself. The current deriving infrastructure means doing this - // assertion without using a method on this trait is nearly - // impossible. + // this method is used solely by `impl Eq or #[derive(Eq)]` to assert that every component of a + // type implements `Eq` itself. The current deriving infrastructure means doing this assertion + // without using a method on this trait is nearly impossible. // // This should never be implemented by hand. #[doc(hidden)] @@ -375,7 +380,7 @@ pub struct AssertParamIsEq { #[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Debug, Hash)] #[stable(feature = "rust1", since = "1.0.0")] // This is a lang item only so that `BinOp::Cmp` in MIR can return it. -// It has no special behaviour, but does require that the three variants +// It has no special behavior, but does require that the three variants // `Less`/`Equal`/`Greater` remain `-1_i8`/`0_i8`/`+1_i8` respectively. #[lang = "Ordering"] #[repr(i8)] @@ -693,17 +698,14 @@ impl Clone for Reverse { /// Trait for types that form a [total order](https://en.wikipedia.org/wiki/Total_order). /// -/// Implementations must be consistent with the [`PartialOrd`] implementation, and ensure -/// `max`, `min`, and `clamp` are consistent with `cmp`: +/// Implementations must be consistent with the [`PartialOrd`] implementation, and ensure `max`, +/// `min`, and `clamp` are consistent with `cmp`: /// /// - `partial_cmp(a, b) == Some(cmp(a, b))`. /// - `max(a, b) == max_by(a, b, cmp)` (ensured by the default implementation). /// - `min(a, b) == min_by(a, b, cmp)` (ensured by the default implementation). -/// - For `a.clamp(min, max)`, see the [method docs](#method.clamp) -/// (ensured by the default implementation). -/// -/// It's easy to accidentally make `cmp` and `partial_cmp` disagree by -/// deriving some of the traits and manually implementing others. +/// - For `a.clamp(min, max)`, see the [method docs](#method.clamp) (ensured by the default +/// implementation). /// /// Violating these requirements is a logic error. The behavior resulting from a logic error is not /// specified, but users of the trait must ensure that such logic errors do *not* result in @@ -712,15 +714,14 @@ impl Clone for Reverse { /// /// ## Corollaries /// -/// From the above and the requirements of `PartialOrd`, it follows that for -/// all `a`, `b` and `c`: +/// From the above and the requirements of `PartialOrd`, it follows that for all `a`, `b` and `c`: /// /// - exactly one of `a < b`, `a == b` or `a > b` is true; and -/// - `<` is transitive: `a < b` and `b < c` implies `a < c`. The same must hold for both `==` and `>`. +/// - `<` is transitive: `a < b` and `b < c` implies `a < c`. The same must hold for both `==` and +/// `>`. /// -/// Mathematically speaking, the `<` operator defines a strict [weak order]. In -/// cases where `==` conforms to mathematical equality, it also defines a -/// strict [total order]. +/// Mathematically speaking, the `<` operator defines a strict [weak order]. In cases where `==` +/// conforms to mathematical equality, it also defines a strict [total order]. /// /// [weak order]: https://en.wikipedia.org/wiki/Weak_ordering /// [total order]: https://en.wikipedia.org/wiki/Total_order @@ -730,13 +731,12 @@ impl Clone for Reverse { /// This trait can be used with `#[derive]`. /// /// When `derive`d on structs, it will produce a -/// [lexicographic](https://en.wikipedia.org/wiki/Lexicographic_order) ordering -/// based on the top-to-bottom declaration order of the struct's members. +/// [lexicographic](https://en.wikipedia.org/wiki/Lexicographic_order) ordering based on the +/// top-to-bottom declaration order of the struct's members. /// -/// When `derive`d on enums, variants are ordered primarily by their discriminants. -/// Secondarily, they are ordered by their fields. -/// By default, the discriminant is smallest for variants at the top, and -/// largest for variants at the bottom. Here's an example: +/// When `derive`d on enums, variants are ordered primarily by their discriminants. Secondarily, +/// they are ordered by their fields. By default, the discriminant is smallest for variants at the +/// top, and largest for variants at the bottom. Here's an example: /// /// ``` /// #[derive(PartialEq, Eq, PartialOrd, Ord)] @@ -748,8 +748,7 @@ impl Clone for Reverse { /// assert!(E::Top < E::Bottom); /// ``` /// -/// However, manually setting the discriminants can override this default -/// behavior: +/// However, manually setting the discriminants can override this default behavior: /// /// ``` /// #[derive(PartialEq, Eq, PartialOrd, Ord)] @@ -765,51 +764,178 @@ impl Clone for Reverse { /// /// Lexicographical comparison is an operation with the following properties: /// - Two sequences are compared element by element. -/// - The first mismatching element defines which sequence is lexicographically less or greater than the other. -/// - If one sequence is a prefix of another, the shorter sequence is lexicographically less than the other. -/// - If two sequences have equivalent elements and are of the same length, then the sequences are lexicographically equal. +/// - The first mismatching element defines which sequence is lexicographically less or greater +/// than the other. +/// - If one sequence is a prefix of another, the shorter sequence is lexicographically less than +/// the other. +/// - If two sequences have equivalent elements and are of the same length, then the sequences are +/// lexicographically equal. /// - An empty sequence is lexicographically less than any non-empty sequence. /// - Two empty sequences are lexicographically equal. /// /// ## How can I implement `Ord`? /// -/// `Ord` requires that the type also be [`PartialOrd`] and [`Eq`] (which requires [`PartialEq`]). +/// `Ord` requires that the type also be [`PartialOrd`], [`PartialEq`], and [`Eq`]. /// -/// Then you must define an implementation for [`cmp`]. You may find it useful to use -/// [`cmp`] on your type's fields. +/// Because `Ord` implies a stronger ordering relationship than [`PartialOrd`], and both `Ord` and +/// [`PartialOrd`] must agree, you must choose how to implement `Ord` **first**. You can choose to +/// derive it, or implement it manually. If you derive it, you should derive all four traits. If you +/// implement it manually, you should manually implement all four traits, based on the +/// implementation of `Ord`. /// -/// Here's an example where you want to sort people by height only, disregarding `id` -/// and `name`: +/// Here's an example where you want to define the `Character` comparison by `health` and +/// `experience` only, disregarding the field `mana`: /// /// ``` /// use std::cmp::Ordering; /// -/// #[derive(Eq)] -/// struct Person { -/// id: u32, -/// name: String, -/// height: u32, +/// struct Character { +/// health: u32, +/// experience: u32, +/// mana: f32, /// } /// -/// impl Ord for Person { -/// fn cmp(&self, other: &Self) -> Ordering { -/// self.height.cmp(&other.height) +/// impl Ord for Character { +/// fn cmp(&self, other: &Self) -> std::cmp::Ordering { +/// self.experience +/// .cmp(&other.experience) +/// .then(self.health.cmp(&other.health)) /// } /// } /// -/// impl PartialOrd for Person { +/// impl PartialOrd for Character { /// fn partial_cmp(&self, other: &Self) -> Option { /// Some(self.cmp(other)) /// } /// } /// -/// impl PartialEq for Person { +/// impl PartialEq for Character { /// fn eq(&self, other: &Self) -> bool { -/// self.height == other.height +/// self.health == other.health && self.experience == other.experience +/// } +/// } +/// +/// impl Eq for Character {} +/// ``` +/// +/// If all you need is to `slice::sort` a type by a field value, it can be simpler to use +/// `slice::sort_by_key`. +/// +/// ## Examples of incorrect `Ord` implementations +/// +/// ``` +/// use std::cmp::Ordering; +/// +/// #[derive(Debug)] +/// struct Character { +/// health: f32, +/// } +/// +/// impl Ord for Character { +/// fn cmp(&self, other: &Self) -> std::cmp::Ordering { +/// if self.health < other.health { +/// Ordering::Less +/// } else if self.health > other.health { +/// Ordering::Greater +/// } else { +/// Ordering::Equal +/// } +/// } +/// } +/// +/// impl PartialOrd for Character { +/// fn partial_cmp(&self, other: &Self) -> Option { +/// Some(self.cmp(other)) +/// } +/// } +/// +/// impl PartialEq for Character { +/// fn eq(&self, other: &Self) -> bool { +/// self.health == other.health +/// } +/// } +/// +/// impl Eq for Character {} +/// +/// let a = Character { health: 4.5 }; +/// let b = Character { health: f32::NAN }; +/// +/// // Mistake: floating-point values do not form a total order and using the built-in comparison +/// // operands to implement `Ord` irregardless of that reality does not change it. Use +/// // `f32::total_cmp` if you need a total order for floating-point values. +/// +/// // Reflexivity requirement of `Ord` is not given. +/// assert!(a == a); +/// assert!(b != b); +/// +/// // Antisymmetry requirement of `Ord` is not given. Only one of a < c and c < a is allowed to be +/// // true, not both or neither. +/// assert_eq!((a < b) as u8 + (b < a) as u8, 0); +/// ``` +/// +/// ``` +/// use std::cmp::Ordering; +/// +/// #[derive(Debug)] +/// struct Character { +/// health: u32, +/// experience: u32, +/// } +/// +/// impl PartialOrd for Character { +/// fn partial_cmp(&self, other: &Self) -> Option { +/// Some(self.cmp(other)) +/// } +/// } +/// +/// impl Ord for Character { +/// fn cmp(&self, other: &Self) -> std::cmp::Ordering { +/// if self.health < 50 { +/// self.health.cmp(&other.health) +/// } else { +/// self.experience.cmp(&other.experience) +/// } +/// } +/// } +/// +/// // For performance reasons implementing `PartialEq` this way is not the idiomatic way, but it +/// // ensures consistent behavior between `PartialEq`, `PartialOrd` and `Ord` in this example. +/// impl PartialEq for Character { +/// fn eq(&self, other: &Self) -> bool { +/// self.cmp(other) == Ordering::Equal /// } /// } +/// +/// impl Eq for Character {} +/// +/// let a = Character { +/// health: 3, +/// experience: 5, +/// }; +/// let b = Character { +/// health: 10, +/// experience: 77, +/// }; +/// let c = Character { +/// health: 143, +/// experience: 2, +/// }; +/// +/// // Mistake: The implementation of `Ord` compares different fields depending on the value of +/// // `self.health`, the resulting order is not total. +/// +/// // Transitivity requirement of `Ord` is not given. If a is smaller than b and b is smaller than +/// // c, by transitive property a must also be smaller than c. +/// assert!(a < b && b < c && c < a); +/// +/// // Antisymmetry requirement of `Ord` is not given. Only one of a < c and c < a is allowed to be +/// // true, not both or neither. +/// assert_eq!((a < c) as u8 + (c < a) as u8, 2); /// ``` /// +/// The documentation of [`PartialOrd`] contains further examples, for example it's wrong for +/// [`PartialOrd`] and [`PartialEq`] to disagree. +/// /// [`cmp`]: Ord::cmp #[doc(alias = "<")] #[doc(alias = ">")] @@ -901,7 +1027,6 @@ pub trait Ord: Eq + PartialOrd { fn clamp(self, min: Self, max: Self) -> Self where Self: Sized, - Self: PartialOrd, { assert!(min <= max); if self < min { @@ -925,8 +1050,12 @@ pub macro Ord($item:item) { /// Trait for types that form a [partial order](https://en.wikipedia.org/wiki/Partial_order). /// -/// The `lt`, `le`, `gt`, and `ge` methods of this trait can be called using -/// the `<`, `<=`, `>`, and `>=` operators, respectively. +/// The `lt`, `le`, `gt`, and `ge` methods of this trait can be called using the `<`, `<=`, `>`, and +/// `>=` operators, respectively. +/// +/// This trait should **only** contain the comparison logic for a type **if one plans on only +/// implementing `PartialOrd` but not [`Ord`]**. Otherwise the comparison logic should be in [`Ord`] +/// and this trait implemented with `Some(self.cmp(other))`. /// /// The methods of this trait must be consistent with each other and with those of [`PartialEq`]. /// The following conditions must hold: @@ -938,26 +1067,25 @@ pub macro Ord($item:item) { /// 5. `a >= b` if and only if `a > b || a == b` /// 6. `a != b` if and only if `!(a == b)`. /// -/// Conditions 2–5 above are ensured by the default implementation. -/// Condition 6 is already ensured by [`PartialEq`]. +/// Conditions 2–5 above are ensured by the default implementation. Condition 6 is already ensured +/// by [`PartialEq`]. /// /// If [`Ord`] is also implemented for `Self` and `Rhs`, it must also be consistent with -/// `partial_cmp` (see the documentation of that trait for the exact requirements). It's -/// easy to accidentally make them disagree by deriving some of the traits and manually -/// implementing others. +/// `partial_cmp` (see the documentation of that trait for the exact requirements). It's easy to +/// accidentally make them disagree by deriving some of the traits and manually implementing others. /// -/// The comparison relations must satisfy the following conditions -/// (for all `a`, `b`, `c` of type `A`, `B`, `C`): +/// The comparison relations must satisfy the following conditions (for all `a`, `b`, `c` of type +/// `A`, `B`, `C`): /// -/// - **Transitivity**: if `A: PartialOrd` and `B: PartialOrd` and `A: -/// PartialOrd`, then `a < b` and `b < c` implies `a < c`. The same must hold for both `==` and `>`. -/// This must also work for longer chains, such as when `A: PartialOrd`, `B: PartialOrd`, -/// `C: PartialOrd`, and `A: PartialOrd` all exist. -/// - **Duality**: if `A: PartialOrd` and `B: PartialOrd`, then `a < b` if and only if `b > a`. +/// - **Transitivity**: if `A: PartialOrd` and `B: PartialOrd` and `A: PartialOrd`, then `a +/// < b` and `b < c` implies `a < c`. The same must hold for both `==` and `>`. This must also +/// work for longer chains, such as when `A: PartialOrd`, `B: PartialOrd`, `C: +/// PartialOrd`, and `A: PartialOrd` all exist. +/// - **Duality**: if `A: PartialOrd` and `B: PartialOrd`, then `a < b` if and only if `b > +/// a`. /// -/// Note that the `B: PartialOrd` (dual) and `A: PartialOrd` -/// (transitive) impls are not forced to exist, but these requirements apply -/// whenever they do exist. +/// Note that the `B: PartialOrd` (dual) and `A: PartialOrd` (transitive) impls are not forced +/// to exist, but these requirements apply whenever they do exist. /// /// Violating these requirements is a logic error. The behavior resulting from a logic error is not /// specified, but users of the trait must ensure that such logic errors do *not* result in @@ -993,12 +1121,10 @@ pub macro Ord($item:item) { /// /// ## Strict and non-strict partial orders /// -/// The `<` and `>` operators behave according to a *strict* partial order. -/// However, `<=` and `>=` do **not** behave according to a *non-strict* -/// partial order. -/// That is because mathematically, a non-strict partial order would require -/// reflexivity, i.e. `a <= a` would need to be true for every `a`. This isn't -/// always the case for types that implement `PartialOrd`, for example: +/// The `<` and `>` operators behave according to a *strict* partial order. However, `<=` and `>=` +/// do **not** behave according to a *non-strict* partial order. That is because mathematically, a +/// non-strict partial order would require reflexivity, i.e. `a <= a` would need to be true for +/// every `a`. This isn't always the case for types that implement `PartialOrd`, for example: /// /// ``` /// let a = f64::sqrt(-1.0); @@ -1010,13 +1136,12 @@ pub macro Ord($item:item) { /// This trait can be used with `#[derive]`. /// /// When `derive`d on structs, it will produce a -/// [lexicographic](https://en.wikipedia.org/wiki/Lexicographic_order) ordering -/// based on the top-to-bottom declaration order of the struct's members. +/// [lexicographic](https://en.wikipedia.org/wiki/Lexicographic_order) ordering based on the +/// top-to-bottom declaration order of the struct's members. /// -/// When `derive`d on enums, variants are primarily ordered by their discriminants. -/// Secondarily, they are ordered by their fields. -/// By default, the discriminant is smallest for variants at the top, and -/// largest for variants at the bottom. Here's an example: +/// When `derive`d on enums, variants are primarily ordered by their discriminants. Secondarily, +/// they are ordered by their fields. By default, the discriminant is smallest for variants at the +/// top, and largest for variants at the bottom. Here's an example: /// /// ``` /// #[derive(PartialEq, PartialOrd)] @@ -1028,8 +1153,7 @@ pub macro Ord($item:item) { /// assert!(E::Top < E::Bottom); /// ``` /// -/// However, manually setting the discriminants can override this default -/// behavior: +/// However, manually setting the discriminants can override this default behavior: /// /// ``` /// #[derive(PartialEq, PartialOrd)] @@ -1047,8 +1171,8 @@ pub macro Ord($item:item) { /// generated from default implementations. /// /// However it remains possible to implement the others separately for types which do not have a -/// total order. For example, for floating point numbers, `NaN < 0 == false` and `NaN >= 0 == -/// false` (cf. IEEE 754-2008 section 5.11). +/// total order. For example, for floating point numbers, `NaN < 0 == false` and `NaN >= 0 == false` +/// (cf. IEEE 754-2008 section 5.11). /// /// `PartialOrd` requires your type to be [`PartialEq`]. /// @@ -1057,7 +1181,6 @@ pub macro Ord($item:item) { /// ``` /// use std::cmp::Ordering; /// -/// #[derive(Eq)] /// struct Person { /// id: u32, /// name: String, @@ -1081,11 +1204,13 @@ pub macro Ord($item:item) { /// self.height == other.height /// } /// } +/// +/// impl Eq for Person {} /// ``` /// -/// You may also find it useful to use [`partial_cmp`] on your type's fields. Here -/// is an example of `Person` types who have a floating-point `height` field that -/// is the only field to be used for sorting: +/// You may also find it useful to use [`partial_cmp`] on your type's fields. Here is an example of +/// `Person` types who have a floating-point `height` field that is the only field to be used for +/// sorting: /// /// ``` /// use std::cmp::Ordering; @@ -1109,6 +1234,38 @@ pub macro Ord($item:item) { /// } /// ``` /// +/// ## Examples of incorrect `PartialOrd` implementations +/// +/// ``` +/// use std::cmp::Ordering; +/// +/// #[derive(PartialEq, Debug)] +/// struct Character { +/// health: u32, +/// experience: u32, +/// } +/// +/// impl PartialOrd for Character { +/// fn partial_cmp(&self, other: &Self) -> Option { +/// Some(self.health.cmp(&other.health)) +/// } +/// } +/// +/// let a = Character { +/// health: 10, +/// experience: 5, +/// }; +/// let b = Character { +/// health: 10, +/// experience: 77, +/// }; +/// +/// // Mistake: `PartialEq` and `PartialOrd` disagree with each other. +/// +/// assert_eq!(a.partial_cmp(&b).unwrap(), Ordering::Equal); // a == b according to `PartialOrd`. +/// assert_ne!(a, b); // a != b according to `PartialEq`. +/// ``` +/// /// # Examples /// /// ``` diff --git a/library/core/src/error.rs b/library/core/src/error.rs index cac00b37d1fa7..95a39cc3aed38 100644 --- a/library/core/src/error.rs +++ b/library/core/src/error.rs @@ -335,16 +335,17 @@ impl dyn Error { #[unstable(feature = "error_iter", issue = "58520")] #[inline] pub fn sources(&self) -> Source<'_> { - // You may think this method would be better in the Error trait, and you'd be right. - // Unfortunately that doesn't work, not because of the object safety rules but because we - // save a reference to self in Sources below as a trait object. If this method was - // declared in Error, then self would have the type &T where T is some concrete type which - // implements Error. We would need to coerce self to have type &dyn Error, but that requires - // that Self has a known size (i.e., Self: Sized). We can't put that bound on Error - // since that would forbid Error trait objects, and we can't put that bound on the method - // because that means the method can't be called on trait objects (we'd also need the - // 'static bound, but that isn't allowed because methods with bounds on Self other than - // Sized are not object-safe). Requiring an Unsize bound is not backwards compatible. + // You may think this method would be better in the `Error` trait, and you'd be right. + // Unfortunately that doesn't work, not because of the dyn-incompatibility rules but + // because we save a reference to `self` in `Source`s below as a trait object. + // If this method was declared in `Error`, then `self` would have the type `&T` where + // `T` is some concrete type which implements `Error`. We would need to coerce `self` + // to have type `&dyn Error`, but that requires that `Self` has a known size + // (i.e., `Self: Sized`). We can't put that bound on `Error` since that would forbid + // `Error` trait objects, and we can't put that bound on the method because that means + // the method can't be called on trait objects (we'd also need the `'static` bound, + // but that isn't allowed because methods with bounds on `Self` other than `Sized` are + // dyn-incompatible). Requiring an `Unsize` bound is not backwards compatible. Source { current: Some(self) } } diff --git a/library/core/src/escape.rs b/library/core/src/escape.rs index b213cc2b9167c..0685f525dca83 100644 --- a/library/core/src/escape.rs +++ b/library/core/src/escape.rs @@ -18,38 +18,106 @@ const fn backslash(a: ascii::Char) -> ([ascii::Char; N], Range(byte: u8) -> ([ascii::Char; N], Range) { + const { assert!(N >= 4) }; + + let mut output = [ascii::Char::Null; N]; + + let hi = HEX_DIGITS[(byte >> 4) as usize]; + let lo = HEX_DIGITS[(byte & 0xf) as usize]; + + output[0] = ascii::Char::ReverseSolidus; + output[1] = ascii::Char::SmallX; + output[2] = hi; + output[3] = lo; + + (output, 0..4) +} + +#[inline] +const fn verbatim(a: ascii::Char) -> ([ascii::Char; N], Range) { + const { assert!(N >= 1) }; + + let mut output = [ascii::Char::Null; N]; + + output[0] = a; + + (output, 0..1) +} + /// Escapes an ASCII character. /// /// Returns a buffer and the length of the escaped representation. const fn escape_ascii(byte: u8) -> ([ascii::Char; N], Range) { const { assert!(N >= 4) }; - match byte { - b'\t' => backslash(ascii::Char::SmallT), - b'\r' => backslash(ascii::Char::SmallR), - b'\n' => backslash(ascii::Char::SmallN), - b'\\' => backslash(ascii::Char::ReverseSolidus), - b'\'' => backslash(ascii::Char::Apostrophe), - b'\"' => backslash(ascii::Char::QuotationMark), - byte => { - let mut output = [ascii::Char::Null; N]; - - if let Some(c) = byte.as_ascii() - && !byte.is_ascii_control() - { - output[0] = c; - (output, 0..1) - } else { - let hi = HEX_DIGITS[(byte >> 4) as usize]; - let lo = HEX_DIGITS[(byte & 0xf) as usize]; + #[cfg(feature = "optimize_for_size")] + { + match byte { + b'\t' => backslash(ascii::Char::SmallT), + b'\r' => backslash(ascii::Char::SmallR), + b'\n' => backslash(ascii::Char::SmallN), + b'\\' => backslash(ascii::Char::ReverseSolidus), + b'\'' => backslash(ascii::Char::Apostrophe), + b'"' => backslash(ascii::Char::QuotationMark), + 0x00..=0x1F | 0x7F => hex_escape(byte), + _ => match ascii::Char::from_u8(byte) { + Some(a) => verbatim(a), + None => hex_escape(byte), + }, + } + } + + #[cfg(not(feature = "optimize_for_size"))] + { + /// Lookup table helps us determine how to display character. + /// + /// Since ASCII characters will always be 7 bits, we can exploit this to store the 8th bit to + /// indicate whether the result is escaped or unescaped. + /// + /// We additionally use 0x80 (escaped NUL character) to indicate hex-escaped bytes, since + /// escaped NUL will not occur. + const LOOKUP: [u8; 256] = { + let mut arr = [0; 256]; + let mut idx = 0; + while idx <= 255 { + arr[idx] = match idx as u8 { + // use 8th bit to indicate escaped + b'\t' => 0x80 | b't', + b'\r' => 0x80 | b'r', + b'\n' => 0x80 | b'n', + b'\\' => 0x80 | b'\\', + b'\'' => 0x80 | b'\'', + b'"' => 0x80 | b'"', + + // use NUL to indicate hex-escaped + 0x00..=0x1F | 0x7F..=0xFF => 0x80 | b'\0', + + idx => idx, + }; + idx += 1; + } + arr + }; - output[0] = ascii::Char::ReverseSolidus; - output[1] = ascii::Char::SmallX; - output[2] = hi; - output[3] = lo; + let lookup = LOOKUP[byte as usize]; - (output, 0..4) + // 8th bit indicates escape + let lookup_escaped = lookup & 0x80 != 0; + + // SAFETY: We explicitly mask out the eighth bit to get a 7-bit ASCII character. + let lookup_ascii = unsafe { ascii::Char::from_u8_unchecked(lookup & 0x7F) }; + + if lookup_escaped { + // NUL indicates hex-escaped + if matches!(lookup_ascii, ascii::Char::Null) { + hex_escape(byte) + } else { + backslash(lookup_ascii) } + } else { + verbatim(lookup_ascii) } } } diff --git a/library/core/src/ffi/c_str.rs b/library/core/src/ffi/c_str.rs index 7808d42ab5de4..93dd351b02958 100644 --- a/library/core/src/ffi/c_str.rs +++ b/library/core/src/ffi/c_str.rs @@ -5,7 +5,7 @@ use crate::error::Error; use crate::ffi::c_char; use crate::iter::FusedIterator; use crate::marker::PhantomData; -use crate::ptr::{addr_of, NonNull}; +use crate::ptr::NonNull; use crate::slice::memchr; use crate::{fmt, intrinsics, ops, slice, str}; @@ -137,11 +137,11 @@ enum FromBytesWithNulErrorKind { // FIXME: const stability attributes should not be required here, I think impl FromBytesWithNulError { - #[rustc_const_stable(feature = "const_cstr_methods", since = "1.72.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_cstr_methods", since = "1.72.0"))] const fn interior_nul(pos: usize) -> FromBytesWithNulError { FromBytesWithNulError { kind: FromBytesWithNulErrorKind::InteriorNul(pos) } } - #[rustc_const_stable(feature = "const_cstr_methods", since = "1.72.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_cstr_methods", since = "1.72.0"))] const fn not_nul_terminated() -> FromBytesWithNulError { FromBytesWithNulError { kind: FromBytesWithNulErrorKind::NotNulTerminated } } @@ -464,7 +464,9 @@ impl CStr { /// behavior when `ptr` is used inside the `unsafe` block: /// /// ```no_run - /// # #![allow(unused_must_use)] #![allow(temporary_cstring_as_ptr)] + /// # #![allow(unused_must_use)] + /// # #![cfg_attr(bootstrap, expect(temporary_cstring_as_ptr))] + /// # #![cfg_attr(not(bootstrap), expect(dangling_pointers_from_temporaries))] /// use std::ffi::CString; /// /// // Do not do this: @@ -623,7 +625,7 @@ impl CStr { pub const fn to_bytes_with_nul(&self) -> &[u8] { // SAFETY: Transmuting a slice of `c_char`s to a slice of `u8`s // is safe on all supported targets. - unsafe { &*(addr_of!(self.inner) as *const [u8]) } + unsafe { &*((&raw const self.inner) as *const [u8]) } } /// Iterates over the bytes in this C string. @@ -730,7 +732,7 @@ impl AsRef for CStr { /// located within `isize::MAX` from `ptr`. #[inline] #[unstable(feature = "cstr_internals", issue = "none")] -#[rustc_const_stable(feature = "const_cstr_from_ptr", since = "1.81.0")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "const_cstr_from_ptr", since = "1.81.0"))] #[rustc_allow_const_fn_unstable(const_eval_select)] const unsafe fn strlen(ptr: *const c_char) -> usize { const fn strlen_ct(s: *const c_char) -> usize { diff --git a/library/core/src/fmt/builders.rs b/library/core/src/fmt/builders.rs index c7c462a4df1f5..1862be0e86c5d 100644 --- a/library/core/src/fmt/builders.rs +++ b/library/core/src/fmt/builders.rs @@ -366,8 +366,6 @@ impl<'a, 'b: 'a> DebugTuple<'a, 'b> { /// # Examples /// /// ``` - /// #![feature(debug_more_non_exhaustive)] - /// /// use std::fmt; /// /// struct Foo(i32, String); @@ -385,7 +383,7 @@ impl<'a, 'b: 'a> DebugTuple<'a, 'b> { /// "Foo(10, ..)", /// ); /// ``` - #[unstable(feature = "debug_more_non_exhaustive", issue = "127942")] + #[stable(feature = "debug_more_non_exhaustive", since = "1.83.0")] pub fn finish_non_exhaustive(&mut self) -> fmt::Result { self.result = self.result.and_then(|_| { if self.fields > 0 { @@ -606,8 +604,6 @@ impl<'a, 'b: 'a> DebugSet<'a, 'b> { /// # Examples /// /// ``` - /// #![feature(debug_more_non_exhaustive)] - /// /// use std::fmt; /// /// struct Foo(Vec); @@ -630,7 +626,7 @@ impl<'a, 'b: 'a> DebugSet<'a, 'b> { /// "{1, 2, ..}", /// ); /// ``` - #[unstable(feature = "debug_more_non_exhaustive", issue = "127942")] + #[stable(feature = "debug_more_non_exhaustive", since = "1.83.0")] pub fn finish_non_exhaustive(&mut self) -> fmt::Result { self.inner.result = self.inner.result.and_then(|_| { if self.inner.has_fields { @@ -800,8 +796,6 @@ impl<'a, 'b: 'a> DebugList<'a, 'b> { /// # Examples /// /// ``` - /// #![feature(debug_more_non_exhaustive)] - /// /// use std::fmt; /// /// struct Foo(Vec); @@ -824,7 +818,7 @@ impl<'a, 'b: 'a> DebugList<'a, 'b> { /// "[1, 2, ..]", /// ); /// ``` - #[unstable(feature = "debug_more_non_exhaustive", issue = "127942")] + #[stable(feature = "debug_more_non_exhaustive", since = "1.83.0")] pub fn finish_non_exhaustive(&mut self) -> fmt::Result { self.inner.result.and_then(|_| { if self.inner.has_fields { @@ -1126,8 +1120,6 @@ impl<'a, 'b: 'a> DebugMap<'a, 'b> { /// # Examples /// /// ``` - /// #![feature(debug_more_non_exhaustive)] - /// /// use std::fmt; /// /// struct Foo(Vec<(String, i32)>); @@ -1154,7 +1146,7 @@ impl<'a, 'b: 'a> DebugMap<'a, 'b> { /// r#"{"A": 10, "B": 11, ..}"#, /// ); /// ``` - #[unstable(feature = "debug_more_non_exhaustive", issue = "127942")] + #[stable(feature = "debug_more_non_exhaustive", since = "1.83.0")] pub fn finish_non_exhaustive(&mut self) -> fmt::Result { self.result = self.result.and_then(|_| { assert!(!self.has_key, "attempted to finish a map with a partial entry"); diff --git a/library/core/src/fmt/float.rs b/library/core/src/fmt/float.rs index 20ea0352c2dce..c70dbf54304de 100644 --- a/library/core/src/fmt/float.rs +++ b/library/core/src/fmt/float.rs @@ -196,39 +196,40 @@ where } macro_rules! floating { - ($ty:ident) => { - #[stable(feature = "rust1", since = "1.0.0")] - impl Debug for $ty { - fn fmt(&self, fmt: &mut Formatter<'_>) -> Result { - float_to_general_debug(fmt, self) + ($($ty:ident)*) => { + $( + #[stable(feature = "rust1", since = "1.0.0")] + impl Debug for $ty { + fn fmt(&self, fmt: &mut Formatter<'_>) -> Result { + float_to_general_debug(fmt, self) + } } - } - #[stable(feature = "rust1", since = "1.0.0")] - impl Display for $ty { - fn fmt(&self, fmt: &mut Formatter<'_>) -> Result { - float_to_decimal_display(fmt, self) + #[stable(feature = "rust1", since = "1.0.0")] + impl Display for $ty { + fn fmt(&self, fmt: &mut Formatter<'_>) -> Result { + float_to_decimal_display(fmt, self) + } } - } - #[stable(feature = "rust1", since = "1.0.0")] - impl LowerExp for $ty { - fn fmt(&self, fmt: &mut Formatter<'_>) -> Result { - float_to_exponential_common(fmt, self, false) + #[stable(feature = "rust1", since = "1.0.0")] + impl LowerExp for $ty { + fn fmt(&self, fmt: &mut Formatter<'_>) -> Result { + float_to_exponential_common(fmt, self, false) + } } - } - #[stable(feature = "rust1", since = "1.0.0")] - impl UpperExp for $ty { - fn fmt(&self, fmt: &mut Formatter<'_>) -> Result { - float_to_exponential_common(fmt, self, true) + #[stable(feature = "rust1", since = "1.0.0")] + impl UpperExp for $ty { + fn fmt(&self, fmt: &mut Formatter<'_>) -> Result { + float_to_exponential_common(fmt, self, true) + } } - } + )* }; } -floating! { f32 } -floating! { f64 } +floating! { f32 f64 } #[stable(feature = "rust1", since = "1.0.0")] impl Debug for f16 { diff --git a/library/core/src/fmt/mod.rs b/library/core/src/fmt/mod.rs index 45c2b6a6a0f73..f3b54230bc1a5 100644 --- a/library/core/src/fmt/mod.rs +++ b/library/core/src/fmt/mod.rs @@ -33,10 +33,10 @@ pub enum Alignment { Center, } -#[unstable(feature = "debug_closure_helpers", issue = "117729")] -pub use self::builders::{from_fn, FromFn}; #[stable(feature = "debug_builders", since = "1.2.0")] pub use self::builders::{DebugList, DebugMap, DebugSet, DebugStruct, DebugTuple}; +#[unstable(feature = "debug_closure_helpers", issue = "117729")] +pub use self::builders::{FromFn, from_fn}; /// The type returned by formatter methods. /// @@ -333,7 +333,10 @@ pub struct Arguments<'a> { #[unstable(feature = "fmt_internals", issue = "none")] impl<'a> Arguments<'a> { #[inline] - #[rustc_const_unstable(feature = "const_fmt_arguments_new", issue = "none")] + #[cfg_attr( + bootstrap, + rustc_const_unstable(feature = "const_fmt_arguments_new", issue = "none") + )] pub const fn new_const(pieces: &'a [&'static str; N]) -> Self { const { assert!(N <= 1) }; Arguments { pieces, fmt: None, args: &[] } @@ -438,6 +441,7 @@ impl<'a> Arguments<'a> { #[rustc_const_unstable(feature = "const_arguments_as_str", issue = "103900")] #[must_use] #[inline] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] pub const fn as_str(&self) -> Option<&'static str> { match (self.pieces, self.args) { ([], []) => Some(""), @@ -975,9 +979,17 @@ pub trait UpperHex { /// `p` formatting. /// /// The `Pointer` trait should format its output as a memory location. This is commonly presented -/// as hexadecimal. +/// as hexadecimal. For more information on formatters, see [the module-level documentation][module]. /// -/// For more information on formatters, see [the module-level documentation][module]. +/// Printing of pointers is not a reliable way to discover how Rust programs are implemented. +/// The act of reading an address changes the program itself, and may change how the data is represented +/// in memory, and may affect which optimizations are applied to the code. +/// +/// The printed pointer values are not guaranteed to be stable nor unique identifiers of objects. +/// Rust allows moving values to different memory locations, and may reuse the same memory locations +/// for different purposes. +/// +/// There is no guarantee that the printed value can be converted back to a pointer. /// /// [module]: ../../std/fmt/index.html /// diff --git a/library/core/src/fmt/nofloat.rs b/library/core/src/fmt/nofloat.rs index 6b07236f1da12..29aaee75d22bc 100644 --- a/library/core/src/fmt/nofloat.rs +++ b/library/core/src/fmt/nofloat.rs @@ -1,18 +1,17 @@ use crate::fmt::{Debug, Formatter, Result}; macro_rules! floating { - ($ty:ident) => { - #[stable(feature = "rust1", since = "1.0.0")] - impl Debug for $ty { - #[inline] - fn fmt(&self, _fmt: &mut Formatter<'_>) -> Result { - panic!("floating point support is turned off"); + ($($ty:ident)*) => { + $( + #[stable(feature = "rust1", since = "1.0.0")] + impl Debug for $ty { + #[inline] + fn fmt(&self, _fmt: &mut Formatter<'_>) -> Result { + panic!("floating point fmt support is turned off"); + } } - } + )* }; } -floating! { f16 } -floating! { f32 } -floating! { f64 } -floating! { f128 } +floating! { f16 f32 f64 f128 } diff --git a/library/core/src/fmt/num.rs b/library/core/src/fmt/num.rs index e7726da8d91f2..f1540803f978d 100644 --- a/library/core/src/fmt/num.rs +++ b/library/core/src/fmt/num.rs @@ -20,33 +20,22 @@ trait DisplayInt: macro_rules! impl_int { ($($t:ident)*) => ( - $(impl DisplayInt for $t { - fn zero() -> Self { 0 } - fn from_u8(u: u8) -> Self { u as Self } - fn to_u8(&self) -> u8 { *self as u8 } - #[cfg(not(any(target_pointer_width = "64", target_arch = "wasm32")))] - fn to_u32(&self) -> u32 { *self as u32 } - fn to_u64(&self) -> u64 { *self as u64 } - fn to_u128(&self) -> u128 { *self as u128 } - })* - ) -} -macro_rules! impl_uint { - ($($t:ident)*) => ( - $(impl DisplayInt for $t { - fn zero() -> Self { 0 } - fn from_u8(u: u8) -> Self { u as Self } - fn to_u8(&self) -> u8 { *self as u8 } - #[cfg(not(any(target_pointer_width = "64", target_arch = "wasm32")))] - fn to_u32(&self) -> u32 { *self as u32 } - fn to_u64(&self) -> u64 { *self as u64 } - fn to_u128(&self) -> u128 { *self as u128 } - })* + $(impl DisplayInt for $t { + fn zero() -> Self { 0 } + fn from_u8(u: u8) -> Self { u as Self } + fn to_u8(&self) -> u8 { *self as u8 } + #[cfg(not(any(target_pointer_width = "64", target_arch = "wasm32")))] + fn to_u32(&self) -> u32 { *self as u32 } + fn to_u64(&self) -> u64 { *self as u64 } + fn to_u128(&self) -> u128 { *self as u128 } + })* ) } -impl_int! { i8 i16 i32 i64 i128 isize } -impl_uint! { u8 u16 u32 u64 u128 usize } +impl_int! { + i8 i16 i32 i64 i128 isize + u8 u16 u32 u64 u128 usize +} /// A type that represents a specific radix /// @@ -178,26 +167,25 @@ integer! { i16, u16 } integer! { i32, u32 } integer! { i64, u64 } integer! { i128, u128 } -macro_rules! debug { - ($($T:ident)*) => {$( - #[stable(feature = "rust1", since = "1.0.0")] - impl fmt::Debug for $T { - #[inline] - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - if f.debug_lower_hex() { - fmt::LowerHex::fmt(self, f) - } else if f.debug_upper_hex() { - fmt::UpperHex::fmt(self, f) - } else { - fmt::Display::fmt(self, f) + +macro_rules! impl_Debug { + ($($T:ident)*) => { + $( + #[stable(feature = "rust1", since = "1.0.0")] + impl fmt::Debug for $T { + #[inline] + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + if f.debug_lower_hex() { + fmt::LowerHex::fmt(self, f) + } else if f.debug_upper_hex() { + fmt::UpperHex::fmt(self, f) + } else { + fmt::Display::fmt(self, f) + } } } - } - )*}; -} -debug! { - i8 i16 i32 i64 i128 isize - u8 u16 u32 u64 u128 usize + )* + }; } // 2 digit decimal look up table @@ -208,75 +196,119 @@ static DEC_DIGITS_LUT: &[u8; 200] = b"0001020304050607080910111213141516171819\ 8081828384858687888990919293949596979899"; macro_rules! impl_Display { - ($($t:ident),* as $u:ident via $conv_fn:ident named $name:ident) => { - #[cfg(not(feature = "optimize_for_size"))] - fn $name(mut n: $u, is_nonnegative: bool, f: &mut fmt::Formatter<'_>) -> fmt::Result { - // 2^128 is about 3*10^38, so 39 gives an extra byte of space - let mut buf = [MaybeUninit::::uninit(); 39]; - let mut curr = buf.len(); - let buf_ptr = MaybeUninit::slice_as_mut_ptr(&mut buf); - let lut_ptr = DEC_DIGITS_LUT.as_ptr(); + ($($t:ident $(as $positive:ident)? named $name:ident,)* ; as $u:ident via $conv_fn:ident named $gen_name:ident) => { - // SAFETY: Since `d1` and `d2` are always less than or equal to `198`, we - // can copy from `lut_ptr[d1..d1 + 1]` and `lut_ptr[d2..d2 + 1]`. To show - // that it's OK to copy into `buf_ptr`, notice that at the beginning - // `curr == buf.len() == 39 > log(n)` since `n < 2^128 < 10^39`, and at - // each step this is kept the same as `n` is divided. Since `n` is always - // non-negative, this means that `curr > 0` so `buf_ptr[curr..curr + 1]` - // is safe to access. - unsafe { - // need at least 16 bits for the 4-characters-at-a-time to work. - assert!(crate::mem::size_of::<$u>() >= 2); + $( + #[stable(feature = "rust1", since = "1.0.0")] + impl fmt::Display for $t { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + // If it's a signed integer. + $( + let is_nonnegative = *self >= 0; - // eagerly decode 4 characters at a time - while n >= 10000 { - let rem = (n % 10000) as usize; - n /= 10000; + #[cfg(not(feature = "optimize_for_size"))] + { + if !is_nonnegative { + // convert the negative num to positive by summing 1 to its 2s complement + return (!self as $positive).wrapping_add(1)._fmt(false, f); + } + } + #[cfg(feature = "optimize_for_size")] + { + if !is_nonnegative { + // convert the negative num to positive by summing 1 to its 2s complement + return $gen_name((!self.$conv_fn()).wrapping_add(1), false, f); + } + } + )? + // If it's a positive integer. + #[cfg(not(feature = "optimize_for_size"))] + { + self._fmt(true, f) + } + #[cfg(feature = "optimize_for_size")] + { + $gen_name(self.$conv_fn(), true, f) + } + } + } - let d1 = (rem / 100) << 1; - let d2 = (rem % 100) << 1; - curr -= 4; + #[cfg(not(feature = "optimize_for_size"))] + impl $t { + fn _fmt(mut self: $t, is_nonnegative: bool, f: &mut fmt::Formatter<'_>) -> fmt::Result { + const SIZE: usize = $t::MAX.ilog(10) as usize + 1; + let mut buf = [MaybeUninit::::uninit(); SIZE]; + let mut curr = SIZE; + let buf_ptr = MaybeUninit::slice_as_mut_ptr(&mut buf); + let lut_ptr = DEC_DIGITS_LUT.as_ptr(); + + // SAFETY: Since `d1` and `d2` are always less than or equal to `198`, we + // can copy from `lut_ptr[d1..d1 + 1]` and `lut_ptr[d2..d2 + 1]`. To show + // that it's OK to copy into `buf_ptr`, notice that at the beginning + // `curr == buf.len() == 39 > log(n)` since `n < 2^128 < 10^39`, and at + // each step this is kept the same as `n` is divided. Since `n` is always + // non-negative, this means that `curr > 0` so `buf_ptr[curr..curr + 1]` + // is safe to access. + unsafe { + // need at least 16 bits for the 4-characters-at-a-time to work. + #[allow(overflowing_literals)] + #[allow(unused_comparisons)] + // This block will be removed for smaller types at compile time and in the worst + // case, it will prevent to have the `10000` literal to overflow for `i8` and `u8`. + if core::mem::size_of::<$t>() >= 2 { + // eagerly decode 4 characters at a time + while self >= 10000 { + let rem = (self % 10000) as usize; + self /= 10000; + + let d1 = (rem / 100) << 1; + let d2 = (rem % 100) << 1; + curr -= 4; + + // We are allowed to copy to `buf_ptr[curr..curr + 3]` here since + // otherwise `curr < 0`. But then `n` was originally at least `10000^10` + // which is `10^40 > 2^128 > n`. + ptr::copy_nonoverlapping(lut_ptr.add(d1 as usize), buf_ptr.add(curr), 2); + ptr::copy_nonoverlapping(lut_ptr.add(d2 as usize), buf_ptr.add(curr + 2), 2); + } + } - // We are allowed to copy to `buf_ptr[curr..curr + 3]` here since - // otherwise `curr < 0`. But then `n` was originally at least `10000^10` - // which is `10^40 > 2^128 > n`. - ptr::copy_nonoverlapping(lut_ptr.add(d1), buf_ptr.add(curr), 2); - ptr::copy_nonoverlapping(lut_ptr.add(d2), buf_ptr.add(curr + 2), 2); - } + // if we reach here numbers are <= 9999, so at most 4 chars long + let mut n = self as usize; // possibly reduce 64bit math - // if we reach here numbers are <= 9999, so at most 4 chars long - let mut n = n as usize; // possibly reduce 64bit math + // decode 2 more chars, if > 2 chars + if n >= 100 { + let d1 = (n % 100) << 1; + n /= 100; + curr -= 2; + ptr::copy_nonoverlapping(lut_ptr.add(d1), buf_ptr.add(curr), 2); + } - // decode 2 more chars, if > 2 chars - if n >= 100 { - let d1 = (n % 100) << 1; - n /= 100; - curr -= 2; - ptr::copy_nonoverlapping(lut_ptr.add(d1), buf_ptr.add(curr), 2); + // if we reach here numbers are <= 100, so at most 2 chars long + // The biggest it can be is 99, and 99 << 1 == 198, so a `u8` is enough. + // decode last 1 or 2 chars + if n < 10 { + curr -= 1; + *buf_ptr.add(curr) = (n as u8) + b'0'; + } else { + let d1 = n << 1; + curr -= 2; + ptr::copy_nonoverlapping(lut_ptr.add(d1), buf_ptr.add(curr), 2); + } } - // decode last 1 or 2 chars - if n < 10 { - curr -= 1; - *buf_ptr.add(curr) = (n as u8) + b'0'; - } else { - let d1 = n << 1; - curr -= 2; - ptr::copy_nonoverlapping(lut_ptr.add(d1), buf_ptr.add(curr), 2); - } + // SAFETY: `curr` > 0 (since we made `buf` large enough), and all the chars are valid + // UTF-8 since `DEC_DIGITS_LUT` is + let buf_slice = unsafe { + str::from_utf8_unchecked( + slice::from_raw_parts(buf_ptr.add(curr), buf.len() - curr)) + }; + f.pad_integral(is_nonnegative, "", buf_slice) } - - // SAFETY: `curr` > 0 (since we made `buf` large enough), and all the chars are valid - // UTF-8 since `DEC_DIGITS_LUT` is - let buf_slice = unsafe { - str::from_utf8_unchecked( - slice::from_raw_parts(buf_ptr.add(curr), buf.len() - curr)) - }; - f.pad_integral(is_nonnegative, "", buf_slice) - } + })* #[cfg(feature = "optimize_for_size")] - fn $name(mut n: $u, is_nonnegative: bool, f: &mut fmt::Formatter<'_>) -> fmt::Result { + fn $gen_name(mut n: $u, is_nonnegative: bool, f: &mut fmt::Formatter<'_>) -> fmt::Result { // 2^128 is about 3*10^38, so 39 gives an extra byte of space let mut buf = [MaybeUninit::::uninit(); 39]; let mut curr = buf.len(); @@ -306,21 +338,6 @@ macro_rules! impl_Display { }; f.pad_integral(is_nonnegative, "", buf_slice) } - - $(#[stable(feature = "rust1", since = "1.0.0")] - impl fmt::Display for $t { - #[allow(unused_comparisons)] - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - let is_nonnegative = *self >= 0; - let n = if is_nonnegative { - self.$conv_fn() - } else { - // convert the negative num to positive by summing 1 to it's 2 complement - (!self.$conv_fn()).wrapping_add(1) - }; - $name(n, is_nonnegative, f) - } - })* }; } @@ -374,7 +391,6 @@ macro_rules! impl_Exp { (n, exponent, exponent, added_precision) }; - // 39 digits (worst case u128) + . = 40 // Since `curr` always decreases by the number of digits copied, this means // that `curr >= 0`. let mut buf = [MaybeUninit::::uninit(); 40]; @@ -469,7 +485,7 @@ macro_rules! impl_Exp { let n = if is_nonnegative { self.$conv_fn() } else { - // convert the negative num to positive by summing 1 to it's 2 complement + // convert the negative num to positive by summing 1 to its 2s complement (!self.$conv_fn()).wrapping_add(1) }; $name(n, is_nonnegative, false, f) @@ -484,7 +500,7 @@ macro_rules! impl_Exp { let n = if is_nonnegative { self.$conv_fn() } else { - // convert the negative num to positive by summing 1 to it's 2 complement + // convert the negative num to positive by summing 1 to its 2s complement (!self.$conv_fn()).wrapping_add(1) }; $name(n, is_nonnegative, true, f) @@ -493,14 +509,28 @@ macro_rules! impl_Exp { }; } +impl_Debug! { + i8 i16 i32 i64 i128 isize + u8 u16 u32 u64 u128 usize +} + // Include wasm32 in here since it doesn't reflect the native pointer size, and // often cares strongly about getting a smaller code size. #[cfg(any(target_pointer_width = "64", target_arch = "wasm32"))] mod imp { use super::*; impl_Display!( - i8, u8, i16, u16, i32, u32, i64, u64, usize, isize - as u64 via to_u64 named fmt_u64 + i8 as u8 named fmt_i8, + u8 named fmt_u8, + i16 as u16 named fmt_i16, + u16 named fmt_u16, + i32 as u32 named fmt_i32, + u32 named fmt_u32, + i64 as u64 named fmt_i64, + u64 named fmt_u64, + isize as usize named fmt_isize, + usize named fmt_usize, + ; as u64 via to_u64 named fmt_u64 ); impl_Exp!( i8, u8, i16, u16, i32, u32, i64, u64, usize, isize @@ -511,8 +541,21 @@ mod imp { #[cfg(not(any(target_pointer_width = "64", target_arch = "wasm32")))] mod imp { use super::*; - impl_Display!(i8, u8, i16, u16, i32, u32, isize, usize as u32 via to_u32 named fmt_u32); - impl_Display!(i64, u64 as u64 via to_u64 named fmt_u64); + impl_Display!( + i8 as u8 named fmt_i8, + u8 named fmt_u8, + i16 as u16 named fmt_i16, + u16 named fmt_u16, + i32 as u32 named fmt_i32, + u32 named fmt_u32, + isize as usize named fmt_isize, + usize named fmt_usize, + ; as u32 via to_u32 named fmt_u32); + impl_Display!( + i64 as u64 named fmt_i64, + u64 named fmt_u64, + ; as u64 via to_u64 named fmt_u64); + impl_Exp!(i8, u8, i16, u16, i32, u32, isize, usize as u32 via to_u32 named exp_u32); impl_Exp!(i64, u64 as u64 via to_u64 named exp_u64); } @@ -619,7 +662,7 @@ impl fmt::Display for i128 { let n = if is_nonnegative { self.to_u128() } else { - // convert the negative num to positive by summing 1 to it's 2 complement + // convert the negative num to positive by summing 1 to its 2s complement (!self.to_u128()).wrapping_add(1) }; fmt_u128(n, is_nonnegative, f) diff --git a/library/core/src/fmt/rt.rs b/library/core/src/fmt/rt.rs index eee4a9e4c6c89..af6f0da88de67 100644 --- a/library/core/src/fmt/rt.rs +++ b/library/core/src/fmt/rt.rs @@ -94,11 +94,11 @@ pub struct Argument<'a> { } #[rustc_diagnostic_item = "ArgumentMethods"] -impl<'a> Argument<'a> { +impl Argument<'_> { #[inline(always)] - fn new<'b, T>(x: &'b T, f: fn(&T, &mut Formatter<'_>) -> Result) -> Argument<'b> { + fn new<'a, T>(x: &'a T, f: fn(&T, &mut Formatter<'_>) -> Result) -> Argument<'a> { Argument { - // INVARIANT: this creates an `ArgumentType<'b>` from a `&'b T` and + // INVARIANT: this creates an `ArgumentType<'a>` from a `&'a T` and // a `fn(&T, ...)`, so the invariant is maintained. ty: ArgumentType::Placeholder { value: NonNull::from(x).cast(), @@ -110,43 +110,43 @@ impl<'a> Argument<'a> { } #[inline(always)] - pub fn new_display<'b, T: Display>(x: &'b T) -> Argument<'b> { + pub fn new_display(x: &T) -> Argument<'_> { Self::new(x, Display::fmt) } #[inline(always)] - pub fn new_debug<'b, T: Debug>(x: &'b T) -> Argument<'b> { + pub fn new_debug(x: &T) -> Argument<'_> { Self::new(x, Debug::fmt) } #[inline(always)] - pub fn new_debug_noop<'b, T: Debug>(x: &'b T) -> Argument<'b> { + pub fn new_debug_noop(x: &T) -> Argument<'_> { Self::new(x, |_, _| Ok(())) } #[inline(always)] - pub fn new_octal<'b, T: Octal>(x: &'b T) -> Argument<'b> { + pub fn new_octal(x: &T) -> Argument<'_> { Self::new(x, Octal::fmt) } #[inline(always)] - pub fn new_lower_hex<'b, T: LowerHex>(x: &'b T) -> Argument<'b> { + pub fn new_lower_hex(x: &T) -> Argument<'_> { Self::new(x, LowerHex::fmt) } #[inline(always)] - pub fn new_upper_hex<'b, T: UpperHex>(x: &'b T) -> Argument<'b> { + pub fn new_upper_hex(x: &T) -> Argument<'_> { Self::new(x, UpperHex::fmt) } #[inline(always)] - pub fn new_pointer<'b, T: Pointer>(x: &'b T) -> Argument<'b> { + pub fn new_pointer(x: &T) -> Argument<'_> { Self::new(x, Pointer::fmt) } #[inline(always)] - pub fn new_binary<'b, T: Binary>(x: &'b T) -> Argument<'b> { + pub fn new_binary(x: &T) -> Argument<'_> { Self::new(x, Binary::fmt) } #[inline(always)] - pub fn new_lower_exp<'b, T: LowerExp>(x: &'b T) -> Argument<'b> { + pub fn new_lower_exp(x: &T) -> Argument<'_> { Self::new(x, LowerExp::fmt) } #[inline(always)] - pub fn new_upper_exp<'b, T: UpperExp>(x: &'b T) -> Argument<'b> { + pub fn new_upper_exp(x: &T) -> Argument<'_> { Self::new(x, UpperExp::fmt) } #[inline(always)] diff --git a/library/core/src/future/async_drop.rs b/library/core/src/future/async_drop.rs index 16ac77fa15045..7de5fe67cd096 100644 --- a/library/core/src/future/async_drop.rs +++ b/library/core/src/future/async_drop.rs @@ -6,7 +6,7 @@ use crate::intrinsics::discriminant_value; use crate::marker::{DiscriminantKind, PhantomPinned}; use crate::mem::MaybeUninit; use crate::pin::Pin; -use crate::task::{ready, Context, Poll}; +use crate::task::{Context, Poll, ready}; /// Asynchronously drops a value by running `AsyncDrop::async_drop` /// on a value and its fields recursively. diff --git a/library/core/src/future/join.rs b/library/core/src/future/join.rs index 3f35179ddc29b..18bc8d9658678 100644 --- a/library/core/src/future/join.rs +++ b/library/core/src/future/join.rs @@ -1,10 +1,10 @@ #![allow(unused_imports, unused_macros)] // items are used by the macro use crate::cell::UnsafeCell; -use crate::future::{poll_fn, Future}; +use crate::future::{Future, poll_fn}; use crate::mem; use crate::pin::Pin; -use crate::task::{ready, Context, Poll}; +use crate::task::{Context, Poll, ready}; /// Polls multiple futures simultaneously, returning a tuple /// of all results once complete. diff --git a/library/core/src/future/mod.rs b/library/core/src/future/mod.rs index d188f1c713079..e5a368796ec93 100644 --- a/library/core/src/future/mod.rs +++ b/library/core/src/future/mod.rs @@ -21,15 +21,15 @@ mod poll_fn; mod ready; #[unstable(feature = "async_drop", issue = "126482")] -pub use async_drop::{async_drop, async_drop_in_place, AsyncDrop, AsyncDropInPlace}; +pub use async_drop::{AsyncDrop, AsyncDropInPlace, async_drop, async_drop_in_place}; #[stable(feature = "into_future", since = "1.64.0")] pub use into_future::IntoFuture; #[stable(feature = "future_readiness_fns", since = "1.48.0")] -pub use pending::{pending, Pending}; +pub use pending::{Pending, pending}; #[stable(feature = "future_poll_fn", since = "1.64.0")] -pub use poll_fn::{poll_fn, PollFn}; +pub use poll_fn::{PollFn, poll_fn}; #[stable(feature = "future_readiness_fns", since = "1.48.0")] -pub use ready::{ready, Ready}; +pub use ready::{Ready, ready}; #[stable(feature = "futures_api", since = "1.36.0")] pub use self::future::Future; diff --git a/library/core/src/hint.rs b/library/core/src/hint.rs index a69f0afdb0a59..78df51f2bc47d 100644 --- a/library/core/src/hint.rs +++ b/library/core/src/hint.rs @@ -506,7 +506,7 @@ pub const fn black_box(dummy: T) -> T { /// # } /// ``` #[unstable(feature = "hint_must_use", issue = "94745")] -#[rustc_const_unstable(feature = "hint_must_use", issue = "94745")] +#[cfg_attr(bootstrap, rustc_const_unstable(feature = "hint_must_use", issue = "94745"))] #[must_use] // <-- :) #[inline(always)] pub const fn must_use(value: T) -> T { diff --git a/library/core/src/intrinsics.rs b/library/core/src/intrinsics.rs index 61c17421e550b..11106951584d6 100644 --- a/library/core/src/intrinsics.rs +++ b/library/core/src/intrinsics.rs @@ -14,9 +14,10 @@ //! `#[rustc_const_unstable(feature = "const_such_and_such", issue = "01234")]` to the intrinsic declaration. //! //! If an intrinsic is supposed to be used from a `const fn` with a `rustc_const_stable` attribute, -//! the intrinsic's attribute must be `rustc_const_stable`, too. Such a change should not be done -//! without T-lang consultation, because it bakes a feature into the language that cannot be -//! replicated in user code without compiler support. +//! `#[rustc_const_stable_indirect]` needs to be added to the intrinsic (`#[rustc_const_unstable]` +//! can be removed then). Such a change should not be done without T-lang consultation, because it +//! may bake a feature into the language that cannot be replicated in user code without compiler +//! support. //! //! # Volatiles //! @@ -65,6 +66,7 @@ use safety::requires; use crate::marker::{DiscriminantKind, Tuple}; +use crate::mem::SizedTypeProperties; use crate::{ptr, ub_checks}; #[cfg(kani)] @@ -933,7 +935,7 @@ extern "rust-intrinsic" { /// on most platforms. /// On Unix, the /// process will probably terminate with a signal like `SIGABRT`, `SIGILL`, `SIGTRAP`, `SIGSEGV` or - /// `SIGBUS`. The precise behaviour is not guaranteed and not stable. + /// `SIGBUS`. The precise behavior is not guaranteed and not stable. #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn abort() -> !; @@ -946,7 +948,11 @@ extern "rust-intrinsic" { /// reach code marked with this function. /// /// The stabilized version of this intrinsic is [`core::hint::unreachable_unchecked`]. - #[rustc_const_stable(feature = "const_unreachable_unchecked", since = "1.57.0")] + #[cfg_attr( + bootstrap, + rustc_const_stable(feature = "const_unreachable_unchecked", since = "1.57.0") + )] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn unreachable() -> !; } @@ -961,7 +967,8 @@ extern "rust-intrinsic" { /// own, or if it does not enable any significant optimizations. /// /// The stabilized version of this intrinsic is [`core::hint::assert_unchecked`]. -#[rustc_const_stable(feature = "const_assume", since = "1.77.0")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "const_assume", since = "1.77.0"))] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] #[unstable(feature = "core_intrinsics", issue = "none")] #[rustc_intrinsic] @@ -983,7 +990,11 @@ pub const unsafe fn assume(b: bool) { /// any safety invariants. /// /// This intrinsic does not have a stable counterpart. -#[rustc_const_unstable(feature = "const_likely", issue = "none")] +#[cfg_attr( + bootstrap, + rustc_const_stable(feature = "const_likely", since = "CURRENT_RUSTC_VERSION") +)] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[unstable(feature = "core_intrinsics", issue = "none")] #[rustc_intrinsic] #[rustc_nounwind] @@ -1003,7 +1014,11 @@ pub const fn likely(b: bool) -> bool { /// any safety invariants. /// /// This intrinsic does not have a stable counterpart. -#[rustc_const_unstable(feature = "const_likely", issue = "none")] +#[cfg_attr( + bootstrap, + rustc_const_stable(feature = "const_likely", since = "CURRENT_RUSTC_VERSION") +)] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[unstable(feature = "core_intrinsics", issue = "none")] #[rustc_intrinsic] #[rustc_nounwind] @@ -1044,7 +1059,8 @@ extern "rust-intrinsic" { /// This will statically either panic, or do nothing. /// /// This intrinsic does not have a stable counterpart. - #[rustc_const_stable(feature = "const_assert_type", since = "1.59.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_assert_type", since = "1.59.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn assert_inhabited(); @@ -1053,7 +1069,8 @@ extern "rust-intrinsic" { /// zero-initialization: This will statically either panic, or do nothing. /// /// This intrinsic does not have a stable counterpart. - #[rustc_const_stable(feature = "const_assert_type2", since = "1.75.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_assert_type2", since = "1.75.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn assert_zero_valid(); @@ -1061,7 +1078,8 @@ extern "rust-intrinsic" { /// A guard for `std::mem::uninitialized`. This will statically either panic, or do nothing. /// /// This intrinsic does not have a stable counterpart. - #[rustc_const_stable(feature = "const_assert_type2", since = "1.75.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_assert_type2", since = "1.75.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn assert_mem_uninitialized_valid(); @@ -1074,7 +1092,8 @@ extern "rust-intrinsic" { /// any safety invariants. /// /// Consider using [`core::panic::Location::caller`] instead. - #[rustc_const_stable(feature = "const_caller_location", since = "1.79.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_caller_location", since = "1.79.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn caller_location() -> &'static crate::panic::Location<'static>; @@ -1088,7 +1107,8 @@ extern "rust-intrinsic" { /// it does not require an `unsafe` block. /// Therefore, implementations must not require the user to uphold /// any safety invariants. - #[rustc_const_unstable(feature = "const_intrinsic_forget", issue = "none")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_intrinsic_forget", since = "1.83.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn forget(_: T); @@ -1387,14 +1407,15 @@ extern "rust-intrinsic" { /// Like [`transmute`], but even less checked at compile-time: rather than /// giving an error for `size_of::() != size_of::()`, it's - /// **Undefined Behaviour** at runtime. + /// **Undefined Behavior** at runtime. /// /// Prefer normal `transmute` where possible, for the extra checking, since /// both do exactly the same thing at runtime, if they both compile. /// /// This is not expected to ever be exposed directly to users, rather it /// may eventually be exposed through some more-constrained API. - #[rustc_const_stable(feature = "const_transmute", since = "1.56.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_transmute", since = "1.56.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn transmute_unchecked(src: Src) -> Dst; @@ -1411,7 +1432,8 @@ extern "rust-intrinsic" { /// any safety invariants. /// /// The stabilized version of this intrinsic is [`mem::needs_drop`](crate::mem::needs_drop). - #[rustc_const_stable(feature = "const_needs_drop", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_needs_drop", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn needs_drop() -> bool; @@ -1429,12 +1451,12 @@ extern "rust-intrinsic" { /// /// If the computed offset is non-zero, then both the starting and resulting pointer must be /// either in bounds or at the end of an allocated object. If either pointer is out - /// of bounds or arithmetic overflow occurs then any further use of the returned value will - /// result in undefined behavior. + /// of bounds or arithmetic overflow occurs then this operation is undefined behavior. /// /// The stabilized version of this intrinsic is [`pointer::offset`]. #[must_use = "returns a new pointer rather than modifying its argument"] - #[rustc_const_stable(feature = "const_ptr_offset", since = "1.61.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_ptr_offset", since = "1.61.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn offset(dst: Ptr, offset: Delta) -> Ptr; @@ -1452,7 +1474,8 @@ extern "rust-intrinsic" { /// /// The stabilized version of this intrinsic is [`pointer::wrapping_offset`]. #[must_use = "returns a new pointer rather than modifying its argument"] - #[rustc_const_stable(feature = "const_ptr_offset", since = "1.61.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_ptr_offset", since = "1.61.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn arith_offset(dst: *const T, offset: isize) -> *const T; @@ -1800,153 +1823,54 @@ extern "rust-intrinsic" { #[rustc_nounwind] pub fn fmaf128(a: f128, b: f128, c: f128) -> f128; - /// Returns the absolute value of an `f16`. - /// - /// The stabilized version of this intrinsic is - /// [`f16::abs`](../../std/primitive.f16.html#method.abs) - #[rustc_nounwind] - pub fn fabsf16(x: f16) -> f16; - /// Returns the absolute value of an `f32`. - /// - /// The stabilized version of this intrinsic is - /// [`f32::abs`](../../std/primitive.f32.html#method.abs) - #[rustc_nounwind] - pub fn fabsf32(x: f32) -> f32; - /// Returns the absolute value of an `f64`. - /// - /// The stabilized version of this intrinsic is - /// [`f64::abs`](../../std/primitive.f64.html#method.abs) - #[rustc_nounwind] - pub fn fabsf64(x: f64) -> f64; - /// Returns the absolute value of an `f128`. - /// - /// The stabilized version of this intrinsic is - /// [`f128::abs`](../../std/primitive.f128.html#method.abs) - #[rustc_nounwind] - pub fn fabsf128(x: f128) -> f128; - - /// Returns the minimum of two `f16` values. - /// - /// Note that, unlike most intrinsics, this is safe to call; - /// it does not require an `unsafe` block. - /// Therefore, implementations must not require the user to uphold - /// any safety invariants. - /// - /// The stabilized version of this intrinsic is - /// [`f16::min`] - #[rustc_safe_intrinsic] - #[rustc_nounwind] - pub fn minnumf16(x: f16, y: f16) -> f16; - /// Returns the minimum of two `f32` values. - /// - /// Note that, unlike most intrinsics, this is safe to call; - /// it does not require an `unsafe` block. - /// Therefore, implementations must not require the user to uphold - /// any safety invariants. - /// - /// The stabilized version of this intrinsic is - /// [`f32::min`] - #[rustc_safe_intrinsic] - #[rustc_nounwind] - pub fn minnumf32(x: f32, y: f32) -> f32; - /// Returns the minimum of two `f64` values. - /// - /// Note that, unlike most intrinsics, this is safe to call; - /// it does not require an `unsafe` block. - /// Therefore, implementations must not require the user to uphold - /// any safety invariants. - /// - /// The stabilized version of this intrinsic is - /// [`f64::min`] - #[rustc_safe_intrinsic] - #[rustc_nounwind] - pub fn minnumf64(x: f64, y: f64) -> f64; - /// Returns the minimum of two `f128` values. - /// - /// Note that, unlike most intrinsics, this is safe to call; - /// it does not require an `unsafe` block. - /// Therefore, implementations must not require the user to uphold - /// any safety invariants. - /// - /// The stabilized version of this intrinsic is - /// [`f128::min`] - #[rustc_safe_intrinsic] - #[rustc_nounwind] - pub fn minnumf128(x: f128, y: f128) -> f128; - - /// Returns the maximum of two `f16` values. - /// - /// Note that, unlike most intrinsics, this is safe to call; - /// it does not require an `unsafe` block. - /// Therefore, implementations must not require the user to uphold - /// any safety invariants. - /// - /// The stabilized version of this intrinsic is - /// [`f16::max`] - #[rustc_safe_intrinsic] - #[rustc_nounwind] - pub fn maxnumf16(x: f16, y: f16) -> f16; - /// Returns the maximum of two `f32` values. - /// - /// Note that, unlike most intrinsics, this is safe to call; - /// it does not require an `unsafe` block. - /// Therefore, implementations must not require the user to uphold - /// any safety invariants. - /// - /// The stabilized version of this intrinsic is - /// [`f32::max`] - #[rustc_safe_intrinsic] - #[rustc_nounwind] - pub fn maxnumf32(x: f32, y: f32) -> f32; - /// Returns the maximum of two `f64` values. - /// - /// Note that, unlike most intrinsics, this is safe to call; - /// it does not require an `unsafe` block. - /// Therefore, implementations must not require the user to uphold - /// any safety invariants. - /// - /// The stabilized version of this intrinsic is - /// [`f64::max`] - #[rustc_safe_intrinsic] - #[rustc_nounwind] - pub fn maxnumf64(x: f64, y: f64) -> f64; - /// Returns the maximum of two `f128` values. - /// - /// Note that, unlike most intrinsics, this is safe to call; - /// it does not require an `unsafe` block. - /// Therefore, implementations must not require the user to uphold - /// any safety invariants. - /// - /// The stabilized version of this intrinsic is - /// [`f128::max`] - #[rustc_safe_intrinsic] - #[rustc_nounwind] - pub fn maxnumf128(x: f128, y: f128) -> f128; - - /// Copies the sign from `y` to `x` for `f16` values. - /// - /// The stabilized version of this intrinsic is - /// [`f16::copysign`](../../std/primitive.f16.html#method.copysign) - #[rustc_nounwind] - pub fn copysignf16(x: f16, y: f16) -> f16; - /// Copies the sign from `y` to `x` for `f32` values. - /// - /// The stabilized version of this intrinsic is - /// [`f32::copysign`](../../std/primitive.f32.html#method.copysign) - #[rustc_nounwind] - pub fn copysignf32(x: f32, y: f32) -> f32; - /// Copies the sign from `y` to `x` for `f64` values. - /// - /// The stabilized version of this intrinsic is - /// [`f64::copysign`](../../std/primitive.f64.html#method.copysign) - #[rustc_nounwind] - pub fn copysignf64(x: f64, y: f64) -> f64; - /// Copies the sign from `y` to `x` for `f128` values. - /// - /// The stabilized version of this intrinsic is - /// [`f128::copysign`](../../std/primitive.f128.html#method.copysign) - #[rustc_nounwind] - pub fn copysignf128(x: f128, y: f128) -> f128; + /// Returns `a * b + c` for `f16` values, non-deterministically executing + /// either a fused multiply-add or two operations with rounding of the + /// intermediate result. + /// + /// The operation is fused if the code generator determines that target + /// instruction set has support for a fused operation, and that the fused + /// operation is more efficient than the equivalent, separate pair of mul + /// and add instructions. It is unspecified whether or not a fused operation + /// is selected, and that may depend on optimization level and context, for + /// example. + #[rustc_nounwind] + pub fn fmuladdf16(a: f16, b: f16, c: f16) -> f16; + /// Returns `a * b + c` for `f32` values, non-deterministically executing + /// either a fused multiply-add or two operations with rounding of the + /// intermediate result. + /// + /// The operation is fused if the code generator determines that target + /// instruction set has support for a fused operation, and that the fused + /// operation is more efficient than the equivalent, separate pair of mul + /// and add instructions. It is unspecified whether or not a fused operation + /// is selected, and that may depend on optimization level and context, for + /// example. + #[rustc_nounwind] + pub fn fmuladdf32(a: f32, b: f32, c: f32) -> f32; + /// Returns `a * b + c` for `f64` values, non-deterministically executing + /// either a fused multiply-add or two operations with rounding of the + /// intermediate result. + /// + /// The operation is fused if the code generator determines that target + /// instruction set has support for a fused operation, and that the fused + /// operation is more efficient than the equivalent, separate pair of mul + /// and add instructions. It is unspecified whether or not a fused operation + /// is selected, and that may depend on optimization level and context, for + /// example. + #[rustc_nounwind] + pub fn fmuladdf64(a: f64, b: f64, c: f64) -> f64; + /// Returns `a * b + c` for `f128` values, non-deterministically executing + /// either a fused multiply-add or two operations with rounding of the + /// intermediate result. + /// + /// The operation is fused if the code generator determines that target + /// instruction set has support for a fused operation, and that the fused + /// operation is more efficient than the equivalent, separate pair of mul + /// and add instructions. It is unspecified whether or not a fused operation + /// is selected, and that may depend on optimization level and context, for + /// example. + #[rustc_nounwind] + pub fn fmuladdf128(a: f128, b: f128, c: f128) -> f128; /// Returns the largest integer less than or equal to an `f16`. /// @@ -2234,7 +2158,8 @@ extern "rust-intrinsic" { /// The stabilized versions of this intrinsic are available on the integer /// primitives via the `count_ones` method. For example, /// [`u32::count_ones`] - #[rustc_const_stable(feature = "const_ctpop", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_ctpop", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn ctpop(x: T) -> u32; @@ -2275,7 +2200,8 @@ extern "rust-intrinsic" { /// let num_leading = ctlz(x); /// assert_eq!(num_leading, 16); /// ``` - #[rustc_const_stable(feature = "const_ctlz", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_ctlz", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn ctlz(x: T) -> u32; @@ -2297,7 +2223,8 @@ extern "rust-intrinsic" { /// let num_leading = unsafe { ctlz_nonzero(x) }; /// assert_eq!(num_leading, 3); /// ``` - #[rustc_const_stable(feature = "constctlz", since = "1.50.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "constctlz", since = "1.50.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn ctlz_nonzero(x: T) -> u32; @@ -2337,7 +2264,8 @@ extern "rust-intrinsic" { /// let num_trailing = cttz(x); /// assert_eq!(num_trailing, 16); /// ``` - #[rustc_const_stable(feature = "const_cttz", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_cttz", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn cttz(x: T) -> u32; @@ -2359,7 +2287,8 @@ extern "rust-intrinsic" { /// let num_trailing = unsafe { cttz_nonzero(x) }; /// assert_eq!(num_trailing, 3); /// ``` - #[rustc_const_stable(feature = "const_cttz_nonzero", since = "1.53.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_cttz_nonzero", since = "1.53.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn cttz_nonzero(x: T) -> u32; @@ -2373,7 +2302,8 @@ extern "rust-intrinsic" { /// The stabilized versions of this intrinsic are available on the integer /// primitives via the `swap_bytes` method. For example, /// [`u32::swap_bytes`] - #[rustc_const_stable(feature = "const_bswap", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_bswap", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn bswap(x: T) -> T; @@ -2388,7 +2318,8 @@ extern "rust-intrinsic" { /// The stabilized versions of this intrinsic are available on the integer /// primitives via the `reverse_bits` method. For example, /// [`u32::reverse_bits`] - #[rustc_const_stable(feature = "const_bitreverse", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_bitreverse", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn bitreverse(x: T) -> T; @@ -2414,7 +2345,8 @@ extern "rust-intrinsic" { /// The stabilized versions of this intrinsic are available on the integer /// primitives via the `overflowing_add` method. For example, /// [`u32::overflowing_add`] - #[rustc_const_stable(feature = "const_int_overflow", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_overflow", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn add_with_overflow(x: T, y: T) -> (T, bool); @@ -2429,7 +2361,8 @@ extern "rust-intrinsic" { /// The stabilized versions of this intrinsic are available on the integer /// primitives via the `overflowing_sub` method. For example, /// [`u32::overflowing_sub`] - #[rustc_const_stable(feature = "const_int_overflow", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_overflow", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn sub_with_overflow(x: T, y: T) -> (T, bool); @@ -2444,7 +2377,8 @@ extern "rust-intrinsic" { /// The stabilized versions of this intrinsic are available on the integer /// primitives via the `overflowing_mul` method. For example, /// [`u32::overflowing_mul`] - #[rustc_const_stable(feature = "const_int_overflow", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_overflow", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn mul_with_overflow(x: T, y: T) -> (T, bool); @@ -2463,7 +2397,11 @@ extern "rust-intrinsic" { /// Safe wrappers for this intrinsic are available on the integer /// primitives via the `checked_div` method. For example, /// [`u32::checked_div`] - #[rustc_const_stable(feature = "const_int_unchecked_div", since = "1.52.0")] + #[cfg_attr( + bootstrap, + rustc_const_stable(feature = "const_int_unchecked_div", since = "1.52.0") + )] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn unchecked_div(x: T, y: T) -> T; /// Returns the remainder of an unchecked division, resulting in @@ -2472,7 +2410,11 @@ extern "rust-intrinsic" { /// Safe wrappers for this intrinsic are available on the integer /// primitives via the `checked_rem` method. For example, /// [`u32::checked_rem`] - #[rustc_const_stable(feature = "const_int_unchecked_rem", since = "1.52.0")] + #[cfg_attr( + bootstrap, + rustc_const_stable(feature = "const_int_unchecked_rem", since = "1.52.0") + )] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn unchecked_rem(x: T, y: T) -> T; @@ -2482,7 +2424,8 @@ extern "rust-intrinsic" { /// Safe wrappers for this intrinsic are available on the integer /// primitives via the `checked_shl` method. For example, /// [`u32::checked_shl`] - #[rustc_const_stable(feature = "const_int_unchecked", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_unchecked", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn unchecked_shl(x: T, y: U) -> T; /// Performs an unchecked right shift, resulting in undefined behavior when @@ -2491,7 +2434,8 @@ extern "rust-intrinsic" { /// Safe wrappers for this intrinsic are available on the integer /// primitives via the `checked_shr` method. For example, /// [`u32::checked_shr`] - #[rustc_const_stable(feature = "const_int_unchecked", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_unchecked", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn unchecked_shr(x: T, y: U) -> T; @@ -2500,7 +2444,8 @@ extern "rust-intrinsic" { /// /// The stable counterpart of this intrinsic is `unchecked_add` on the various /// integer types, such as [`u16::unchecked_add`] and [`i64::unchecked_add`]. - #[rustc_const_stable(feature = "unchecked_math", since = "1.79.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "unchecked_math", since = "1.79.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn unchecked_add(x: T, y: T) -> T; @@ -2509,7 +2454,8 @@ extern "rust-intrinsic" { /// /// The stable counterpart of this intrinsic is `unchecked_sub` on the various /// integer types, such as [`u16::unchecked_sub`] and [`i64::unchecked_sub`]. - #[rustc_const_stable(feature = "unchecked_math", since = "1.79.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "unchecked_math", since = "1.79.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn unchecked_sub(x: T, y: T) -> T; @@ -2518,7 +2464,8 @@ extern "rust-intrinsic" { /// /// The stable counterpart of this intrinsic is `unchecked_mul` on the various /// integer types, such as [`u16::unchecked_mul`] and [`i64::unchecked_mul`]. - #[rustc_const_stable(feature = "unchecked_math", since = "1.79.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "unchecked_math", since = "1.79.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn unchecked_mul(x: T, y: T) -> T; @@ -2532,7 +2479,8 @@ extern "rust-intrinsic" { /// The stabilized versions of this intrinsic are available on the integer /// primitives via the `rotate_left` method. For example, /// [`u32::rotate_left`] - #[rustc_const_stable(feature = "const_int_rotate", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_rotate", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn rotate_left(x: T, shift: u32) -> T; @@ -2547,7 +2495,8 @@ extern "rust-intrinsic" { /// The stabilized versions of this intrinsic are available on the integer /// primitives via the `rotate_right` method. For example, /// [`u32::rotate_right`] - #[rustc_const_stable(feature = "const_int_rotate", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_rotate", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn rotate_right(x: T, shift: u32) -> T; @@ -2562,7 +2511,8 @@ extern "rust-intrinsic" { /// The stabilized versions of this intrinsic are available on the integer /// primitives via the `wrapping_add` method. For example, /// [`u32::wrapping_add`] - #[rustc_const_stable(feature = "const_int_wrapping", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_wrapping", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn wrapping_add(a: T, b: T) -> T; @@ -2576,7 +2526,8 @@ extern "rust-intrinsic" { /// The stabilized versions of this intrinsic are available on the integer /// primitives via the `wrapping_sub` method. For example, /// [`u32::wrapping_sub`] - #[rustc_const_stable(feature = "const_int_wrapping", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_wrapping", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn wrapping_sub(a: T, b: T) -> T; @@ -2590,7 +2541,8 @@ extern "rust-intrinsic" { /// The stabilized versions of this intrinsic are available on the integer /// primitives via the `wrapping_mul` method. For example, /// [`u32::wrapping_mul`] - #[rustc_const_stable(feature = "const_int_wrapping", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_wrapping", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn wrapping_mul(a: T, b: T) -> T; @@ -2605,7 +2557,8 @@ extern "rust-intrinsic" { /// The stabilized versions of this intrinsic are available on the integer /// primitives via the `saturating_add` method. For example, /// [`u32::saturating_add`] - #[rustc_const_stable(feature = "const_int_saturating", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_saturating", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn saturating_add(a: T, b: T) -> T; @@ -2619,7 +2572,8 @@ extern "rust-intrinsic" { /// The stabilized versions of this intrinsic are available on the integer /// primitives via the `saturating_sub` method. For example, /// [`u32::saturating_sub`] - #[rustc_const_stable(feature = "const_int_saturating", since = "1.40.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_saturating", since = "1.40.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn saturating_sub(a: T, b: T) -> T; @@ -2630,7 +2584,8 @@ extern "rust-intrinsic" { /// This intrinsic can *only* be called where the pointer is a local without /// projections (`read_via_copy(ptr)`, not `read_via_copy(*ptr)`) so that it /// trivially obeys runtime-MIR rules about derefs in operands. - #[rustc_const_stable(feature = "const_ptr_read", since = "1.71.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_ptr_read", since = "1.71.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn read_via_copy(ptr: *const T) -> T; @@ -2640,7 +2595,8 @@ extern "rust-intrinsic" { /// This intrinsic can *only* be called where the pointer is a local without /// projections (`write_via_move(ptr, x)`, not `write_via_move(*ptr, x)`) so /// that it trivially obeys runtime-MIR rules about derefs in operands. - #[rustc_const_unstable(feature = "const_ptr_write", issue = "86302")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_ptr_write", since = "1.83.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn write_via_move(ptr: *mut T, value: T); @@ -2653,7 +2609,8 @@ extern "rust-intrinsic" { /// any safety invariants. /// /// The stabilized version of this intrinsic is [`core::mem::discriminant`]. - #[rustc_const_stable(feature = "const_discriminant", since = "1.75.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_discriminant", since = "1.75.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_safe_intrinsic] #[rustc_nounwind] pub fn discriminant_value(v: &T) -> ::Discriminant; @@ -2663,12 +2620,17 @@ extern "rust-intrinsic" { /// /// `catch_fn` must not unwind. /// - /// The third argument is a function called if an unwind occurs (both Rust unwinds and foreign - /// unwinds). This function takes the data pointer and a pointer to the target-specific - /// exception object that was caught. For more information, see the compiler's source as well as - /// std's `catch_unwind` implementation. + /// The third argument is a function called if an unwind occurs (both Rust `panic` and foreign + /// unwinds). This function takes the data pointer and a pointer to the target- and + /// runtime-specific exception object that was caught. /// - /// The stable version of this intrinsic is `std::panic::catch_unwind`. + /// Note that in the case of a foreign unwinding operation, the exception object data may not be + /// safely usable from Rust, and should not be directly exposed via the standard library. To + /// prevent unsafe access, the library implementation may either abort the process or present an + /// opaque error type to the user. + /// + /// For more information, see the compiler's source, as well as the documentation for the stable + /// version of this intrinsic, `std::panic::catch_unwind`. #[rustc_nounwind] pub fn catch_unwind(try_fn: fn(*mut u8), data: *mut u8, catch_fn: fn(*mut u8, *mut u8)) -> i32; @@ -2682,7 +2644,8 @@ extern "rust-intrinsic" { pub fn nontemporal_store(ptr: *mut T, val: T); /// See documentation of `<*const T>::offset_from` for details. - #[rustc_const_stable(feature = "const_ptr_offset_from", since = "1.65.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_ptr_offset_from", since = "1.65.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn ptr_offset_from(ptr: *const T, base: *const T) -> isize; @@ -2737,7 +2700,7 @@ extern "rust-intrinsic" { /// Lexicographically compare `[left, left + bytes)` and `[right, right + bytes)` /// as unsigned bytes, returning negative if `left` is less, zero if all the - /// bytes match, or positive if `right` is greater. + /// bytes match, or positive if `left` is greater. /// /// This underlies things like `<[u8]>::cmp`, and will usually lower to `memcmp`. /// @@ -2892,7 +2855,6 @@ where /// #![feature(is_val_statically_known)] /// #![feature(core_intrinsics)] /// # #![allow(internal_features)] -/// #![feature(strict_provenance)] /// use std::intrinsics::is_val_statically_known; /// /// fn foo(x: &i32) -> bool { @@ -2955,7 +2917,8 @@ pub const unsafe fn typed_swap(x: *mut T, y: *mut T) { /// assertions are enabled whenever the *user crate* has UB checks enabled. However, if the /// user has UB checks disabled, the checks will still get optimized out. This intrinsic is /// primarily used by [`ub_checks::assert_unsafe_precondition`]. -#[rustc_const_unstable(feature = "const_ub_checks", issue = "none")] +#[cfg_attr(bootstrap, rustc_const_unstable(feature = "const_ub_checks", issue = "none"))] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] // just for UB checks #[unstable(feature = "core_intrinsics", issue = "none")] #[inline(always)] #[rustc_intrinsic] @@ -3040,7 +3003,8 @@ pub unsafe fn vtable_align(_ptr: *const ()) -> usize { /// The stabilized version of this intrinsic is [`core::mem::size_of`]. #[rustc_nounwind] #[unstable(feature = "core_intrinsics", issue = "none")] -#[rustc_const_stable(feature = "const_size_of", since = "1.40.0")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "const_size_of", since = "1.40.0"))] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_intrinsic] #[rustc_intrinsic_must_be_overridden] pub const fn size_of() -> usize { @@ -3057,7 +3021,8 @@ pub const fn size_of() -> usize { /// The stabilized version of this intrinsic is [`core::mem::align_of`]. #[rustc_nounwind] #[unstable(feature = "core_intrinsics", issue = "none")] -#[rustc_const_stable(feature = "const_min_align_of", since = "1.40.0")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "const_min_align_of", since = "1.40.0"))] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_intrinsic] #[rustc_intrinsic_must_be_overridden] pub const fn min_align_of() -> usize { @@ -3170,7 +3135,8 @@ pub const fn type_id() -> u128 { /// change the possible layouts of pointers. #[rustc_nounwind] #[unstable(feature = "core_intrinsics", issue = "none")] -#[rustc_const_unstable(feature = "ptr_metadata", issue = "81513")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "ptr_metadata_const", since = "1.83.0"))] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_intrinsic] #[rustc_intrinsic_must_be_overridden] pub const fn aggregate_raw_ptr, D, M>(_data: D, _meta: M) -> P { @@ -3195,7 +3161,11 @@ impl AggregateRawPtr<*mut T> for *mut P { /// This is used to implement functions like `ptr::metadata`. #[rustc_nounwind] #[unstable(feature = "core_intrinsics", issue = "none")] -#[rustc_const_unstable(feature = "ptr_metadata", issue = "81513")] +#[cfg_attr( + bootstrap, + cfg_attr(bootstrap, rustc_const_stable(feature = "ptr_metadata_const", since = "1.83.0")) +)] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_intrinsic] #[rustc_intrinsic_must_be_overridden] pub const fn ptr_metadata + ?Sized, M>(_ptr: *const P) -> M { @@ -3242,7 +3212,7 @@ pub const fn ptr_metadata + ?Sized, M>(_ptr: *cons /// [violate memory safety][read-ownership]. /// /// Note that even if the effectively copied size (`count * size_of::()`) is -/// `0`, the pointers must be non-null and properly aligned. +/// `0`, the pointers must be properly aligned. /// /// [`read`]: crate::ptr::read /// [read-ownership]: crate::ptr::read#ownership-of-the-returned-value @@ -3296,13 +3266,21 @@ pub const fn ptr_metadata + ?Sized, M>(_ptr: *cons #[doc(alias = "memcpy")] #[stable(feature = "rust1", since = "1.0.0")] #[rustc_allowed_through_unstable_modules] -#[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] +#[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[rustc_diagnostic_item = "ptr_copy_nonoverlapping"] pub const unsafe fn copy_nonoverlapping(src: *const T, dst: *mut T, count: usize) { extern "rust-intrinsic" { - #[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] + #[cfg_attr( + bootstrap, + rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0") + )] + #[cfg_attr( + not(bootstrap), + rustc_const_unstable(feature = "core_intrinsics", issue = "none") + )] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] pub fn copy_nonoverlapping(src: *const T, dst: *mut T, count: usize); } @@ -3317,10 +3295,12 @@ pub const unsafe fn copy_nonoverlapping(src: *const T, dst: *mut T, count: us size: usize = size_of::(), align: usize = align_of::(), count: usize = count, - ) => - ub_checks::is_aligned_and_not_null(src, align) - && ub_checks::is_aligned_and_not_null(dst, align) - && ub_checks::is_nonoverlapping(src, dst, size, count) + ) => { + let zero_size = count == 0 || size == 0; + ub_checks::is_aligned_and_not_null(src, align, zero_size) + && ub_checks::is_aligned_and_not_null(dst, align, zero_size) + && ub_checks::is_nonoverlapping(src, dst, size, count) + } ); // SAFETY: the safety contract for `copy_nonoverlapping` must be @@ -3363,7 +3343,7 @@ pub const unsafe fn copy_nonoverlapping(src: *const T, dst: *mut T, count: us /// [violate memory safety][read-ownership]. /// /// Note that even if the effectively copied size (`count * size_of::()`) is -/// `0`, the pointers must be non-null and properly aligned. +/// `0`, the pointers must be properly aligned. /// /// [`read`]: crate::ptr::read /// [read-ownership]: crate::ptr::read#ownership-of-the-returned-value @@ -3398,13 +3378,21 @@ pub const unsafe fn copy_nonoverlapping(src: *const T, dst: *mut T, count: us #[doc(alias = "memmove")] #[stable(feature = "rust1", since = "1.0.0")] #[rustc_allowed_through_unstable_modules] -#[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] +#[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[rustc_diagnostic_item = "ptr_copy"] pub const unsafe fn copy(src: *const T, dst: *mut T, count: usize) { extern "rust-intrinsic" { - #[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] + #[cfg_attr( + bootstrap, + rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0") + )] + #[cfg_attr( + not(bootstrap), + rustc_const_unstable(feature = "core_intrinsics", issue = "none") + )] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] fn copy(src: *const T, dst: *mut T, count: usize); } @@ -3418,9 +3406,10 @@ pub const unsafe fn copy(src: *const T, dst: *mut T, count: usize) { src: *const () = src as *const (), dst: *mut () = dst as *mut (), align: usize = align_of::(), + zero_size: bool = T::IS_ZST || count == 0, ) => - ub_checks::is_aligned_and_not_null(src, align) - && ub_checks::is_aligned_and_not_null(dst, align) + ub_checks::is_aligned_and_not_null(src, align, zero_size) + && ub_checks::is_aligned_and_not_null(dst, align, zero_size) ); copy(src, dst, count) } @@ -3443,7 +3432,7 @@ pub const unsafe fn copy(src: *const T, dst: *mut T, count: usize) { /// * `dst` must be properly aligned. /// /// Note that even if the effectively copied size (`count * size_of::()`) is -/// `0`, the pointer must be non-null and properly aligned. +/// `0`, the pointer must be properly aligned. /// /// Additionally, note that changing `*dst` in this way can easily lead to undefined behavior (UB) /// later if the written bytes are not a valid representation of some `T`. For instance, the @@ -3478,13 +3467,14 @@ pub const unsafe fn copy(src: *const T, dst: *mut T, count: usize) { #[doc(alias = "memset")] #[stable(feature = "rust1", since = "1.0.0")] #[rustc_allowed_through_unstable_modules] -#[rustc_const_unstable(feature = "const_ptr_write", issue = "86302")] +#[rustc_const_stable(feature = "const_ptr_write", since = "1.83.0")] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[rustc_diagnostic_item = "ptr_write_bytes"] pub const unsafe fn write_bytes(dst: *mut T, val: u8, count: usize) { extern "rust-intrinsic" { - #[rustc_const_unstable(feature = "const_ptr_write", issue = "86302")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_ptr_write", since = "1.83.0"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] #[rustc_nounwind] fn write_bytes(dst: *mut T, val: u8, count: usize); } @@ -3497,14 +3487,255 @@ pub const unsafe fn write_bytes(dst: *mut T, val: u8, count: usize) { ( addr: *const () = dst as *const (), align: usize = align_of::(), - ) => ub_checks::is_aligned_and_not_null(addr, align) + zero_size: bool = T::IS_ZST || count == 0, + ) => ub_checks::is_aligned_and_not_null(addr, align, zero_size) ); write_bytes(dst, val, count) } } +/// Returns the minimum of two `f16` values. +/// +/// Note that, unlike most intrinsics, this is safe to call; +/// it does not require an `unsafe` block. +/// Therefore, implementations must not require the user to uphold +/// any safety invariants. +/// +/// The stabilized version of this intrinsic is +/// [`f16::min`] +#[rustc_nounwind] +// #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_const_unstable(feature = "f16", issue = "116909")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const fn minnumf16(_x: f16, _y: f16) -> f16 { + unimplemented!(); +} + +/// Returns the minimum of two `f32` values. +/// +/// Note that, unlike most intrinsics, this is safe to call; +/// it does not require an `unsafe` block. +/// Therefore, implementations must not require the user to uphold +/// any safety invariants. +/// +/// The stabilized version of this intrinsic is +/// [`f32::min`] +#[rustc_nounwind] +#[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const fn minnumf32(_x: f32, _y: f32) -> f32 { + unimplemented!(); +} + +/// Returns the minimum of two `f64` values. +/// +/// Note that, unlike most intrinsics, this is safe to call; +/// it does not require an `unsafe` block. +/// Therefore, implementations must not require the user to uphold +/// any safety invariants. +/// +/// The stabilized version of this intrinsic is +/// [`f64::min`] +#[rustc_nounwind] +#[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const fn minnumf64(_x: f64, _y: f64) -> f64 { + unimplemented!(); +} + +/// Returns the minimum of two `f128` values. +/// +/// Note that, unlike most intrinsics, this is safe to call; +/// it does not require an `unsafe` block. +/// Therefore, implementations must not require the user to uphold +/// any safety invariants. +/// +/// The stabilized version of this intrinsic is +/// [`f128::min`] +#[rustc_nounwind] +// #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_const_unstable(feature = "f128", issue = "116909")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const fn minnumf128(_x: f128, _y: f128) -> f128 { + unimplemented!(); +} + +/// Returns the maximum of two `f16` values. +/// +/// Note that, unlike most intrinsics, this is safe to call; +/// it does not require an `unsafe` block. +/// Therefore, implementations must not require the user to uphold +/// any safety invariants. +/// +/// The stabilized version of this intrinsic is +/// [`f16::max`] +#[rustc_nounwind] +// #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_const_unstable(feature = "f16", issue = "116909")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const fn maxnumf16(_x: f16, _y: f16) -> f16 { + unimplemented!(); +} + +/// Returns the maximum of two `f32` values. +/// +/// Note that, unlike most intrinsics, this is safe to call; +/// it does not require an `unsafe` block. +/// Therefore, implementations must not require the user to uphold +/// any safety invariants. +/// +/// The stabilized version of this intrinsic is +/// [`f32::max`] +#[rustc_nounwind] +#[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const fn maxnumf32(_x: f32, _y: f32) -> f32 { + unimplemented!(); +} + +/// Returns the maximum of two `f64` values. +/// +/// Note that, unlike most intrinsics, this is safe to call; +/// it does not require an `unsafe` block. +/// Therefore, implementations must not require the user to uphold +/// any safety invariants. +/// +/// The stabilized version of this intrinsic is +/// [`f64::max`] +#[rustc_nounwind] +#[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const fn maxnumf64(_x: f64, _y: f64) -> f64 { + unimplemented!(); +} + +/// Returns the maximum of two `f128` values. +/// +/// Note that, unlike most intrinsics, this is safe to call; +/// it does not require an `unsafe` block. +/// Therefore, implementations must not require the user to uphold +/// any safety invariants. +/// +/// The stabilized version of this intrinsic is +/// [`f128::max`] +#[rustc_nounwind] +// #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_const_unstable(feature = "f128", issue = "116909")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const fn maxnumf128(_x: f128, _y: f128) -> f128 { + unimplemented!(); +} + +/// Returns the absolute value of an `f16`. +/// +/// The stabilized version of this intrinsic is +/// [`f16::abs`](../../std/primitive.f16.html#method.abs) +#[rustc_nounwind] +// #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_const_unstable(feature = "f16", issue = "116909")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const unsafe fn fabsf16(_x: f16) -> f16 { + unimplemented!(); +} + +/// Returns the absolute value of an `f32`. +/// +/// The stabilized version of this intrinsic is +/// [`f32::abs`](../../std/primitive.f32.html#method.abs) +#[rustc_nounwind] +#[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const unsafe fn fabsf32(_x: f32) -> f32 { + unimplemented!(); +} + +/// Returns the absolute value of an `f64`. +/// +/// The stabilized version of this intrinsic is +/// [`f64::abs`](../../std/primitive.f64.html#method.abs) +#[rustc_nounwind] +#[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const unsafe fn fabsf64(_x: f64) -> f64 { + unimplemented!(); +} + +/// Returns the absolute value of an `f128`. +/// +/// The stabilized version of this intrinsic is +/// [`f128::abs`](../../std/primitive.f128.html#method.abs) +#[rustc_nounwind] +// #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_const_unstable(feature = "f128", issue = "116909")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const unsafe fn fabsf128(_x: f128) -> f128 { + unimplemented!(); +} + +/// Copies the sign from `y` to `x` for `f16` values. +/// +/// The stabilized version of this intrinsic is +/// [`f16::copysign`](../../std/primitive.f16.html#method.copysign) +#[rustc_nounwind] +// #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_const_unstable(feature = "f16", issue = "116909")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const unsafe fn copysignf16(_x: f16, _y: f16) -> f16 { + unimplemented!(); +} + +/// Copies the sign from `y` to `x` for `f32` values. +/// +/// The stabilized version of this intrinsic is +/// [`f32::copysign`](../../std/primitive.f32.html#method.copysign) +#[rustc_nounwind] +#[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const unsafe fn copysignf32(_x: f32, _y: f32) -> f32 { + unimplemented!(); +} +/// Copies the sign from `y` to `x` for `f64` values. +/// +/// The stabilized version of this intrinsic is +/// [`f64::copysign`](../../std/primitive.f64.html#method.copysign) +#[rustc_nounwind] +#[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const unsafe fn copysignf64(_x: f64, _y: f64) -> f64 { + unimplemented!(); +} + +/// Copies the sign from `y` to `x` for `f128` values. +/// +/// The stabilized version of this intrinsic is +/// [`f128::copysign`](../../std/primitive.f128.html#method.copysign) +#[rustc_nounwind] +// #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] +#[rustc_const_unstable(feature = "f128", issue = "116909")] +#[rustc_intrinsic] +#[rustc_intrinsic_must_be_overridden] +pub const unsafe fn copysignf128(_x: f128, _y: f128) -> f128 { + unimplemented!(); +} + /// Inform Miri that a given pointer definitely has a certain alignment. #[cfg(miri)] +#[rustc_allow_const_fn_unstable(const_eval_select)] pub(crate) const fn miri_promise_symbolic_alignment(ptr: *const (), align: usize) { extern "Rust" { /// Miri-provided extern function to promise that a given pointer is properly aligned for diff --git a/library/core/src/intrinsics/mir.rs b/library/core/src/intrinsics/mir.rs index fb0aa5398a55b..6539964bc0956 100644 --- a/library/core/src/intrinsics/mir.rs +++ b/library/core/src/intrinsics/mir.rs @@ -213,7 +213,7 @@ //! - All other locals need to be declared with `let` somewhere and then can be accessed by name. //! //! #### Places -//! - Locals implicit convert to places. +//! - Locals implicitly convert to places. //! - Field accesses, derefs, and indexing work normally. //! - Fields in variants can be accessed via the [`Variant`] and [`Field`] associated functions, //! see their documentation for details. @@ -298,7 +298,7 @@ define!( ); define!( "mir_unwind_unreachable", - /// An unwind action that triggers undefined behaviour. + /// An unwind action that triggers undefined behavior. fn UnwindUnreachable() -> UnwindActionArg ); define!( diff --git a/library/core/src/intrinsics/simd.rs b/library/core/src/intrinsics/simd.rs index 5982819809937..5ddca9c4dce88 100644 --- a/library/core/src/intrinsics/simd.rs +++ b/library/core/src/intrinsics/simd.rs @@ -232,7 +232,7 @@ extern "rust-intrinsic" { /// /// `T` must be a vector. /// - /// `U` must be a **const** array or vector of `u32`s. This means it must either refer to a named + /// `U` must be a **const** vector of `u32`s. This means it must either refer to a named /// const or be given as an inline const expression (`const { ... }`). /// /// `V` must be a vector with the same element type as `T` and the same length as `U`. diff --git a/library/core/src/iter/adapters/filter_map.rs b/library/core/src/iter/adapters/filter_map.rs index 914ef6131771f..cc64ceb13f766 100644 --- a/library/core/src/iter/adapters/filter_map.rs +++ b/library/core/src/iter/adapters/filter_map.rs @@ -3,7 +3,6 @@ use crate::iter::{FusedIterator, InPlaceIterable, TrustedFused}; use crate::mem::{ManuallyDrop, MaybeUninit}; use crate::num::NonZero; use crate::ops::{ControlFlow, Try}; -use crate::ptr::addr_of; use crate::{array, fmt}; /// An iterator that uses `f` to both filter and map elements from `iter`. @@ -101,7 +100,7 @@ where unsafe { let opt_payload_at: *const MaybeUninit = - addr_of!(val).byte_add(core::mem::offset_of!(Option, Some.0)).cast(); + (&raw const val).byte_add(core::mem::offset_of!(Option, Some.0)).cast(); let dst = guard.array.as_mut_ptr().add(idx); crate::ptr::copy_nonoverlapping(opt_payload_at, dst, 1); crate::mem::forget(val); diff --git a/library/core/src/iter/adapters/fuse.rs b/library/core/src/iter/adapters/fuse.rs index 7781ed088b76c..e9765f911a252 100644 --- a/library/core/src/iter/adapters/fuse.rs +++ b/library/core/src/iter/adapters/fuse.rs @@ -1,6 +1,6 @@ use crate::intrinsics; -use crate::iter::adapters::zip::try_get_unchecked; use crate::iter::adapters::SourceIter; +use crate::iter::adapters::zip::try_get_unchecked; use crate::iter::{ FusedIterator, TrustedFused, TrustedLen, TrustedRandomAccess, TrustedRandomAccessNoCoerce, }; diff --git a/library/core/src/iter/adapters/map_while.rs b/library/core/src/iter/adapters/map_while.rs index 4e7327938d72a..c047c40de050e 100644 --- a/library/core/src/iter/adapters/map_while.rs +++ b/library/core/src/iter/adapters/map_while.rs @@ -1,6 +1,6 @@ use crate::fmt; -use crate::iter::adapters::SourceIter; use crate::iter::InPlaceIterable; +use crate::iter::adapters::SourceIter; use crate::num::NonZero; use crate::ops::{ControlFlow, Try}; diff --git a/library/core/src/iter/adapters/mod.rs b/library/core/src/iter/adapters/mod.rs index 96158c43318ea..2a0ef0189d165 100644 --- a/library/core/src/iter/adapters/mod.rs +++ b/library/core/src/iter/adapters/mod.rs @@ -48,12 +48,12 @@ pub use self::map_while::MapWhile; pub use self::map_windows::MapWindows; #[stable(feature = "iterator_step_by", since = "1.28.0")] pub use self::step_by::StepBy; -#[stable(feature = "iter_zip", since = "1.59.0")] -pub use self::zip::zip; #[unstable(feature = "trusted_random_access", issue = "none")] pub use self::zip::TrustedRandomAccess; #[unstable(feature = "trusted_random_access", issue = "none")] pub use self::zip::TrustedRandomAccessNoCoerce; +#[stable(feature = "iter_zip", since = "1.59.0")] +pub use self::zip::zip; #[stable(feature = "rust1", since = "1.0.0")] pub use self::{ chain::Chain, cycle::Cycle, enumerate::Enumerate, filter::Filter, filter_map::FilterMap, @@ -71,7 +71,7 @@ pub use self::{ /// this can be useful for specializing [`FromIterator`] implementations or recovering the /// remaining elements after an iterator has been partially exhausted. /// -/// Note that implementations do not necessarily have to provide access to the inner-most +/// Note that implementations do not necessarily have to provide access to the innermost /// source of a pipeline. A stateful intermediate adapter might eagerly evaluate a part /// of the pipeline and expose its internal storage as source. /// diff --git a/library/core/src/iter/adapters/peekable.rs b/library/core/src/iter/adapters/peekable.rs index a11b73cbe8e2d..cc12cd9c35601 100644 --- a/library/core/src/iter/adapters/peekable.rs +++ b/library/core/src/iter/adapters/peekable.rs @@ -269,7 +269,7 @@ impl Peekable { /// let mut iter = (0..5).peekable(); /// // The first item of the iterator is 0; consume it. /// assert_eq!(iter.next_if(|&x| x == 0), Some(0)); - /// // The next item returned is now 1, so `consume` will return `false`. + /// // The next item returned is now 1, so `next_if` will return `None`. /// assert_eq!(iter.next_if(|&x| x == 0), None); /// // `next_if` saves the value of the next item if it was not equal to `expected`. /// assert_eq!(iter.next(), Some(1)); @@ -304,7 +304,7 @@ impl Peekable { /// let mut iter = (0..5).peekable(); /// // The first item of the iterator is 0; consume it. /// assert_eq!(iter.next_if_eq(&0), Some(0)); - /// // The next item returned is now 1, so `consume` will return `false`. + /// // The next item returned is now 1, so `next_if` will return `None`. /// assert_eq!(iter.next_if_eq(&0), None); /// // `next_if_eq` saves the value of the next item if it was not equal to `expected`. /// assert_eq!(iter.next(), Some(1)); diff --git a/library/core/src/iter/adapters/scan.rs b/library/core/src/iter/adapters/scan.rs index 7ba7ed2fdd0d5..e12375c94e067 100644 --- a/library/core/src/iter/adapters/scan.rs +++ b/library/core/src/iter/adapters/scan.rs @@ -1,6 +1,6 @@ use crate::fmt; -use crate::iter::adapters::SourceIter; use crate::iter::InPlaceIterable; +use crate::iter::adapters::SourceIter; use crate::num::NonZero; use crate::ops::{ControlFlow, Try}; diff --git a/library/core/src/iter/adapters/skip.rs b/library/core/src/iter/adapters/skip.rs index 8ba2e2a8f2dd7..55c4a7f14fbd6 100644 --- a/library/core/src/iter/adapters/skip.rs +++ b/library/core/src/iter/adapters/skip.rs @@ -1,6 +1,6 @@ use crate::intrinsics::unlikely; -use crate::iter::adapters::zip::try_get_unchecked; use crate::iter::adapters::SourceIter; +use crate::iter::adapters::zip::try_get_unchecked; use crate::iter::{ FusedIterator, InPlaceIterable, TrustedFused, TrustedLen, TrustedRandomAccess, TrustedRandomAccessNoCoerce, diff --git a/library/core/src/iter/adapters/step_by.rs b/library/core/src/iter/adapters/step_by.rs index 72eb72a76c66c..2d0f210420317 100644 --- a/library/core/src/iter/adapters/step_by.rs +++ b/library/core/src/iter/adapters/step_by.rs @@ -1,5 +1,5 @@ use crate::intrinsics; -use crate::iter::{from_fn, TrustedLen, TrustedRandomAccess}; +use crate::iter::{TrustedLen, TrustedRandomAccess, from_fn}; use crate::num::NonZero; use crate::ops::{Range, Try}; diff --git a/library/core/src/iter/mod.rs b/library/core/src/iter/mod.rs index 387963d0afd01..635e14e769a84 100644 --- a/library/core/src/iter/mod.rs +++ b/library/core/src/iter/mod.rs @@ -380,11 +380,6 @@ macro_rules! impl_fold_via_try_fold { }; } -#[unstable(feature = "iter_chain", reason = "recently added", issue = "125964")] -pub use self::adapters::chain; -pub(crate) use self::adapters::try_process; -#[stable(feature = "iter_zip", since = "1.59.0")] -pub use self::adapters::zip; #[unstable(feature = "iter_array_chunks", reason = "recently added", issue = "100450")] pub use self::adapters::ArrayChunks; #[unstable(feature = "std_internals", issue = "none")] @@ -407,6 +402,11 @@ pub use self::adapters::StepBy; pub use self::adapters::TrustedRandomAccess; #[unstable(feature = "trusted_random_access", issue = "none")] pub use self::adapters::TrustedRandomAccessNoCoerce; +#[unstable(feature = "iter_chain", reason = "recently added", issue = "125964")] +pub use self::adapters::chain; +pub(crate) use self::adapters::try_process; +#[stable(feature = "iter_zip", since = "1.59.0")] +pub use self::adapters::zip; #[stable(feature = "rust1", since = "1.0.0")] pub use self::adapters::{ Chain, Cycle, Enumerate, Filter, FilterMap, FlatMap, Fuse, Inspect, Map, Peekable, Rev, Scan, @@ -427,21 +427,21 @@ pub use self::range::Step; )] pub use self::sources::from_coroutine; #[stable(feature = "iter_empty", since = "1.2.0")] -pub use self::sources::{empty, Empty}; +pub use self::sources::{Empty, empty}; #[stable(feature = "iter_from_fn", since = "1.34.0")] -pub use self::sources::{from_fn, FromFn}; +pub use self::sources::{FromFn, from_fn}; #[stable(feature = "iter_once", since = "1.2.0")] -pub use self::sources::{once, Once}; +pub use self::sources::{Once, once}; #[stable(feature = "iter_once_with", since = "1.43.0")] -pub use self::sources::{once_with, OnceWith}; +pub use self::sources::{OnceWith, once_with}; #[stable(feature = "rust1", since = "1.0.0")] -pub use self::sources::{repeat, Repeat}; +pub use self::sources::{Repeat, repeat}; #[stable(feature = "iter_repeat_n", since = "1.82.0")] -pub use self::sources::{repeat_n, RepeatN}; +pub use self::sources::{RepeatN, repeat_n}; #[stable(feature = "iterator_repeat_with", since = "1.28.0")] -pub use self::sources::{repeat_with, RepeatWith}; +pub use self::sources::{RepeatWith, repeat_with}; #[stable(feature = "iter_successors", since = "1.34.0")] -pub use self::sources::{successors, Successors}; +pub use self::sources::{Successors, successors}; #[stable(feature = "fused", since = "1.26.0")] pub use self::traits::FusedIterator; #[unstable(issue = "none", feature = "inplace_iteration")] diff --git a/library/core/src/iter/sources.rs b/library/core/src/iter/sources.rs index 2c726fbca8760..c635992dfbd1b 100644 --- a/library/core/src/iter/sources.rs +++ b/library/core/src/iter/sources.rs @@ -9,7 +9,7 @@ mod repeat_with; mod successors; #[stable(feature = "iter_empty", since = "1.2.0")] -pub use self::empty::{empty, Empty}; +pub use self::empty::{Empty, empty}; #[unstable( feature = "iter_from_coroutine", issue = "43122", @@ -17,16 +17,16 @@ pub use self::empty::{empty, Empty}; )] pub use self::from_coroutine::from_coroutine; #[stable(feature = "iter_from_fn", since = "1.34.0")] -pub use self::from_fn::{from_fn, FromFn}; +pub use self::from_fn::{FromFn, from_fn}; #[stable(feature = "iter_once", since = "1.2.0")] -pub use self::once::{once, Once}; +pub use self::once::{Once, once}; #[stable(feature = "iter_once_with", since = "1.43.0")] -pub use self::once_with::{once_with, OnceWith}; +pub use self::once_with::{OnceWith, once_with}; #[stable(feature = "rust1", since = "1.0.0")] -pub use self::repeat::{repeat, Repeat}; +pub use self::repeat::{Repeat, repeat}; #[stable(feature = "iter_repeat_n", since = "1.82.0")] -pub use self::repeat_n::{repeat_n, RepeatN}; +pub use self::repeat_n::{RepeatN, repeat_n}; #[stable(feature = "iterator_repeat_with", since = "1.28.0")] -pub use self::repeat_with::{repeat_with, RepeatWith}; +pub use self::repeat_with::{RepeatWith, repeat_with}; #[stable(feature = "iter_successors", since = "1.34.0")] -pub use self::successors::{successors, Successors}; +pub use self::successors::{Successors, successors}; diff --git a/library/core/src/iter/sources/repeat_n.rs b/library/core/src/iter/sources/repeat_n.rs index 9c0621933638e..cc089c617c0e3 100644 --- a/library/core/src/iter/sources/repeat_n.rs +++ b/library/core/src/iter/sources/repeat_n.rs @@ -1,5 +1,6 @@ +use crate::fmt; use crate::iter::{FusedIterator, TrustedLen, UncheckedIterator}; -use crate::mem::ManuallyDrop; +use crate::mem::{self, MaybeUninit}; use crate::num::NonZero; /// Creates a new iterator that repeats a single element a given number of times. @@ -7,9 +8,7 @@ use crate::num::NonZero; /// The `repeat_n()` function repeats a single value exactly `n` times. /// /// This is very similar to using [`repeat()`] with [`Iterator::take()`], -/// but there are two differences: -/// - `repeat_n()` can return the original value, rather than always cloning. -/// - `repeat_n()` produces an [`ExactSizeIterator`]. +/// but `repeat_n()` can return the original value, rather than always cloning. /// /// [`repeat()`]: crate::iter::repeat /// @@ -58,14 +57,12 @@ use crate::num::NonZero; #[inline] #[stable(feature = "iter_repeat_n", since = "1.82.0")] pub fn repeat_n(element: T, count: usize) -> RepeatN { - let mut element = ManuallyDrop::new(element); - - if count == 0 { - // SAFETY: we definitely haven't dropped it yet, since we only just got - // passed it in, and because the count is zero the instance we're about - // to create won't drop it, so to avoid leaking we need to now. - unsafe { ManuallyDrop::drop(&mut element) }; - } + let element = if count == 0 { + // `element` gets dropped eagerly. + MaybeUninit::uninit() + } else { + MaybeUninit::new(element) + }; RepeatN { element, count } } @@ -74,15 +71,23 @@ pub fn repeat_n(element: T, count: usize) -> RepeatN { /// /// This `struct` is created by the [`repeat_n()`] function. /// See its documentation for more. -#[derive(Clone, Debug)] #[stable(feature = "iter_repeat_n", since = "1.82.0")] pub struct RepeatN { count: usize, - // Invariant: has been dropped iff count == 0. - element: ManuallyDrop, + // Invariant: uninit iff count == 0. + element: MaybeUninit, } impl RepeatN { + /// Returns the element if it hasn't been dropped already. + fn element_ref(&self) -> Option<&A> { + if self.count > 0 { + // SAFETY: The count is non-zero, so it must be initialized. + Some(unsafe { self.element.assume_init_ref() }) + } else { + None + } + } /// If we haven't already dropped the element, return it in an option. /// /// Clears the count so it won't be dropped again later. @@ -90,15 +95,36 @@ impl RepeatN { fn take_element(&mut self) -> Option { if self.count > 0 { self.count = 0; + let element = mem::replace(&mut self.element, MaybeUninit::uninit()); // SAFETY: We just set count to zero so it won't be dropped again, // and it used to be non-zero so it hasn't already been dropped. - unsafe { Some(ManuallyDrop::take(&mut self.element)) } + unsafe { Some(element.assume_init()) } } else { None } } } +#[stable(feature = "iter_repeat_n", since = "1.82.0")] +impl Clone for RepeatN { + fn clone(&self) -> RepeatN { + RepeatN { + count: self.count, + element: self.element_ref().cloned().map_or_else(MaybeUninit::uninit, MaybeUninit::new), + } + } +} + +#[stable(feature = "iter_repeat_n", since = "1.82.0")] +impl fmt::Debug for RepeatN { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("RepeatN") + .field("count", &self.count) + .field("element", &self.element_ref()) + .finish() + } +} + #[stable(feature = "iter_repeat_n", since = "1.82.0")] impl Drop for RepeatN { fn drop(&mut self) { @@ -194,9 +220,11 @@ impl UncheckedIterator for RepeatN { // SAFETY: the check above ensured that the count used to be non-zero, // so element hasn't been dropped yet, and we just lowered the count to // zero so it won't be dropped later, and thus it's okay to take it here. - unsafe { ManuallyDrop::take(&mut self.element) } + unsafe { mem::replace(&mut self.element, MaybeUninit::uninit()).assume_init() } } else { - A::clone(&self.element) + // SAFETY: the count is non-zero, so it must have not been dropped yet. + let element = unsafe { self.element.assume_init_ref() }; + A::clone(element) } } } diff --git a/library/core/src/iter/traits/collect.rs b/library/core/src/iter/traits/collect.rs index 86660f2e375c3..2cf2ea58fd4ee 100644 --- a/library/core/src/iter/traits/collect.rs +++ b/library/core/src/iter/traits/collect.rs @@ -346,7 +346,6 @@ pub trait IntoIterator { fn into_iter(self) -> Self::IntoIter; } -#[rustc_const_unstable(feature = "const_intoiterator_identity", issue = "90603")] #[stable(feature = "rust1", since = "1.0.0")] impl IntoIterator for I { type Item = I::Item; diff --git a/library/core/src/iter/traits/iterator.rs b/library/core/src/iter/traits/iterator.rs index 8352486ad416e..ffaf1bc56e942 100644 --- a/library/core/src/iter/traits/iterator.rs +++ b/library/core/src/iter/traits/iterator.rs @@ -1,15 +1,15 @@ use super::super::{ - try_process, ArrayChunks, ByRefSized, Chain, Cloned, Copied, Cycle, Enumerate, Filter, - FilterMap, FlatMap, Flatten, Fuse, Inspect, Intersperse, IntersperseWith, Map, MapWhile, - MapWindows, Peekable, Product, Rev, Scan, Skip, SkipWhile, StepBy, Sum, Take, TakeWhile, - TrustedRandomAccessNoCoerce, Zip, + ArrayChunks, ByRefSized, Chain, Cloned, Copied, Cycle, Enumerate, Filter, FilterMap, FlatMap, + Flatten, Fuse, Inspect, Intersperse, IntersperseWith, Map, MapWhile, MapWindows, Peekable, + Product, Rev, Scan, Skip, SkipWhile, StepBy, Sum, Take, TakeWhile, TrustedRandomAccessNoCoerce, + Zip, try_process, }; use crate::array; use crate::cmp::{self, Ordering}; use crate::num::NonZero; use crate::ops::{ChangeOutputType, ControlFlow, FromResidual, Residual, Try}; -fn _assert_is_object_safe(_: &dyn Iterator) {} +fn _assert_is_dyn_compatible(_: &dyn Iterator) {} /// A trait for dealing with iterators. /// @@ -106,7 +106,6 @@ pub trait Iterator { /// ``` #[inline] #[unstable(feature = "iter_next_chunk", reason = "recently added", issue = "98326")] - #[rustc_do_not_const_check] fn next_chunk( &mut self, ) -> Result<[Self::Item; N], array::IntoIter> @@ -184,7 +183,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn size_hint(&self) -> (usize, Option) { (0, None) } @@ -220,7 +218,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn count(self) -> usize where Self: Sized, @@ -249,7 +246,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn last(self) -> Option where Self: Sized, @@ -297,7 +293,6 @@ pub trait Iterator { /// ``` #[inline] #[unstable(feature = "iter_advance_by", reason = "recently added", issue = "77404")] - #[rustc_do_not_const_check] fn advance_by(&mut self, n: usize) -> Result<(), NonZero> { for i in 0..n { if self.next().is_none() { @@ -349,7 +344,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn nth(&mut self, n: usize) -> Option { self.advance_by(n).ok()?; self.next() @@ -400,7 +394,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "iterator_step_by", since = "1.28.0")] - #[rustc_do_not_const_check] fn step_by(self, step: usize) -> StepBy where Self: Sized, @@ -472,7 +465,6 @@ pub trait Iterator { /// [`OsStr`]: ../../std/ffi/struct.OsStr.html #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn chain(self, other: U) -> Chain where Self: Sized, @@ -591,7 +583,6 @@ pub trait Iterator { /// [`zip`]: crate::iter::zip #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn zip(self, other: U) -> Zip where Self: Sized, @@ -634,7 +625,6 @@ pub trait Iterator { /// [`intersperse_with`]: Iterator::intersperse_with #[inline] #[unstable(feature = "iter_intersperse", reason = "recently added", issue = "79524")] - #[rustc_do_not_const_check] fn intersperse(self, separator: Self::Item) -> Intersperse where Self: Sized, @@ -693,7 +683,6 @@ pub trait Iterator { /// [`intersperse`]: Iterator::intersperse #[inline] #[unstable(feature = "iter_intersperse", reason = "recently added", issue = "79524")] - #[rustc_do_not_const_check] fn intersperse_with(self, separator: G) -> IntersperseWith where Self: Sized, @@ -753,7 +742,6 @@ pub trait Iterator { #[rustc_diagnostic_item = "IteratorMap"] #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn map(self, f: F) -> Map where Self: Sized, @@ -799,7 +787,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "iterator_for_each", since = "1.21.0")] - #[rustc_do_not_const_check] fn for_each(self, f: F) where Self: Sized, @@ -875,7 +862,7 @@ pub trait Iterator { /// Note that `iter.filter(f).next()` is equivalent to `iter.find(f)`. #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] + #[cfg_attr(not(test), rustc_diagnostic_item = "iter_filter")] fn filter

(self, predicate: P) -> Filter where Self: Sized, @@ -921,7 +908,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn filter_map(self, f: F) -> FilterMap where Self: Sized, @@ -968,7 +954,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] #[cfg_attr(not(test), rustc_diagnostic_item = "enumerate_method")] fn enumerate(self) -> Enumerate where @@ -1041,7 +1026,6 @@ pub trait Iterator { /// [`next`]: Iterator::next #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn peekable(self) -> Peekable where Self: Sized, @@ -1107,7 +1091,6 @@ pub trait Iterator { #[inline] #[doc(alias = "drop_while")] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn skip_while

(self, predicate: P) -> SkipWhile where Self: Sized, @@ -1189,7 +1172,6 @@ pub trait Iterator { /// the iteration should stop, but wasn't placed back into the iterator. #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn take_while

(self, predicate: P) -> TakeWhile where Self: Sized, @@ -1278,7 +1260,6 @@ pub trait Iterator { /// [`fuse`]: Iterator::fuse #[inline] #[stable(feature = "iter_map_while", since = "1.57.0")] - #[rustc_do_not_const_check] fn map_while(self, predicate: P) -> MapWhile where Self: Sized, @@ -1308,7 +1289,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn skip(self, n: usize) -> Skip where Self: Sized, @@ -1362,7 +1342,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn take(self, n: usize) -> Take where Self: Sized, @@ -1410,7 +1389,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn scan(self, initial_state: St, f: F) -> Scan where Self: Sized, @@ -1449,7 +1427,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn flat_map(self, f: F) -> FlatMap where Self: Sized, @@ -1534,7 +1511,6 @@ pub trait Iterator { /// [`flat_map()`]: Iterator::flat_map #[inline] #[stable(feature = "iterator_flatten", since = "1.29.0")] - #[rustc_do_not_const_check] fn flatten(self) -> Flatten where Self: Sized, @@ -1691,7 +1667,6 @@ pub trait Iterator { /// ``` #[inline] #[unstable(feature = "iter_map_windows", reason = "recently added", issue = "87155")] - #[rustc_do_not_const_check] fn map_windows(self, f: F) -> MapWindows where Self: Sized, @@ -1758,7 +1733,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn fuse(self) -> Fuse where Self: Sized, @@ -1843,7 +1817,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn inspect(self, f: F) -> Inspect where Self: Sized, @@ -1872,7 +1845,6 @@ pub trait Iterator { /// assert_eq!(of_rust, vec!["of", "Rust"]); /// ``` #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn by_ref(&mut self) -> &mut Self where Self: Sized, @@ -1992,7 +1964,6 @@ pub trait Iterator { #[stable(feature = "rust1", since = "1.0.0")] #[must_use = "if you really need to exhaust the iterator, consider `.for_each(drop)` instead"] #[cfg_attr(not(test), rustc_diagnostic_item = "iterator_collect_fn")] - #[rustc_do_not_const_check] fn collect>(self) -> B where Self: Sized, @@ -2071,7 +2042,6 @@ pub trait Iterator { /// [`collect`]: Iterator::collect #[inline] #[unstable(feature = "iterator_try_collect", issue = "94047")] - #[rustc_do_not_const_check] fn try_collect(&mut self) -> ChangeOutputType where Self: Sized, @@ -2144,7 +2114,6 @@ pub trait Iterator { /// ``` #[inline] #[unstable(feature = "iter_collect_into", reason = "new API", issue = "94780")] - #[rustc_do_not_const_check] fn collect_into>(self, collection: &mut E) -> &mut E where Self: Sized, @@ -2177,7 +2146,6 @@ pub trait Iterator { /// assert_eq!(odd, vec![1, 3]); /// ``` #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn partition(self, f: F) -> (B, B) where Self: Sized, @@ -2240,7 +2208,6 @@ pub trait Iterator { /// assert!(a[i..].iter().all(|&n| n % 2 == 1)); // odds /// ``` #[unstable(feature = "iter_partition_in_place", reason = "new API", issue = "62543")] - #[rustc_do_not_const_check] fn partition_in_place<'a, T: 'a, P>(mut self, ref mut predicate: P) -> usize where Self: Sized + DoubleEndedIterator, @@ -2298,7 +2265,6 @@ pub trait Iterator { /// assert!(!"IntoIterator".chars().is_partitioned(char::is_uppercase)); /// ``` #[unstable(feature = "iter_is_partitioned", reason = "new API", issue = "62544")] - #[rustc_do_not_const_check] fn is_partitioned

(mut self, mut predicate: P) -> bool where Self: Sized, @@ -2393,7 +2359,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "iterator_try_fold", since = "1.27.0")] - #[rustc_do_not_const_check] fn try_fold(&mut self, init: B, mut f: F) -> R where Self: Sized, @@ -2452,7 +2417,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "iterator_try_fold", since = "1.27.0")] - #[rustc_do_not_const_check] fn try_for_each(&mut self, f: F) -> R where Self: Sized, @@ -2572,7 +2536,6 @@ pub trait Iterator { #[doc(alias = "inject", alias = "foldl")] #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn fold(mut self, init: B, mut f: F) -> B where Self: Sized, @@ -2610,7 +2573,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "iterator_fold_self", since = "1.51.0")] - #[rustc_do_not_const_check] fn reduce(mut self, f: F) -> Option where Self: Sized, @@ -2682,7 +2644,6 @@ pub trait Iterator { /// ``` #[inline] #[unstable(feature = "iterator_try_reduce", reason = "new API", issue = "87053")] - #[rustc_do_not_const_check] fn try_reduce( &mut self, f: impl FnMut(Self::Item, Self::Item) -> R, @@ -2741,7 +2702,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn all(&mut self, f: F) -> bool where Self: Sized, @@ -2795,7 +2755,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn any(&mut self, f: F) -> bool where Self: Sized, @@ -2859,7 +2818,6 @@ pub trait Iterator { /// Note that `iter.find(f)` is equivalent to `iter.filter(f).next()`. #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn find

(&mut self, predicate: P) -> Option where Self: Sized, @@ -2891,7 +2849,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "iterator_find_map", since = "1.30.0")] - #[rustc_do_not_const_check] fn find_map(&mut self, f: F) -> Option where Self: Sized, @@ -2950,7 +2907,6 @@ pub trait Iterator { /// ``` #[inline] #[unstable(feature = "try_find", reason = "new API", issue = "63178")] - #[rustc_do_not_const_check] fn try_find( &mut self, f: impl FnMut(&Self::Item) -> R, @@ -3034,7 +2990,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn position

(&mut self, predicate: P) -> Option where Self: Sized, @@ -3099,7 +3054,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn rposition

(self) -> P where Self: Sized, @@ -3623,7 +3565,6 @@ pub trait Iterator { /// assert_eq!([1, 2].iter().cmp([1].iter()), Ordering::Greater); /// ``` #[stable(feature = "iter_order", since = "1.5.0")] - #[rustc_do_not_const_check] fn cmp(self, other: I) -> Ordering where I: IntoIterator, @@ -3651,7 +3592,6 @@ pub trait Iterator { /// assert_eq!(xs.iter().cmp_by(&ys, |&x, &y| (2 * x).cmp(&y)), Ordering::Greater); /// ``` #[unstable(feature = "iter_order_by", issue = "64295")] - #[rustc_do_not_const_check] fn cmp_by(self, other: I, cmp: F) -> Ordering where Self: Sized, @@ -3708,7 +3648,6 @@ pub trait Iterator { /// ``` /// #[stable(feature = "iter_order", since = "1.5.0")] - #[rustc_do_not_const_check] fn partial_cmp(self, other: I) -> Option where I: IntoIterator, @@ -3745,7 +3684,6 @@ pub trait Iterator { /// ); /// ``` #[unstable(feature = "iter_order_by", issue = "64295")] - #[rustc_do_not_const_check] fn partial_cmp_by(self, other: I, partial_cmp: F) -> Option where Self: Sized, @@ -3779,7 +3717,6 @@ pub trait Iterator { /// assert_eq!([1].iter().eq([1, 2].iter()), false); /// ``` #[stable(feature = "iter_order", since = "1.5.0")] - #[rustc_do_not_const_check] fn eq(self, other: I) -> bool where I: IntoIterator, @@ -3803,7 +3740,6 @@ pub trait Iterator { /// assert!(xs.iter().eq_by(&ys, |&x, &y| x * x == y)); /// ``` #[unstable(feature = "iter_order_by", issue = "64295")] - #[rustc_do_not_const_check] fn eq_by(self, other: I, eq: F) -> bool where Self: Sized, @@ -3836,7 +3772,6 @@ pub trait Iterator { /// assert_eq!([1].iter().ne([1, 2].iter()), true); /// ``` #[stable(feature = "iter_order", since = "1.5.0")] - #[rustc_do_not_const_check] fn ne(self, other: I) -> bool where I: IntoIterator, @@ -3858,7 +3793,6 @@ pub trait Iterator { /// assert_eq!([1, 2].iter().lt([1, 2].iter()), false); /// ``` #[stable(feature = "iter_order", since = "1.5.0")] - #[rustc_do_not_const_check] fn lt(self, other: I) -> bool where I: IntoIterator, @@ -3880,7 +3814,6 @@ pub trait Iterator { /// assert_eq!([1, 2].iter().le([1, 2].iter()), true); /// ``` #[stable(feature = "iter_order", since = "1.5.0")] - #[rustc_do_not_const_check] fn le(self, other: I) -> bool where I: IntoIterator, @@ -3902,7 +3835,6 @@ pub trait Iterator { /// assert_eq!([1, 2].iter().gt([1, 2].iter()), false); /// ``` #[stable(feature = "iter_order", since = "1.5.0")] - #[rustc_do_not_const_check] fn gt(self, other: I) -> bool where I: IntoIterator, @@ -3924,7 +3856,6 @@ pub trait Iterator { /// assert_eq!([1, 2].iter().ge([1, 2].iter()), true); /// ``` #[stable(feature = "iter_order", since = "1.5.0")] - #[rustc_do_not_const_check] fn ge(self, other: I) -> bool where I: IntoIterator, @@ -3954,7 +3885,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "is_sorted", since = "1.82.0")] - #[rustc_do_not_const_check] fn is_sorted(self) -> bool where Self: Sized, @@ -3981,7 +3911,6 @@ pub trait Iterator { /// assert!(std::iter::empty::().is_sorted_by(|a, b| true)); /// ``` #[stable(feature = "is_sorted", since = "1.82.0")] - #[rustc_do_not_const_check] fn is_sorted_by(mut self, compare: F) -> bool where Self: Sized, @@ -4026,7 +3955,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "is_sorted", since = "1.82.0")] - #[rustc_do_not_const_check] fn is_sorted_by_key(self, f: F) -> bool where Self: Sized, @@ -4042,7 +3970,6 @@ pub trait Iterator { #[inline] #[doc(hidden)] #[unstable(feature = "trusted_random_access", issue = "none")] - #[rustc_do_not_const_check] unsafe fn __iterator_get_unchecked(&mut self, _idx: usize) -> Self::Item where Self: TrustedRandomAccessNoCoerce, diff --git a/library/core/src/lib.rs b/library/core/src/lib.rs index 7873a2a44c679..2137ad785016e 100644 --- a/library/core/src/lib.rs +++ b/library/core/src/lib.rs @@ -107,6 +107,7 @@ // // Library features: // tidy-alphabetical-start +#![cfg_attr(bootstrap, feature(const_fmt_arguments_new))] #![feature(array_ptr_get)] #![feature(asm_experimental_arch)] #![feature(const_align_of_val)] @@ -114,45 +115,26 @@ #![feature(const_align_offset)] #![feature(const_alloc_layout)] #![feature(const_arguments_as_str)] -#![feature(const_array_from_ref)] #![feature(const_array_into_iter_constructors)] #![feature(const_bigint_helper_methods)] #![feature(const_black_box)] -#![feature(const_cell_into_inner)] +#![feature(const_char_encode_utf16)] #![feature(const_eval_select)] #![feature(const_exact_div)] -#![feature(const_float_bits_conv)] -#![feature(const_float_classify)] -#![feature(const_fmt_arguments_new)] +#![feature(const_float_methods)] #![feature(const_hash)] #![feature(const_heap)] -#![feature(const_index_range_slice_index)] -#![feature(const_intrinsic_copy)] -#![feature(const_intrinsic_forget)] -#![feature(const_ipv4)] -#![feature(const_ipv6)] -#![feature(const_likely)] -#![feature(const_maybe_uninit_as_mut_ptr)] -#![feature(const_maybe_uninit_assume_init)] #![feature(const_nonnull_new)] #![feature(const_num_midpoint)] -#![feature(const_option)] #![feature(const_option_ext)] -#![feature(const_pin)] +#![feature(const_pin_2)] #![feature(const_pointer_is_aligned)] -#![feature(const_ptr_as_ref)] #![feature(const_ptr_is_null)] #![feature(const_ptr_sub_ptr)] -#![feature(const_ptr_write)] #![feature(const_raw_ptr_comparison)] -#![feature(const_replace)] #![feature(const_size_of_val)] #![feature(const_size_of_val_raw)] -#![feature(const_slice_from_raw_parts_mut)] -#![feature(const_slice_from_ref)] -#![feature(const_slice_index)] -#![feature(const_slice_split_at_mut)] -#![feature(const_str_from_utf8_unchecked_mut)] +#![feature(const_sockaddr_setters)] #![feature(const_strict_overflow_ops)] #![feature(const_swap)] #![feature(const_try)] @@ -161,26 +143,26 @@ #![feature(const_typed_swap)] #![feature(const_ub_checks)] #![feature(const_unicode_case_lookup)] -#![feature(const_unsafecell_get_mut)] +#![feature(core_intrinsics)] #![feature(coverage_attribute)] #![feature(do_not_recommend)] -#![feature(duration_consts_float)] #![feature(internal_impls_macro)] #![feature(ip)] #![feature(is_ascii_octdigit)] #![feature(is_val_statically_known)] -#![feature(isqrt)] +#![feature(lazy_get)] #![feature(link_cfg)] +#![feature(non_null_from_ref)] #![feature(offset_of_enum)] #![feature(panic_internals)] #![feature(ptr_alignment_type)] #![feature(ptr_metadata)] #![feature(set_ptr_value)] +#![feature(slice_as_chunks)] #![feature(slice_ptr_get)] #![feature(str_internals)] #![feature(str_split_inclusive_remainder)] #![feature(str_split_remainder)] -#![feature(strict_provenance)] #![feature(ub_checks)] #![feature(unchecked_neg)] #![feature(unchecked_shifts)] @@ -191,6 +173,8 @@ // // Language features: // tidy-alphabetical-start +#![cfg_attr(bootstrap, feature(strict_provenance))] +#![cfg_attr(not(bootstrap), feature(strict_provenance_lints))] #![feature(abi_unadjusted)] #![feature(adt_const_params)] #![feature(allow_internal_unsafe)] @@ -201,9 +185,9 @@ #![feature(cfg_target_has_atomic_equal_alignment)] #![feature(cfg_ub_checks)] #![feature(const_for)] -#![feature(const_mut_refs)] +#![feature(const_is_char_boundary)] #![feature(const_precise_live_drops)] -#![feature(const_refs_to_cell)] +#![feature(const_str_split_at)] #![feature(decl_macro)] #![feature(deprecated_suggestion)] #![feature(doc_cfg)] @@ -288,6 +272,15 @@ pub mod assert_matches { pub use crate::macros::{assert_matches, debug_assert_matches}; } +// We don't export this through #[macro_export] for now, to avoid breakage. +#[cfg(not(bootstrap))] +#[unstable(feature = "autodiff", issue = "124509")] +/// Unstable module containing the unstable `autodiff` macro. +pub mod autodiff { + #[unstable(feature = "autodiff", issue = "124509")] + pub use crate::macros::builtin::autodiff; +} + #[unstable(feature = "cfg_match", issue = "115585")] pub use crate::macros::cfg_match; @@ -393,6 +386,8 @@ pub mod panicking; #[unstable(feature = "core_pattern_types", issue = "123646")] pub mod pat; pub mod pin; +#[unstable(feature = "random", issue = "130703")] +pub mod random; #[unstable(feature = "new_range_api", issue = "125687")] pub mod range; pub mod result; diff --git a/library/core/src/macros/mod.rs b/library/core/src/macros/mod.rs index 888832251f6da..771c2d31b60e0 100644 --- a/library/core/src/macros/mod.rs +++ b/library/core/src/macros/mod.rs @@ -229,8 +229,8 @@ pub macro assert_matches { pub macro cfg_match { // with a final wildcard ( - $(cfg($initial_meta:meta) => { $($initial_tokens:item)* })+ - _ => { $($extra_tokens:item)* } + $(cfg($initial_meta:meta) => { $($initial_tokens:tt)* })+ + _ => { $($extra_tokens:tt)* } ) => { cfg_match! { @__items (); @@ -241,7 +241,7 @@ pub macro cfg_match { // without a final wildcard ( - $(cfg($extra_meta:meta) => { $($extra_tokens:item)* })* + $(cfg($extra_meta:meta) => { $($extra_tokens:tt)* })* ) => { cfg_match! { @__items (); @@ -256,7 +256,7 @@ pub macro cfg_match { (@__items ($($_:meta,)*);) => {}, ( @__items ($($no:meta,)*); - (($($yes:meta)?) ($($tokens:item)*)), + (($($yes:meta)?) ($($tokens:tt)*)), $($rest:tt,)* ) => { // Emit all items within one block, applying an appropriate #[cfg]. The @@ -279,7 +279,7 @@ pub macro cfg_match { // Internal macro to make __apply work out right for different match types, // because of how macros match/expand stuff. - (@__identity $($tokens:item)*) => { + (@__identity $($tokens:tt)*) => { $($tokens)* } } @@ -1107,17 +1107,19 @@ pub(crate) mod builtin { /// /// If the named environment variable is present at compile time, this will /// expand into an expression of type `Option<&'static str>` whose value is - /// `Some` of the value of the environment variable. If the environment - /// variable is not present, then this will expand to `None`. See - /// [`Option`][Option] for more information on this type. Use - /// [`std::env::var`] instead if you want to read the value at runtime. + /// `Some` of the value of the environment variable (a compilation error + /// will be emitted if the environment variable is not a valid Unicode + /// string). If the environment variable is not present, then this will + /// expand to `None`. See [`Option`][Option] for more information on this + /// type. Use [`std::env::var`] instead if you want to read the value at + /// runtime. /// /// [`std::env::var`]: ../std/env/fn.var.html /// - /// A compile time error is never emitted when using this macro regardless - /// of whether the environment variable is present or not. - /// To emit a compile error if the environment variable is not present, - /// use the [`env!`] macro instead. + /// A compile time error is only emitted when using this macro if the + /// environment variable exists and is not a valid Unicode string. To also + /// emit a compile error if the environment variable is not present, use the + /// [`env!`] macro instead. /// /// # Examples /// @@ -1539,6 +1541,24 @@ pub(crate) mod builtin { ($file:expr $(,)?) => {{ /* compiler built-in */ }}; } + /// Automatic Differentiation macro which allows generating a new function to compute + /// the derivative of a given function. It may only be applied to a function. + /// The expected usage syntax is + /// `#[autodiff(NAME, MODE, INPUT_ACTIVITIES, OUTPUT_ACTIVITY)]` + /// where: + /// NAME is a string that represents a valid function name. + /// MODE is any of Forward, Reverse, ForwardFirst, ReverseFirst. + /// INPUT_ACTIVITIES consists of one valid activity for each input parameter. + /// OUTPUT_ACTIVITY must not be set if we implicitely return nothing (or explicitely return + /// `-> ()`). Otherwise it must be set to one of the allowed activities. + #[unstable(feature = "autodiff", issue = "124509")] + #[allow_internal_unstable(rustc_attrs)] + #[rustc_builtin_macro] + #[cfg(not(bootstrap))] + pub macro autodiff($item:item) { + /* compiler built-in */ + } + /// Asserts that a boolean expression is `true` at runtime. /// /// This will invoke the [`panic!`] macro if the provided expression cannot be diff --git a/library/core/src/marker.rs b/library/core/src/marker.rs index fd41b80cdbd0a..1c5c58d64a2b2 100644 --- a/library/core/src/marker.rs +++ b/library/core/src/marker.rs @@ -158,7 +158,7 @@ pub trait Sized { /// - Arrays `[T; N]` implement `Unsize<[T]>`. /// - A type implements `Unsize` if all of these conditions are met: /// - The type implements `Trait`. -/// - `Trait` is object safe. +/// - `Trait` is dyn-compatible[^1]. /// - The type is sized. /// - The type outlives `'a`. /// - Structs `Foo<..., T1, ..., Tn, ...>` implement `Unsize>` @@ -178,6 +178,7 @@ pub trait Sized { /// [`Rc`]: ../../std/rc/struct.Rc.html /// [RFC982]: https://github.com/rust-lang/rfcs/blob/master/text/0982-dst-coercion.md /// [nomicon-coerce]: ../../nomicon/coercions.html +/// [^1]: Formerly known as *object safe*. #[unstable(feature = "unsize", issue = "18598")] #[lang = "unsize"] #[rustc_deny_explicit_impl(implement_via_object = false)] @@ -1062,53 +1063,10 @@ pub trait FnPtr: Copy + Clone { } /// Derive macro generating impls of traits related to smart pointers. -#[rustc_builtin_macro(SmartPointer, attributes(pointee))] +#[rustc_builtin_macro(CoercePointee, attributes(pointee))] #[allow_internal_unstable(dispatch_from_dyn, coerce_unsized, unsize)] -#[unstable(feature = "derive_smart_pointer", issue = "123430")] -pub macro SmartPointer($item:item) { +#[unstable(feature = "derive_coerce_pointee", issue = "123430")] +#[cfg(not(bootstrap))] +pub macro CoercePointee($item:item) { /* compiler built-in */ } - -// Support traits and types for the desugaring of const traits and -// `~const` bounds. Not supposed to be used by anything other than -// the compiler. -#[doc(hidden)] -#[unstable( - feature = "effect_types", - issue = "none", - reason = "internal module for implementing effects" -)] -#[allow(missing_debug_implementations)] // these unit structs don't need `Debug` impls. -pub mod effects { - #[lang = "EffectsNoRuntime"] - pub struct NoRuntime; - #[lang = "EffectsMaybe"] - pub struct Maybe; - #[lang = "EffectsRuntime"] - pub struct Runtime; - - #[lang = "EffectsCompat"] - pub trait Compat<#[rustc_runtime] const RUNTIME: bool> {} - - impl Compat for NoRuntime {} - impl Compat for Runtime {} - impl<#[rustc_runtime] const RUNTIME: bool> Compat for Maybe {} - - #[lang = "EffectsTyCompat"] - #[marker] - pub trait TyCompat {} - - impl TyCompat for T {} - impl TyCompat for T {} - - #[lang = "EffectsIntersection"] - pub trait Intersection { - #[lang = "EffectsIntersectionOutput"] - type Output: ?Sized; - } - - // FIXME(effects): remove this after next trait solver lands - impl Intersection for () { - type Output = Maybe; - } -} diff --git a/library/core/src/mem/manually_drop.rs b/library/core/src/mem/manually_drop.rs index 3e47785ee488e..7d519384e373c 100644 --- a/library/core/src/mem/manually_drop.rs +++ b/library/core/src/mem/manually_drop.rs @@ -1,22 +1,21 @@ use crate::ops::{Deref, DerefMut, DerefPure}; use crate::ptr; -/// A wrapper to inhibit the compiler from automatically calling `T`’s destructor. -/// This wrapper is 0-cost. +/// A wrapper to inhibit the compiler from automatically calling `T`’s +/// destructor. This wrapper is 0-cost. /// /// `ManuallyDrop` is guaranteed to have the same layout and bit validity as -/// `T`, and is subject to the same layout optimizations as `T`. As a consequence, -/// it has *no effect* on the assumptions that the compiler makes about its -/// contents. For example, initializing a `ManuallyDrop<&mut T>` with [`mem::zeroed`] -/// is undefined behavior. If you need to handle uninitialized data, use -/// [`MaybeUninit`] instead. +/// `T`, and is subject to the same layout optimizations as `T`. As a +/// consequence, it has *no effect* on the assumptions that the compiler makes +/// about its contents. For example, initializing a `ManuallyDrop<&mut T>` with +/// [`mem::zeroed`] is undefined behavior. If you need to handle uninitialized +/// data, use [`MaybeUninit`] instead. /// -/// Note that accessing the value inside a `ManuallyDrop` is safe. -/// This means that a `ManuallyDrop` whose content has been dropped must not -/// be exposed through a public safe API. -/// Correspondingly, `ManuallyDrop::drop` is unsafe. +/// Note that accessing the value inside a `ManuallyDrop` is safe. This means +/// that a `ManuallyDrop` whose content has been dropped must not be exposed +/// through a public safe API. Correspondingly, `ManuallyDrop::drop` is unsafe. /// -/// # `ManuallyDrop` and drop order. +/// # `ManuallyDrop` and drop order /// /// Rust has a well-defined [drop order] of values. To make sure that fields or /// locals are dropped in a specific order, reorder the declarations such that @@ -40,9 +39,116 @@ use crate::ptr; /// } /// ``` /// +/// # Interaction with `Box` +/// +/// Currently, if you have a `ManuallyDrop`, where the type `T` is a `Box` or +/// contains a `Box` inside, then dropping the `T` followed by moving the +/// `ManuallyDrop` is [considered to be undefined +/// behavior](https://github.com/rust-lang/unsafe-code-guidelines/issues/245). +/// That is, the following code causes undefined behavior: +/// +/// ```no_run +/// use std::mem::ManuallyDrop; +/// +/// let mut x = ManuallyDrop::new(Box::new(42)); +/// unsafe { +/// ManuallyDrop::drop(&mut x); +/// } +/// let y = x; // Undefined behavior! +/// ``` +/// +/// This is [likely to change in the +/// future](https://rust-lang.github.io/rfcs/3336-maybe-dangling.html). In the +/// meantime, consider using [`MaybeUninit`] instead. +/// +/// # Safety hazards when storing `ManuallyDrop` in a struct or an enum. +/// +/// Special care is needed when all of the conditions below are met: +/// * A struct or enum contains a `ManuallyDrop`. +/// * The `ManuallyDrop` is not inside a `union`. +/// * The struct or enum is part of public API, or is stored in a struct or an +/// enum that is part of public API. +/// * There is code that drops the contents of the `ManuallyDrop` field, and +/// this code is outside the struct or enum's `Drop` implementation. +/// +/// In particular, the following hazards may occur: +/// +/// #### Storing generic types +/// +/// If the `ManuallyDrop` contains a client-supplied generic type, the client +/// might provide a `Box` as that type. This would cause undefined behavior when +/// the struct or enum is later moved, as mentioned in the previous section. For +/// example, the following code causes undefined behavior: +/// +/// ```no_run +/// use std::mem::ManuallyDrop; +/// +/// pub struct BadOption { +/// // Invariant: Has been dropped iff `is_some` is false. +/// value: ManuallyDrop, +/// is_some: bool, +/// } +/// impl BadOption { +/// pub fn new(value: T) -> Self { +/// Self { value: ManuallyDrop::new(value), is_some: true } +/// } +/// pub fn change_to_none(&mut self) { +/// if self.is_some { +/// self.is_some = false; +/// unsafe { +/// // SAFETY: `value` hasn't been dropped yet, as per the invariant +/// // (This is actually unsound!) +/// ManuallyDrop::drop(&mut self.value); +/// } +/// } +/// } +/// } +/// +/// // In another crate: +/// +/// let mut option = BadOption::new(Box::new(42)); +/// option.change_to_none(); +/// let option2 = option; // Undefined behavior! +/// ``` +/// +/// #### Deriving traits +/// +/// Deriving `Debug`, `Clone`, `PartialEq`, `PartialOrd`, `Ord`, or `Hash` on +/// the struct or enum could be unsound, since the derived implementations of +/// these traits would access the `ManuallyDrop` field. For example, the +/// following code causes undefined behavior: +/// +/// ```no_run +/// use std::mem::ManuallyDrop; +/// +/// // This derive is unsound in combination with the `ManuallyDrop::drop` call. +/// #[derive(Debug)] +/// pub struct Foo { +/// value: ManuallyDrop, +/// } +/// impl Foo { +/// pub fn new() -> Self { +/// let mut temp = Self { +/// value: ManuallyDrop::new(String::from("Unsafe rust is hard.")) +/// }; +/// unsafe { +/// // SAFETY: `value` hasn't been dropped yet. +/// ManuallyDrop::drop(&mut temp.value); +/// } +/// temp +/// } +/// } +/// +/// // In another crate: +/// +/// let foo = Foo::new(); +/// println!("{:?}", foo); // Undefined behavior! +/// ``` +/// /// [drop order]: https://doc.rust-lang.org/reference/destructors.html /// [`mem::zeroed`]: crate::mem::zeroed /// [`MaybeUninit`]: crate::mem::MaybeUninit +/// [`MaybeUninit`]: crate::mem::MaybeUninit #[stable(feature = "manually_drop", since = "1.20.0")] #[lang = "manually_drop"] #[derive(Copy, Clone, Debug, Default, PartialEq, Eq, PartialOrd, Ord, Hash)] diff --git a/library/core/src/mem/maybe_uninit.rs b/library/core/src/mem/maybe_uninit.rs index 4be2e5ef1eade..a57e265c7cc00 100644 --- a/library/core/src/mem/maybe_uninit.rs +++ b/library/core/src/mem/maybe_uninit.rs @@ -351,6 +351,9 @@ impl MaybeUninit { /// but `MaybeUninit<&'static i32>::zeroed()` is not because references must not /// be null. /// + /// Note that if `T` has padding bytes, those bytes are *not* preserved when the + /// `MaybeUninit` value is returned from this function, so those bytes will *not* be zeroed. + /// /// Note that dropping a `MaybeUninit` will never call `T`'s drop code. /// It is your responsibility to make sure `T` gets dropped if it got initialized. /// @@ -387,10 +390,6 @@ impl MaybeUninit { #[must_use] #[rustc_diagnostic_item = "maybe_uninit_zeroed"] #[stable(feature = "maybe_uninit", since = "1.36.0")] - // These are OK to allow since we do not leak &mut to user-visible API - #[rustc_allow_const_fn_unstable(const_mut_refs)] - #[rustc_allow_const_fn_unstable(const_ptr_write)] - #[rustc_allow_const_fn_unstable(const_maybe_uninit_as_mut_ptr)] #[rustc_const_stable(feature = "const_maybe_uninit_zeroed", since = "1.75.0")] pub const fn zeroed() -> MaybeUninit { let mut u = MaybeUninit::::uninit(); @@ -567,7 +566,7 @@ impl MaybeUninit { /// (Notice that the rules around references to uninitialized data are not finalized yet, but /// until they are, it is advisable to avoid them.) #[stable(feature = "maybe_uninit", since = "1.36.0")] - #[rustc_const_unstable(feature = "const_maybe_uninit_as_mut_ptr", issue = "75251")] + #[rustc_const_stable(feature = "const_maybe_uninit_as_mut_ptr", since = "1.83.0")] #[inline(always)] pub const fn as_mut_ptr(&mut self) -> *mut T { // `MaybeUninit` and `ManuallyDrop` are both `repr(transparent)` so we can cast the pointer. @@ -721,7 +720,7 @@ impl MaybeUninit { /// this does not constitute a stable guarantee), because the only /// requirement the compiler knows about it is that the data pointer must be /// non-null. Dropping such a `Vec` however will cause undefined - /// behaviour. + /// behavior. /// /// [`assume_init`]: MaybeUninit::assume_init /// [`Vec`]: ../../std/vec/struct.Vec.html @@ -907,7 +906,10 @@ impl MaybeUninit { /// }; /// ``` #[stable(feature = "maybe_uninit_ref", since = "1.55.0")] - #[rustc_const_unstable(feature = "const_maybe_uninit_assume_init", issue = "none")] + #[rustc_const_stable( + feature = "const_maybe_uninit_assume_init", + since = "CURRENT_RUSTC_VERSION" + )] #[inline(always)] pub const unsafe fn assume_init_mut(&mut self) -> &mut T { // SAFETY: the caller must guarantee that `self` is initialized. @@ -993,7 +995,7 @@ impl MaybeUninit { /// /// [`assume_init_mut`]: MaybeUninit::assume_init_mut #[unstable(feature = "maybe_uninit_slice", issue = "63569")] - #[rustc_const_unstable(feature = "const_maybe_uninit_assume_init", issue = "none")] + #[rustc_const_unstable(feature = "maybe_uninit_slice", issue = "63569")] #[inline(always)] pub const unsafe fn slice_assume_init_mut(slice: &mut [Self]) -> &mut [T] { // SAFETY: similar to safety notes for `slice_get_ref`, but we have a diff --git a/library/core/src/mem/mod.rs b/library/core/src/mem/mod.rs index 2abc99ddeecaa..28b2b406c9bcc 100644 --- a/library/core/src/mem/mod.rs +++ b/library/core/src/mem/mod.rs @@ -862,7 +862,7 @@ pub fn take(dest: &mut T) -> T { #[inline] #[stable(feature = "rust1", since = "1.0.0")] #[must_use = "if you don't need the old value, you can just assign the new value directly"] -#[rustc_const_unstable(feature = "const_replace", issue = "83164")] +#[rustc_const_stable(feature = "const_replace", since = "1.83.0")] #[cfg_attr(not(test), rustc_diagnostic_item = "mem_replace")] pub const fn replace(dest: &mut T, src: T) -> T { // It may be tempting to use `swap` to avoid `unsafe` here. Don't! @@ -1259,11 +1259,9 @@ impl SizedTypeProperties for T {} /// /// Nested field accesses may be used, but not array indexes. /// -/// Enum variants may be traversed as if they were fields. Variants themselves do -/// not have an offset. -/// -/// However, on stable only a single field name is supported, which blocks the use of -/// enum support. +/// If the nightly-only feature `offset_of_enum` is enabled, +/// variants may be traversed as if they were fields. +/// Variants themselves do not have an offset. /// /// Visibility is respected - all types and fields must be visible to the call site: /// diff --git a/library/core/src/net/ip_addr.rs b/library/core/src/net/ip_addr.rs index 919f681f911f9..0d1f4a9ea3eed 100644 --- a/library/core/src/net/ip_addr.rs +++ b/library/core/src/net/ip_addr.rs @@ -295,7 +295,6 @@ impl IpAddr { /// assert_eq!(IpAddr::V4(Ipv4Addr::new(80, 9, 12, 3)).is_global(), true); /// assert_eq!(IpAddr::V6(Ipv6Addr::new(0, 0, 0x1c9, 0, 0, 0xafc8, 0, 0x1)).is_global(), true); /// ``` - #[rustc_const_unstable(feature = "const_ip", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -348,7 +347,6 @@ impl IpAddr { /// true /// ); /// ``` - #[rustc_const_unstable(feature = "const_ip", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -375,6 +373,7 @@ impl IpAddr { /// assert_eq!(IpAddr::V6(Ipv6Addr::new(0x2001, 0x2, 0, 0, 0, 0, 0, 0)).is_benchmarking(), true); /// ``` #[unstable(feature = "ip", issue = "27709")] + #[rustc_const_unstable(feature = "const_ipv4", issue = "76205")] #[must_use] #[inline] pub const fn is_benchmarking(&self) -> bool { @@ -600,6 +599,24 @@ impl Ipv4Addr { self.octets } + /// Creates an `Ipv4Addr` from a four element byte array. + /// + /// # Examples + /// + /// ``` + /// #![feature(ip_from)] + /// use std::net::Ipv4Addr; + /// + /// let addr = Ipv4Addr::from_octets([13u8, 12u8, 11u8, 10u8]); + /// assert_eq!(Ipv4Addr::new(13, 12, 11, 10), addr); + /// ``` + #[unstable(feature = "ip_from", issue = "131360")] + #[must_use] + #[inline] + pub const fn from_octets(octets: [u8; 4]) -> Ipv4Addr { + Ipv4Addr { octets } + } + /// Returns [`true`] for the special 'unspecified' address (`0.0.0.0`). /// /// This property is defined in _UNIX Network Programming, Second Edition_, @@ -776,7 +793,6 @@ impl Ipv4Addr { /// /// // For a complete overview see the IANA IPv4 Special-Purpose Address Registry. /// ``` - #[rustc_const_unstable(feature = "const_ipv4", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -813,7 +829,6 @@ impl Ipv4Addr { /// assert_eq!(Ipv4Addr::new(100, 127, 255, 255).is_shared(), true); /// assert_eq!(Ipv4Addr::new(100, 128, 0, 0).is_shared(), false); /// ``` - #[rustc_const_unstable(feature = "const_ipv4", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -841,7 +856,6 @@ impl Ipv4Addr { /// assert_eq!(Ipv4Addr::new(198, 19, 255, 255).is_benchmarking(), true); /// assert_eq!(Ipv4Addr::new(198, 20, 0, 0).is_benchmarking(), false); /// ``` - #[rustc_const_unstable(feature = "const_ipv4", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -878,7 +892,6 @@ impl Ipv4Addr { /// // The broadcast address is not considered as reserved for future use by this implementation /// assert_eq!(Ipv4Addr::new(255, 255, 255, 255).is_reserved(), false); /// ``` - #[rustc_const_unstable(feature = "const_ipv4", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -1400,6 +1413,34 @@ impl Ipv6Addr { ] } + /// Creates an `Ipv6Addr` from an eight element 16-bit array. + /// + /// # Examples + /// + /// ``` + /// #![feature(ip_from)] + /// use std::net::Ipv6Addr; + /// + /// let addr = Ipv6Addr::from_segments([ + /// 0x20du16, 0x20cu16, 0x20bu16, 0x20au16, + /// 0x209u16, 0x208u16, 0x207u16, 0x206u16, + /// ]); + /// assert_eq!( + /// Ipv6Addr::new( + /// 0x20d, 0x20c, 0x20b, 0x20a, + /// 0x209, 0x208, 0x207, 0x206, + /// ), + /// addr + /// ); + /// ``` + #[unstable(feature = "ip_from", issue = "131360")] + #[must_use] + #[inline] + pub const fn from_segments(segments: [u16; 8]) -> Ipv6Addr { + let [a, b, c, d, e, f, g, h] = segments; + Ipv6Addr::new(a, b, c, d, e, f, g, h) + } + /// Returns [`true`] for the special 'unspecified' address (`::`). /// /// This property is defined in [IETF RFC 4291]. @@ -1510,7 +1551,6 @@ impl Ipv6Addr { /// /// // For a complete overview see the IANA IPv6 Special-Purpose Address Registry. /// ``` - #[rustc_const_unstable(feature = "const_ipv6", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -1562,7 +1602,6 @@ impl Ipv6Addr { /// assert_eq!(Ipv6Addr::new(0, 0, 0, 0, 0, 0xffff, 0xc00a, 0x2ff).is_unique_local(), false); /// assert_eq!(Ipv6Addr::new(0xfc02, 0, 0, 0, 0, 0, 0, 0).is_unique_local(), true); /// ``` - #[rustc_const_unstable(feature = "const_ipv6", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -1591,7 +1630,6 @@ impl Ipv6Addr { /// assert_eq!(Ipv6Addr::new(0x2001, 0xdb8, 0, 0, 0, 0, 0, 0).is_unicast(), true); /// assert_eq!(Ipv6Addr::new(0xff00, 0, 0, 0, 0, 0, 0, 0).is_unicast(), false); /// ``` - #[rustc_const_unstable(feature = "const_ipv6", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -1643,7 +1681,6 @@ impl Ipv6Addr { /// assert_eq!(Ipv6Addr::new(0xfe80, 0, 0, 1, 0, 0, 0, 0).is_unicast_link_local(), true); /// assert_eq!(Ipv6Addr::new(0xfe81, 0, 0, 0, 0, 0, 0, 0).is_unicast_link_local(), true); /// ``` - #[rustc_const_unstable(feature = "const_ipv6", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -1668,7 +1705,6 @@ impl Ipv6Addr { /// assert_eq!(Ipv6Addr::new(0, 0, 0, 0, 0, 0xffff, 0xc00a, 0x2ff).is_documentation(), false); /// assert_eq!(Ipv6Addr::new(0x2001, 0xdb8, 0, 0, 0, 0, 0, 0).is_documentation(), true); /// ``` - #[rustc_const_unstable(feature = "const_ipv6", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -1729,7 +1765,6 @@ impl Ipv6Addr { /// assert_eq!(Ipv6Addr::new(0x2001, 0xdb8, 0, 0, 0, 0, 0, 0).is_unicast_global(), false); /// assert_eq!(Ipv6Addr::new(0, 0, 0, 0, 0, 0xffff, 0xc00a, 0x2ff).is_unicast_global(), true); /// ``` - #[rustc_const_unstable(feature = "const_ipv6", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -1758,7 +1793,6 @@ impl Ipv6Addr { /// ); /// assert_eq!(Ipv6Addr::new(0, 0, 0, 0, 0, 0xffff, 0xc00a, 0x2ff).multicast_scope(), None); /// ``` - #[rustc_const_unstable(feature = "const_ipv6", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -1818,7 +1852,6 @@ impl Ipv6Addr { /// /// assert_eq!(Ipv6Addr::new(0x2001, 0xdb8, 0, 0, 0, 0, 0, 0).is_ipv4_mapped(), false); /// ``` - #[rustc_const_unstable(feature = "const_ipv6", issue = "76205")] #[unstable(feature = "ip", issue = "27709")] #[must_use] #[inline] @@ -1932,7 +1965,7 @@ impl Ipv6Addr { /// use std::net::Ipv6Addr; /// /// assert_eq!(Ipv6Addr::new(0xff00, 0, 0, 0, 0, 0, 0, 0).octets(), - /// [255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]); + /// [0xff, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]); /// ``` #[rustc_const_stable(feature = "const_ip_32", since = "1.32.0")] #[stable(feature = "ipv6_to_octets", since = "1.12.0")] @@ -1941,6 +1974,33 @@ impl Ipv6Addr { pub const fn octets(&self) -> [u8; 16] { self.octets } + + /// Creates an `Ipv6Addr` from a sixteen element byte array. + /// + /// # Examples + /// + /// ``` + /// #![feature(ip_from)] + /// use std::net::Ipv6Addr; + /// + /// let addr = Ipv6Addr::from_octets([ + /// 0x19u8, 0x18u8, 0x17u8, 0x16u8, 0x15u8, 0x14u8, 0x13u8, 0x12u8, + /// 0x11u8, 0x10u8, 0x0fu8, 0x0eu8, 0x0du8, 0x0cu8, 0x0bu8, 0x0au8, + /// ]); + /// assert_eq!( + /// Ipv6Addr::new( + /// 0x1918, 0x1716, 0x1514, 0x1312, + /// 0x1110, 0x0f0e, 0x0d0c, 0x0b0a, + /// ), + /// addr + /// ); + /// ``` + #[unstable(feature = "ip_from", issue = "131360")] + #[must_use] + #[inline] + pub const fn from_octets(octets: [u8; 16]) -> Ipv6Addr { + Ipv6Addr { octets } + } } /// Writes an Ipv6Addr, conforming to the canonical style described by @@ -2113,15 +2173,13 @@ impl From<[u8; 16]> for Ipv6Addr { /// use std::net::Ipv6Addr; /// /// let addr = Ipv6Addr::from([ - /// 25u8, 24u8, 23u8, 22u8, 21u8, 20u8, 19u8, 18u8, - /// 17u8, 16u8, 15u8, 14u8, 13u8, 12u8, 11u8, 10u8, + /// 0x19u8, 0x18u8, 0x17u8, 0x16u8, 0x15u8, 0x14u8, 0x13u8, 0x12u8, + /// 0x11u8, 0x10u8, 0x0fu8, 0x0eu8, 0x0du8, 0x0cu8, 0x0bu8, 0x0au8, /// ]); /// assert_eq!( /// Ipv6Addr::new( - /// 0x1918, 0x1716, - /// 0x1514, 0x1312, - /// 0x1110, 0x0f0e, - /// 0x0d0c, 0x0b0a + /// 0x1918, 0x1716, 0x1514, 0x1312, + /// 0x1110, 0x0f0e, 0x0d0c, 0x0b0a, /// ), /// addr /// ); @@ -2142,15 +2200,13 @@ impl From<[u16; 8]> for Ipv6Addr { /// use std::net::Ipv6Addr; /// /// let addr = Ipv6Addr::from([ - /// 525u16, 524u16, 523u16, 522u16, - /// 521u16, 520u16, 519u16, 518u16, + /// 0x20du16, 0x20cu16, 0x20bu16, 0x20au16, + /// 0x209u16, 0x208u16, 0x207u16, 0x206u16, /// ]); /// assert_eq!( /// Ipv6Addr::new( - /// 0x20d, 0x20c, - /// 0x20b, 0x20a, - /// 0x209, 0x208, - /// 0x207, 0x206 + /// 0x20d, 0x20c, 0x20b, 0x20a, + /// 0x209, 0x208, 0x207, 0x206, /// ), /// addr /// ); @@ -2172,15 +2228,13 @@ impl From<[u8; 16]> for IpAddr { /// use std::net::{IpAddr, Ipv6Addr}; /// /// let addr = IpAddr::from([ - /// 25u8, 24u8, 23u8, 22u8, 21u8, 20u8, 19u8, 18u8, - /// 17u8, 16u8, 15u8, 14u8, 13u8, 12u8, 11u8, 10u8, + /// 0x19u8, 0x18u8, 0x17u8, 0x16u8, 0x15u8, 0x14u8, 0x13u8, 0x12u8, + /// 0x11u8, 0x10u8, 0x0fu8, 0x0eu8, 0x0du8, 0x0cu8, 0x0bu8, 0x0au8, /// ]); /// assert_eq!( /// IpAddr::V6(Ipv6Addr::new( - /// 0x1918, 0x1716, - /// 0x1514, 0x1312, - /// 0x1110, 0x0f0e, - /// 0x0d0c, 0x0b0a + /// 0x1918, 0x1716, 0x1514, 0x1312, + /// 0x1110, 0x0f0e, 0x0d0c, 0x0b0a, /// )), /// addr /// ); @@ -2201,15 +2255,13 @@ impl From<[u16; 8]> for IpAddr { /// use std::net::{IpAddr, Ipv6Addr}; /// /// let addr = IpAddr::from([ - /// 525u16, 524u16, 523u16, 522u16, - /// 521u16, 520u16, 519u16, 518u16, + /// 0x20du16, 0x20cu16, 0x20bu16, 0x20au16, + /// 0x209u16, 0x208u16, 0x207u16, 0x206u16, /// ]); /// assert_eq!( /// IpAddr::V6(Ipv6Addr::new( - /// 0x20d, 0x20c, - /// 0x20b, 0x20a, - /// 0x209, 0x208, - /// 0x207, 0x206 + /// 0x20d, 0x20c, 0x20b, 0x20a, + /// 0x209, 0x208, 0x207, 0x206, /// )), /// addr /// ); diff --git a/library/core/src/net/socket_addr.rs b/library/core/src/net/socket_addr.rs index 4e339172b682f..9204797e6e157 100644 --- a/library/core/src/net/socket_addr.rs +++ b/library/core/src/net/socket_addr.rs @@ -49,6 +49,15 @@ pub enum SocketAddr { /// [IETF RFC 793]: https://tools.ietf.org/html/rfc793 /// [`IPv4` address]: Ipv4Addr /// +/// # Textual representation +/// +/// `SocketAddrV4` provides a [`FromStr`](crate::str::FromStr) implementation. +/// It accepts an IPv4 address in its [textual representation], followed by a +/// single `:`, followed by the port encoded as a decimal integer. Other +/// formats are not accepted. +/// +/// [textual representation]: Ipv4Addr#textual-representation +/// /// # Examples /// /// ``` @@ -82,6 +91,32 @@ pub struct SocketAddrV4 { /// [IETF RFC 2553, Section 3.3]: https://tools.ietf.org/html/rfc2553#section-3.3 /// [`IPv6` address]: Ipv6Addr /// +/// # Textual representation +/// +/// `SocketAddrV6` provides a [`FromStr`](crate::str::FromStr) implementation, +/// based on the bracketed format recommended by [IETF RFC 5952], +/// with scope identifiers based on those specified in [IETF RFC 4007]. +/// +/// It accepts addresses consisting of the following elements, in order: +/// - A left square bracket (`[`) +/// - The [textual representation] of an IPv6 address +/// - _Optionally_, a percent sign (`%`) followed by the scope identifier +/// encoded as a decimal integer +/// - A right square bracket (`]`) +/// - A colon (`:`) +/// - The port, encoded as a decimal integer. +/// +/// For example, the string `[2001:db8::413]:443` represents a `SocketAddrV6` +/// with the address `2001:db8::413` and port `443`. The string +/// `[2001:db8::413%612]:443` represents the same address and port, with a +/// scope identifier of `612`. +/// +/// Other formats are not accepted. +/// +/// [IETF RFC 5952]: https://tools.ietf.org/html/rfc5952#section-6 +/// [IETF RFC 4007]: https://tools.ietf.org/html/rfc4007#section-11 +/// [textual representation]: Ipv6Addr#textual-representation +/// /// # Examples /// /// ``` @@ -92,6 +127,10 @@ pub struct SocketAddrV4 { /// assert_eq!("[2001:db8::1]:8080".parse(), Ok(socket)); /// assert_eq!(socket.ip(), &Ipv6Addr::new(0x2001, 0xdb8, 0, 0, 0, 0, 0, 1)); /// assert_eq!(socket.port(), 8080); +/// +/// let mut with_scope = socket.clone(); +/// with_scope.set_scope_id(3); +/// assert_eq!("[2001:db8::1%3]:8080".parse(), Ok(with_scope)); /// ``` #[derive(Copy, Clone, Eq, PartialEq, Ord, PartialOrd, Hash)] #[stable(feature = "rust1", since = "1.0.0")] @@ -159,9 +198,10 @@ impl SocketAddr { /// socket.set_ip(IpAddr::V4(Ipv4Addr::new(10, 10, 0, 1))); /// assert_eq!(socket.ip(), IpAddr::V4(Ipv4Addr::new(10, 10, 0, 1))); /// ``` - #[stable(feature = "sockaddr_setters", since = "1.9.0")] #[inline] - pub fn set_ip(&mut self, new_ip: IpAddr) { + #[stable(feature = "sockaddr_setters", since = "1.9.0")] + #[rustc_const_unstable(feature = "const_sockaddr_setters", issue = "131714")] + pub const fn set_ip(&mut self, new_ip: IpAddr) { // `match (*self, new_ip)` would have us mutate a copy of self only to throw it away. match (self, new_ip) { (&mut SocketAddr::V4(ref mut a), IpAddr::V4(new_ip)) => a.set_ip(new_ip), @@ -202,9 +242,10 @@ impl SocketAddr { /// socket.set_port(1025); /// assert_eq!(socket.port(), 1025); /// ``` - #[stable(feature = "sockaddr_setters", since = "1.9.0")] #[inline] - pub fn set_port(&mut self, new_port: u16) { + #[stable(feature = "sockaddr_setters", since = "1.9.0")] + #[rustc_const_unstable(feature = "const_sockaddr_setters", issue = "131714")] + pub const fn set_port(&mut self, new_port: u16) { match *self { SocketAddr::V4(ref mut a) => a.set_port(new_port), SocketAddr::V6(ref mut a) => a.set_port(new_port), @@ -307,9 +348,10 @@ impl SocketAddrV4 { /// socket.set_ip(Ipv4Addr::new(192, 168, 0, 1)); /// assert_eq!(socket.ip(), &Ipv4Addr::new(192, 168, 0, 1)); /// ``` - #[stable(feature = "sockaddr_setters", since = "1.9.0")] #[inline] - pub fn set_ip(&mut self, new_ip: Ipv4Addr) { + #[stable(feature = "sockaddr_setters", since = "1.9.0")] + #[rustc_const_unstable(feature = "const_sockaddr_setters", issue = "131714")] + pub const fn set_ip(&mut self, new_ip: Ipv4Addr) { self.ip = new_ip; } @@ -342,9 +384,10 @@ impl SocketAddrV4 { /// socket.set_port(4242); /// assert_eq!(socket.port(), 4242); /// ``` - #[stable(feature = "sockaddr_setters", since = "1.9.0")] #[inline] - pub fn set_port(&mut self, new_port: u16) { + #[stable(feature = "sockaddr_setters", since = "1.9.0")] + #[rustc_const_unstable(feature = "const_sockaddr_setters", issue = "131714")] + pub const fn set_port(&mut self, new_port: u16) { self.port = new_port; } } @@ -403,9 +446,10 @@ impl SocketAddrV6 { /// socket.set_ip(Ipv6Addr::new(76, 45, 0, 0, 0, 0, 0, 0)); /// assert_eq!(socket.ip(), &Ipv6Addr::new(76, 45, 0, 0, 0, 0, 0, 0)); /// ``` - #[stable(feature = "sockaddr_setters", since = "1.9.0")] #[inline] - pub fn set_ip(&mut self, new_ip: Ipv6Addr) { + #[stable(feature = "sockaddr_setters", since = "1.9.0")] + #[rustc_const_unstable(feature = "const_sockaddr_setters", issue = "131714")] + pub const fn set_ip(&mut self, new_ip: Ipv6Addr) { self.ip = new_ip; } @@ -438,9 +482,10 @@ impl SocketAddrV6 { /// socket.set_port(4242); /// assert_eq!(socket.port(), 4242); /// ``` - #[stable(feature = "sockaddr_setters", since = "1.9.0")] #[inline] - pub fn set_port(&mut self, new_port: u16) { + #[stable(feature = "sockaddr_setters", since = "1.9.0")] + #[rustc_const_unstable(feature = "const_sockaddr_setters", issue = "131714")] + pub const fn set_port(&mut self, new_port: u16) { self.port = new_port; } @@ -485,9 +530,10 @@ impl SocketAddrV6 { /// socket.set_flowinfo(56); /// assert_eq!(socket.flowinfo(), 56); /// ``` - #[stable(feature = "sockaddr_setters", since = "1.9.0")] #[inline] - pub fn set_flowinfo(&mut self, new_flowinfo: u32) { + #[stable(feature = "sockaddr_setters", since = "1.9.0")] + #[rustc_const_unstable(feature = "const_sockaddr_setters", issue = "131714")] + pub const fn set_flowinfo(&mut self, new_flowinfo: u32) { self.flowinfo = new_flowinfo; } @@ -527,9 +573,10 @@ impl SocketAddrV6 { /// socket.set_scope_id(42); /// assert_eq!(socket.scope_id(), 42); /// ``` - #[stable(feature = "sockaddr_setters", since = "1.9.0")] #[inline] - pub fn set_scope_id(&mut self, new_scope_id: u32) { + #[stable(feature = "sockaddr_setters", since = "1.9.0")] + #[rustc_const_unstable(feature = "const_sockaddr_setters", issue = "131714")] + pub const fn set_scope_id(&mut self, new_scope_id: u32) { self.scope_id = new_scope_id; } } diff --git a/library/core/src/num/dec2flt/decimal.rs b/library/core/src/num/dec2flt/decimal.rs index 350f64bb4f7a3..be9c0eccd5eb8 100644 --- a/library/core/src/num/dec2flt/decimal.rs +++ b/library/core/src/num/dec2flt/decimal.rs @@ -9,7 +9,7 @@ //! algorithm can be found in "ParseNumberF64 by Simple Decimal Conversion", //! available online: . -use crate::num::dec2flt::common::{is_8digits, ByteSlice}; +use crate::num::dec2flt::common::{ByteSlice, is_8digits}; #[derive(Clone)] pub struct Decimal { diff --git a/library/core/src/num/dec2flt/parse.rs b/library/core/src/num/dec2flt/parse.rs index 975bb8ad6bc1f..06ee8e95fbc47 100644 --- a/library/core/src/num/dec2flt/parse.rs +++ b/library/core/src/num/dec2flt/parse.rs @@ -1,6 +1,6 @@ //! Functions to parse floating-point numbers. -use crate::num::dec2flt::common::{is_8digits, ByteSlice}; +use crate::num::dec2flt::common::{ByteSlice, is_8digits}; use crate::num::dec2flt::float::RawFloat; use crate::num::dec2flt::number::Number; diff --git a/library/core/src/num/dec2flt/slow.rs b/library/core/src/num/dec2flt/slow.rs index bf1044033e69e..85d4b13284b7d 100644 --- a/library/core/src/num/dec2flt/slow.rs +++ b/library/core/src/num/dec2flt/slow.rs @@ -1,7 +1,7 @@ //! Slow, fallback algorithm for cases the Eisel-Lemire algorithm cannot round. use crate::num::dec2flt::common::BiasedFp; -use crate::num::dec2flt::decimal::{parse_decimal, Decimal}; +use crate::num::dec2flt::decimal::{Decimal, parse_decimal}; use crate::num::dec2flt::float::RawFloat; /// Parse the significant digits and biased, binary exponent of a float. diff --git a/library/core/src/num/f128.rs b/library/core/src/num/f128.rs index d4236e47bfe3b..e8161cce2fe29 100644 --- a/library/core/src/num/f128.rs +++ b/library/core/src/num/f128.rs @@ -288,7 +288,6 @@ impl f128 { // concerns about portability, so this implementation is for // private use internally. #[inline] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub(crate) const fn abs_private(self) -> f128 { // SAFETY: This transmutation is fine just like in `to_bits`/`from_bits`. unsafe { @@ -319,7 +318,6 @@ impl f128 { #[inline] #[must_use] #[unstable(feature = "f128", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub const fn is_infinite(self) -> bool { (self == f128::INFINITY) | (self == f128::NEG_INFINITY) } @@ -346,7 +344,6 @@ impl f128 { #[inline] #[must_use] #[unstable(feature = "f128", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub const fn is_finite(self) -> bool { // There's no need to handle NaN separately: if self is NaN, // the comparison is not true, exactly as desired. @@ -380,7 +377,6 @@ impl f128 { #[inline] #[must_use] #[unstable(feature = "f128", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub const fn is_subnormal(self) -> bool { matches!(self.classify(), FpCategory::Subnormal) } @@ -412,7 +408,6 @@ impl f128 { #[inline] #[must_use] #[unstable(feature = "f128", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub const fn is_normal(self) -> bool { matches!(self.classify(), FpCategory::Normal) } @@ -437,12 +432,7 @@ impl f128 { /// ``` #[inline] #[unstable(feature = "f128", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub const fn classify(self) -> FpCategory { - // Other float types suffer from various platform bugs that violate the usual IEEE semantics - // and also make bitwise classification not always work reliably. However, `f128` cannot fit - // into any other float types so this is not a concern, and we can rely on bit patterns. - let bits = self.to_bits(); match (bits & Self::MAN_MASK, bits & Self::EXP_MASK) { (0, Self::EXP_MASK) => FpCategory::Infinite, @@ -475,7 +465,7 @@ impl f128 { #[inline] #[must_use] #[unstable(feature = "f128", issue = "116909")] - pub fn is_sign_positive(self) -> bool { + pub const fn is_sign_positive(self) -> bool { !self.is_sign_negative() } @@ -501,7 +491,7 @@ impl f128 { #[inline] #[must_use] #[unstable(feature = "f128", issue = "116909")] - pub fn is_sign_negative(self) -> bool { + pub const fn is_sign_negative(self) -> bool { // IEEE754 says: isSignMinus(x) is true if and only if x has negative sign. isSignMinus // applies to zeros and NaNs as well. // SAFETY: This is just transmuting to get the sign bit, it's fine. @@ -542,7 +532,7 @@ impl f128 { #[inline] #[unstable(feature = "f128", issue = "116909")] // #[unstable(feature = "float_next_up_down", issue = "91399")] - pub fn next_up(self) -> Self { + pub const fn next_up(self) -> Self { // Some targets violate Rust's assumption of IEEE semantics, e.g. by flushing // denormals to zero. This is in general unsound and unsupported, but here // we do our best to still produce the correct result on such targets. @@ -596,7 +586,7 @@ impl f128 { #[inline] #[unstable(feature = "f128", issue = "116909")] // #[unstable(feature = "float_next_up_down", issue = "91399")] - pub fn next_down(self) -> Self { + pub const fn next_down(self) -> Self { // Some targets violate Rust's assumption of IEEE semantics, e.g. by flushing // denormals to zero. This is in general unsound and unsupported, but here // we do our best to still produce the correct result on such targets. @@ -631,8 +621,9 @@ impl f128 { /// ``` #[inline] #[unstable(feature = "f128", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "this returns the result of the operation, without modifying the original"] - pub fn recip(self) -> Self { + pub const fn recip(self) -> Self { 1.0 / self } @@ -651,8 +642,9 @@ impl f128 { /// ``` #[inline] #[unstable(feature = "f128", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "this returns the result of the operation, without modifying the original"] - pub fn to_degrees(self) -> Self { + pub const fn to_degrees(self) -> Self { // Use a literal for better precision. const PIS_IN_180: f128 = 57.2957795130823208767981548141051703324054724665643215491602_f128; self * PIS_IN_180 @@ -674,8 +666,9 @@ impl f128 { /// ``` #[inline] #[unstable(feature = "f128", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "this returns the result of the operation, without modifying the original"] - pub fn to_radians(self) -> f128 { + pub const fn to_radians(self) -> f128 { // Use a literal for better precision. const RADS_PER_DEG: f128 = 0.0174532925199432957692369076848861271344287188854172545609719_f128; @@ -702,8 +695,9 @@ impl f128 { /// ``` #[inline] #[unstable(feature = "f128", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "this returns the result of the comparison, without modifying either input"] - pub fn max(self, other: f128) -> f128 { + pub const fn max(self, other: f128) -> f128 { intrinsics::maxnumf128(self, other) } @@ -727,8 +721,9 @@ impl f128 { /// ``` #[inline] #[unstable(feature = "f128", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "this returns the result of the comparison, without modifying either input"] - pub fn min(self, other: f128) -> f128 { + pub const fn min(self, other: f128) -> f128 { intrinsics::minnumf128(self, other) } @@ -761,7 +756,7 @@ impl f128 { #[unstable(feature = "f128", issue = "116909")] // #[unstable(feature = "float_minimum_maximum", issue = "91079")] #[must_use = "this returns the result of the comparison, without modifying either input"] - pub fn maximum(self, other: f128) -> f128 { + pub const fn maximum(self, other: f128) -> f128 { if self > other { self } else if other > self { @@ -802,7 +797,7 @@ impl f128 { #[unstable(feature = "f128", issue = "116909")] // #[unstable(feature = "float_minimum_maximum", issue = "91079")] #[must_use = "this returns the result of the comparison, without modifying either input"] - pub fn minimum(self, other: f128) -> f128 { + pub const fn minimum(self, other: f128) -> f128 { if self < other { self } else if other < self { @@ -914,7 +909,6 @@ impl f128 { /// ``` #[inline] #[unstable(feature = "f128", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] #[must_use = "this returns the result of the operation, without modifying the original"] pub const fn to_bits(self) -> u128 { // SAFETY: `u128` is a plain old datatype so we can always transmute to it. @@ -963,7 +957,6 @@ impl f128 { #[inline] #[must_use] #[unstable(feature = "f128", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] pub const fn from_bits(v: u128) -> Self { // It turns out the safety issues with sNaN were overblown! Hooray! // SAFETY: `u128` is a plain old datatype so we can always transmute from it. @@ -990,7 +983,6 @@ impl f128 { /// ``` #[inline] #[unstable(feature = "f128", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] #[must_use = "this returns the result of the operation, without modifying the original"] pub const fn to_be_bytes(self) -> [u8; 16] { self.to_bits().to_be_bytes() @@ -1016,7 +1008,6 @@ impl f128 { /// ``` #[inline] #[unstable(feature = "f128", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] #[must_use = "this returns the result of the operation, without modifying the original"] pub const fn to_le_bytes(self) -> [u8; 16] { self.to_bits().to_le_bytes() @@ -1053,7 +1044,6 @@ impl f128 { /// ``` #[inline] #[unstable(feature = "f128", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] #[must_use = "this returns the result of the operation, without modifying the original"] pub const fn to_ne_bytes(self) -> [u8; 16] { self.to_bits().to_ne_bytes() @@ -1081,7 +1071,6 @@ impl f128 { #[inline] #[must_use] #[unstable(feature = "f128", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] pub const fn from_be_bytes(bytes: [u8; 16]) -> Self { Self::from_bits(u128::from_be_bytes(bytes)) } @@ -1108,7 +1097,6 @@ impl f128 { #[inline] #[must_use] #[unstable(feature = "f128", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] pub const fn from_le_bytes(bytes: [u8; 16]) -> Self { Self::from_bits(u128::from_le_bytes(bytes)) } @@ -1145,7 +1133,6 @@ impl f128 { #[inline] #[must_use] #[unstable(feature = "f128", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] pub const fn from_ne_bytes(bytes: [u8; 16]) -> Self { Self::from_bits(u128::from_ne_bytes(bytes)) } @@ -1273,9 +1260,20 @@ impl f128 { /// ``` #[inline] #[unstable(feature = "f128", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "method returns a new number and does not mutate the original value"] - pub fn clamp(mut self, min: f128, max: f128) -> f128 { - assert!(min <= max, "min > max, or either was NaN. min = {min:?}, max = {max:?}"); + pub const fn clamp(mut self, min: f128, max: f128) -> f128 { + #[inline] // inline to avoid LLVM crash + const fn assert_at_const(min: f128, max: f128) { + // Note that we cannot format in constant expressions. + assert!(min <= max, "min > max, or either was NaN"); + } + #[inline] // inline to avoid codegen regression + fn assert_at_rt(min: f128, max: f128) { + assert!(min <= max, "min > max, or either was NaN. min = {min:?}, max = {max:?}"); + } + // FIXME(const-hack): We would prefer to have streamlined panics when formatters become const-friendly. + intrinsics::const_eval_select((min, max), assert_at_const, assert_at_rt); if self < min { self = min; } diff --git a/library/core/src/num/f16.rs b/library/core/src/num/f16.rs index 1e2f841aca733..8b3f3b7d19bf7 100644 --- a/library/core/src/num/f16.rs +++ b/library/core/src/num/f16.rs @@ -282,7 +282,6 @@ impl f16 { // concerns about portability, so this implementation is for // private use internally. #[inline] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub(crate) const fn abs_private(self) -> f16 { // SAFETY: This transmutation is fine just like in `to_bits`/`from_bits`. unsafe { mem::transmute::(mem::transmute::(self) & !Self::SIGN_MASK) } @@ -310,7 +309,6 @@ impl f16 { #[inline] #[must_use] #[unstable(feature = "f16", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub const fn is_infinite(self) -> bool { (self == f16::INFINITY) | (self == f16::NEG_INFINITY) } @@ -336,7 +334,6 @@ impl f16 { #[inline] #[must_use] #[unstable(feature = "f16", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub const fn is_finite(self) -> bool { // There's no need to handle NaN separately: if self is NaN, // the comparison is not true, exactly as desired. @@ -368,7 +365,6 @@ impl f16 { #[inline] #[must_use] #[unstable(feature = "f16", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub const fn is_subnormal(self) -> bool { matches!(self.classify(), FpCategory::Subnormal) } @@ -398,7 +394,6 @@ impl f16 { #[inline] #[must_use] #[unstable(feature = "f16", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub const fn is_normal(self) -> bool { matches!(self.classify(), FpCategory::Normal) } @@ -422,44 +417,14 @@ impl f16 { /// ``` #[inline] #[unstable(feature = "f16", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub const fn classify(self) -> FpCategory { - // A previous implementation for f32/f64 tried to only use bitmask-based checks, - // using `to_bits` to transmute the float to its bit repr and match on that. - // If we only cared about being "technically" correct, that's an entirely legit - // implementation. - // - // Unfortunately, there are platforms out there that do not correctly implement the IEEE - // float semantics Rust relies on: some hardware flushes denormals to zero, and some - // platforms convert to `f32` to perform operations without properly rounding back (e.g. - // WASM, see llvm/llvm-project#96437). These are platforms bugs, and Rust will misbehave on - // such platforms, but we can at least try to make things seem as sane as possible by being - // careful here. - if self.is_infinite() { - // Thus, a value may compare unequal to infinity, despite having a "full" exponent mask. - FpCategory::Infinite - } else if self.is_nan() { - // And it may not be NaN, as it can simply be an "overextended" finite value. - FpCategory::Nan - } else { - // However, std can't simply compare to zero to check for zero, either, - // as correctness requires avoiding equality tests that may be Subnormal == -0.0 - // because it may be wrong under "denormals are zero" and "flush to zero" modes. - // Most of std's targets don't use those, but they are used for thumbv7neon. - // So, this does use bitpattern matching for the rest. On x87, due to the incorrect - // float codegen on this hardware, this doesn't actually return a right answer for NaN - // because it cannot correctly discern between a floating point NaN, and some normal - // floating point numbers truncated from an x87 FPU -- but we took care of NaN above, so - // we are fine. - // FIXME(jubilee): This probably could at least answer things correctly for Infinity, - // like the f64 version does, but I need to run more checks on how things go on x86. - // I fear losing mantissa data that would have answered that differently. - let b = self.to_bits(); - match (b & Self::MAN_MASK, b & Self::EXP_MASK) { - (0, 0) => FpCategory::Zero, - (_, 0) => FpCategory::Subnormal, - _ => FpCategory::Normal, - } + let b = self.to_bits(); + match (b & Self::MAN_MASK, b & Self::EXP_MASK) { + (0, Self::EXP_MASK) => FpCategory::Infinite, + (_, Self::EXP_MASK) => FpCategory::Nan, + (0, 0) => FpCategory::Zero, + (_, 0) => FpCategory::Subnormal, + _ => FpCategory::Normal, } } @@ -488,7 +453,7 @@ impl f16 { #[inline] #[must_use] #[unstable(feature = "f16", issue = "116909")] - pub fn is_sign_positive(self) -> bool { + pub const fn is_sign_positive(self) -> bool { !self.is_sign_negative() } @@ -517,7 +482,7 @@ impl f16 { #[inline] #[must_use] #[unstable(feature = "f16", issue = "116909")] - pub fn is_sign_negative(self) -> bool { + pub const fn is_sign_negative(self) -> bool { // IEEE754 says: isSignMinus(x) is true if and only if x has negative sign. isSignMinus // applies to zeros and NaNs as well. // SAFETY: This is just transmuting to get the sign bit, it's fine. @@ -558,7 +523,7 @@ impl f16 { #[inline] #[unstable(feature = "f16", issue = "116909")] // #[unstable(feature = "float_next_up_down", issue = "91399")] - pub fn next_up(self) -> Self { + pub const fn next_up(self) -> Self { // Some targets violate Rust's assumption of IEEE semantics, e.g. by flushing // denormals to zero. This is in general unsound and unsupported, but here // we do our best to still produce the correct result on such targets. @@ -612,7 +577,7 @@ impl f16 { #[inline] #[unstable(feature = "f16", issue = "116909")] // #[unstable(feature = "float_next_up_down", issue = "91399")] - pub fn next_down(self) -> Self { + pub const fn next_down(self) -> Self { // Some targets violate Rust's assumption of IEEE semantics, e.g. by flushing // denormals to zero. This is in general unsound and unsupported, but here // we do our best to still produce the correct result on such targets. @@ -647,8 +612,9 @@ impl f16 { /// ``` #[inline] #[unstable(feature = "f16", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "this returns the result of the operation, without modifying the original"] - pub fn recip(self) -> Self { + pub const fn recip(self) -> Self { 1.0 / self } @@ -667,8 +633,9 @@ impl f16 { /// ``` #[inline] #[unstable(feature = "f16", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "this returns the result of the operation, without modifying the original"] - pub fn to_degrees(self) -> Self { + pub const fn to_degrees(self) -> Self { // Use a literal for better precision. const PIS_IN_180: f16 = 57.2957795130823208767981548141051703_f16; self * PIS_IN_180 @@ -690,8 +657,9 @@ impl f16 { /// ``` #[inline] #[unstable(feature = "f16", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "this returns the result of the operation, without modifying the original"] - pub fn to_radians(self) -> f16 { + pub const fn to_radians(self) -> f16 { // Use a literal for better precision. const RADS_PER_DEG: f16 = 0.017453292519943295769236907684886_f16; self * RADS_PER_DEG @@ -716,8 +684,9 @@ impl f16 { /// ``` #[inline] #[unstable(feature = "f16", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "this returns the result of the comparison, without modifying either input"] - pub fn max(self, other: f16) -> f16 { + pub const fn max(self, other: f16) -> f16 { intrinsics::maxnumf16(self, other) } @@ -740,8 +709,9 @@ impl f16 { /// ``` #[inline] #[unstable(feature = "f16", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "this returns the result of the comparison, without modifying either input"] - pub fn min(self, other: f16) -> f16 { + pub const fn min(self, other: f16) -> f16 { intrinsics::minnumf16(self, other) } @@ -773,7 +743,7 @@ impl f16 { #[unstable(feature = "f16", issue = "116909")] // #[unstable(feature = "float_minimum_maximum", issue = "91079")] #[must_use = "this returns the result of the comparison, without modifying either input"] - pub fn maximum(self, other: f16) -> f16 { + pub const fn maximum(self, other: f16) -> f16 { if self > other { self } else if other > self { @@ -813,7 +783,7 @@ impl f16 { #[unstable(feature = "f16", issue = "116909")] // #[unstable(feature = "float_minimum_maximum", issue = "91079")] #[must_use = "this returns the result of the comparison, without modifying either input"] - pub fn minimum(self, other: f16) -> f16 { + pub const fn minimum(self, other: f16) -> f16 { if self < other { self } else if other < self { @@ -925,7 +895,6 @@ impl f16 { /// ``` #[inline] #[unstable(feature = "f16", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] #[must_use = "this returns the result of the operation, without modifying the original"] pub const fn to_bits(self) -> u16 { // SAFETY: `u16` is a plain old datatype so we can always transmute to it. @@ -973,7 +942,6 @@ impl f16 { #[inline] #[must_use] #[unstable(feature = "f16", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] pub const fn from_bits(v: u16) -> Self { // It turns out the safety issues with sNaN were overblown! Hooray! // SAFETY: `u16` is a plain old datatype so we can always transmute from it. @@ -999,7 +967,6 @@ impl f16 { /// ``` #[inline] #[unstable(feature = "f16", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] #[must_use = "this returns the result of the operation, without modifying the original"] pub const fn to_be_bytes(self) -> [u8; 2] { self.to_bits().to_be_bytes() @@ -1024,7 +991,6 @@ impl f16 { /// ``` #[inline] #[unstable(feature = "f16", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] #[must_use = "this returns the result of the operation, without modifying the original"] pub const fn to_le_bytes(self) -> [u8; 2] { self.to_bits().to_le_bytes() @@ -1062,7 +1028,6 @@ impl f16 { /// ``` #[inline] #[unstable(feature = "f16", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] #[must_use = "this returns the result of the operation, without modifying the original"] pub const fn to_ne_bytes(self) -> [u8; 2] { self.to_bits().to_ne_bytes() @@ -1086,7 +1051,6 @@ impl f16 { #[inline] #[must_use] #[unstable(feature = "f16", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] pub const fn from_be_bytes(bytes: [u8; 2]) -> Self { Self::from_bits(u16::from_be_bytes(bytes)) } @@ -1109,7 +1073,6 @@ impl f16 { #[inline] #[must_use] #[unstable(feature = "f16", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] pub const fn from_le_bytes(bytes: [u8; 2]) -> Self { Self::from_bits(u16::from_le_bytes(bytes)) } @@ -1143,7 +1106,6 @@ impl f16 { #[inline] #[must_use] #[unstable(feature = "f16", issue = "116909")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] pub const fn from_ne_bytes(bytes: [u8; 2]) -> Self { Self::from_bits(u16::from_ne_bytes(bytes)) } @@ -1273,9 +1235,20 @@ impl f16 { /// ``` #[inline] #[unstable(feature = "f16", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "method returns a new number and does not mutate the original value"] - pub fn clamp(mut self, min: f16, max: f16) -> f16 { - assert!(min <= max, "min > max, or either was NaN. min = {min:?}, max = {max:?}"); + pub const fn clamp(mut self, min: f16, max: f16) -> f16 { + #[inline] // inline to avoid LLVM crash + const fn assert_at_const(min: f16, max: f16) { + // Note that we cannot format in constant expressions. + assert!(min <= max, "min > max, or either was NaN"); + } + #[inline] // inline to avoid codegen regression + fn assert_at_rt(min: f16, max: f16) { + assert!(min <= max, "min > max, or either was NaN. min = {min:?}, max = {max:?}"); + } + // FIXME(const-hack): We would prefer to have streamlined panics when formatters become const-friendly. + intrinsics::const_eval_select((min, max), assert_at_const, assert_at_rt); if self < min { self = min; } diff --git a/library/core/src/num/f32.rs b/library/core/src/num/f32.rs index c1adcc753f2e5..a01761ee5d4a3 100644 --- a/library/core/src/num/f32.rs +++ b/library/core/src/num/f32.rs @@ -415,6 +415,7 @@ impl f32 { /// [Machine epsilon]: https://en.wikipedia.org/wiki/Machine_epsilon /// [`MANTISSA_DIGITS`]: f32::MANTISSA_DIGITS #[stable(feature = "assoc_int_consts", since = "1.43.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "f32_epsilon")] pub const EPSILON: f32 = 1.19209290e-07_f32; /// Smallest finite `f32` value. @@ -516,7 +517,7 @@ impl f32 { /// ``` #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] #[allow(clippy::eq_op)] // > if you intended to check if the operand is NaN, use `.is_nan()` instead :) pub const fn is_nan(self) -> bool { @@ -527,7 +528,6 @@ impl f32 { // concerns about portability, so this implementation is for // private use internally. #[inline] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub(crate) const fn abs_private(self) -> f32 { // SAFETY: This transmutation is fine just like in `to_bits`/`from_bits`. unsafe { mem::transmute::(mem::transmute::(self) & !Self::SIGN_MASK) } @@ -550,7 +550,7 @@ impl f32 { /// ``` #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] pub const fn is_infinite(self) -> bool { // Getting clever with transmutation can result in incorrect answers on some FPUs @@ -575,7 +575,7 @@ impl f32 { /// ``` #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] pub const fn is_finite(self) -> bool { // There's no need to handle NaN separately: if self is NaN, @@ -603,7 +603,7 @@ impl f32 { /// [subnormal]: https://en.wikipedia.org/wiki/Denormal_number #[must_use] #[stable(feature = "is_subnormal", since = "1.53.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] pub const fn is_subnormal(self) -> bool { matches!(self.classify(), FpCategory::Subnormal) @@ -630,7 +630,7 @@ impl f32 { /// [subnormal]: https://en.wikipedia.org/wiki/Denormal_number #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] pub const fn is_normal(self) -> bool { matches!(self.classify(), FpCategory::Normal) @@ -650,47 +650,20 @@ impl f32 { /// assert_eq!(inf.classify(), FpCategory::Infinite); /// ``` #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] pub const fn classify(self) -> FpCategory { - // A previous implementation tried to only use bitmask-based checks, - // using f32::to_bits to transmute the float to its bit repr and match on that. - // If we only cared about being "technically" correct, that's an entirely legit - // implementation. - // - // Unfortunately, there is hardware out there that does not correctly implement the IEEE - // float semantics Rust relies on: x87 uses a too-large mantissa and exponent, and some - // hardware flushes subnormals to zero. These are platforms bugs, and Rust will misbehave on - // such hardware, but we can at least try to make things seem as sane as possible by being - // careful here. - // - // FIXME(jubilee): Using x87 operations is never necessary in order to function - // on x86 processors for Rust-to-Rust calls, so this issue should not happen. - // Code generation should be adjusted to use non-C calling conventions, avoiding this. - if self.is_infinite() { - // A value may compare unequal to infinity, despite having a "full" exponent mask. - FpCategory::Infinite - } else if self.is_nan() { - // And it may not be NaN, as it can simply be an "overextended" finite value. - FpCategory::Nan - } else { - // However, std can't simply compare to zero to check for zero, either, - // as correctness requires avoiding equality tests that may be Subnormal == -0.0 - // because it may be wrong under "denormals are zero" and "flush to zero" modes. - // Most of std's targets don't use those, but they are used for thumbv7neon. - // So, this does use bitpattern matching for the rest. On x87, due to the incorrect - // float codegen on this hardware, this doesn't actually return a right answer for NaN - // because it cannot correctly discern between a floating point NaN, and some normal - // floating point numbers truncated from an x87 FPU -- but we took care of NaN above, so - // we are fine. - // FIXME(jubilee): This probably could at least answer things correctly for Infinity, - // like the f64 version does, but I need to run more checks on how things go on x86. - // I fear losing mantissa data that would have answered that differently. - let b = self.to_bits(); - match (b & Self::MAN_MASK, b & Self::EXP_MASK) { - (0, 0) => FpCategory::Zero, - (_, 0) => FpCategory::Subnormal, - _ => FpCategory::Normal, - } + // We used to have complicated logic here that avoids the simple bit-based tests to work + // around buggy codegen for x87 targets (see + // https://github.com/rust-lang/rust/issues/114479). However, some LLVM versions later, none + // of our tests is able to find any difference between the complicated and the naive + // version, so now we are back to the naive version. + let b = self.to_bits(); + match (b & Self::MAN_MASK, b & Self::EXP_MASK) { + (0, Self::EXP_MASK) => FpCategory::Infinite, + (_, Self::EXP_MASK) => FpCategory::Nan, + (0, 0) => FpCategory::Zero, + (_, 0) => FpCategory::Subnormal, + _ => FpCategory::Normal, } } @@ -713,7 +686,7 @@ impl f32 { /// ``` #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] pub const fn is_sign_positive(self) -> bool { !self.is_sign_negative() @@ -738,7 +711,7 @@ impl f32 { /// ``` #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] pub const fn is_sign_negative(self) -> bool { // IEEE754 says: isSignMinus(x) is true if and only if x has negative sign. isSignMinus @@ -855,8 +828,9 @@ impl f32 { /// ``` #[must_use = "this returns the result of the operation, without modifying the original"] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn recip(self) -> f32 { + pub const fn recip(self) -> f32 { 1.0 / self } @@ -872,8 +846,9 @@ impl f32 { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[stable(feature = "f32_deg_rad_conversions", since = "1.7.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn to_degrees(self) -> f32 { + pub const fn to_degrees(self) -> f32 { // Use a constant for better precision. const PIS_IN_180: f32 = 57.2957795130823208767981548141051703_f32; self * PIS_IN_180 @@ -891,8 +866,9 @@ impl f32 { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[stable(feature = "f32_deg_rad_conversions", since = "1.7.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn to_radians(self) -> f32 { + pub const fn to_radians(self) -> f32 { const RADS_PER_DEG: f32 = consts::PI / 180.0; self * RADS_PER_DEG } @@ -912,8 +888,9 @@ impl f32 { /// ``` #[must_use = "this returns the result of the comparison, without modifying either input"] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn max(self, other: f32) -> f32 { + pub const fn max(self, other: f32) -> f32 { intrinsics::maxnumf32(self, other) } @@ -932,8 +909,9 @@ impl f32 { /// ``` #[must_use = "this returns the result of the comparison, without modifying either input"] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn min(self, other: f32) -> f32 { + pub const fn min(self, other: f32) -> f32 { intrinsics::minnumf32(self, other) } @@ -960,7 +938,7 @@ impl f32 { #[must_use = "this returns the result of the comparison, without modifying either input"] #[unstable(feature = "float_minimum_maximum", issue = "91079")] #[inline] - pub fn maximum(self, other: f32) -> f32 { + pub const fn maximum(self, other: f32) -> f32 { if self > other { self } else if other > self { @@ -995,7 +973,7 @@ impl f32 { #[must_use = "this returns the result of the comparison, without modifying either input"] #[unstable(feature = "float_minimum_maximum", issue = "91079")] #[inline] - pub fn minimum(self, other: f32) -> f32 { + pub const fn minimum(self, other: f32) -> f32 { if self < other { self } else if other < self { @@ -1115,7 +1093,7 @@ impl f32 { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[stable(feature = "float_bits_conv", since = "1.20.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[inline] pub const fn to_bits(self) -> u32 { // SAFETY: `u32` is a plain old datatype so we can always transmute to it. @@ -1159,7 +1137,7 @@ impl f32 { /// assert_eq!(v, 12.5); /// ``` #[stable(feature = "float_bits_conv", since = "1.20.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[must_use] #[inline] pub const fn from_bits(v: u32) -> Self { @@ -1183,7 +1161,7 @@ impl f32 { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[stable(feature = "float_to_from_bytes", since = "1.40.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[inline] pub const fn to_be_bytes(self) -> [u8; 4] { self.to_bits().to_be_bytes() @@ -1204,7 +1182,7 @@ impl f32 { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[stable(feature = "float_to_from_bytes", since = "1.40.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[inline] pub const fn to_le_bytes(self) -> [u8; 4] { self.to_bits().to_le_bytes() @@ -1238,7 +1216,7 @@ impl f32 { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[stable(feature = "float_to_from_bytes", since = "1.40.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[inline] pub const fn to_ne_bytes(self) -> [u8; 4] { self.to_bits().to_ne_bytes() @@ -1256,7 +1234,7 @@ impl f32 { /// assert_eq!(value, 12.5); /// ``` #[stable(feature = "float_to_from_bytes", since = "1.40.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[must_use] #[inline] pub const fn from_be_bytes(bytes: [u8; 4]) -> Self { @@ -1275,7 +1253,7 @@ impl f32 { /// assert_eq!(value, 12.5); /// ``` #[stable(feature = "float_to_from_bytes", since = "1.40.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[must_use] #[inline] pub const fn from_le_bytes(bytes: [u8; 4]) -> Self { @@ -1305,7 +1283,7 @@ impl f32 { /// assert_eq!(value, 12.5); /// ``` #[stable(feature = "float_to_from_bytes", since = "1.40.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[must_use] #[inline] pub const fn from_ne_bytes(bytes: [u8; 4]) -> Self { @@ -1428,9 +1406,19 @@ impl f32 { /// ``` #[must_use = "method returns a new number and does not mutate the original value"] #[stable(feature = "clamp", since = "1.50.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn clamp(mut self, min: f32, max: f32) -> f32 { - assert!(min <= max, "min > max, or either was NaN. min = {min:?}, max = {max:?}"); + pub const fn clamp(mut self, min: f32, max: f32) -> f32 { + const fn assert_at_const(min: f32, max: f32) { + // Note that we cannot format in constant expressions. + assert!(min <= max, "min > max, or either was NaN"); + } + #[inline] // inline to avoid codegen regression + fn assert_at_rt(min: f32, max: f32) { + assert!(min <= max, "min > max, or either was NaN. min = {min:?}, max = {max:?}"); + } + // FIXME(const-hack): We would prefer to have streamlined panics when formatters become const-friendly. + intrinsics::const_eval_select((min, max), assert_at_const, assert_at_rt); if self < min { self = min; } diff --git a/library/core/src/num/f64.rs b/library/core/src/num/f64.rs index e6406771ad333..2995e41cd6ea0 100644 --- a/library/core/src/num/f64.rs +++ b/library/core/src/num/f64.rs @@ -414,6 +414,7 @@ impl f64 { /// [Machine epsilon]: https://en.wikipedia.org/wiki/Machine_epsilon /// [`MANTISSA_DIGITS`]: f64::MANTISSA_DIGITS #[stable(feature = "assoc_int_consts", since = "1.43.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "f64_epsilon")] pub const EPSILON: f64 = 2.2204460492503131e-16_f64; /// Smallest finite `f64` value. @@ -515,7 +516,7 @@ impl f64 { /// ``` #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] #[allow(clippy::eq_op)] // > if you intended to check if the operand is NaN, use `.is_nan()` instead :) pub const fn is_nan(self) -> bool { @@ -526,7 +527,6 @@ impl f64 { // concerns about portability, so this implementation is for // private use internally. #[inline] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] pub(crate) const fn abs_private(self) -> f64 { // SAFETY: This transmutation is fine just like in `to_bits`/`from_bits`. unsafe { mem::transmute::(mem::transmute::(self) & !Self::SIGN_MASK) } @@ -549,7 +549,7 @@ impl f64 { /// ``` #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] pub const fn is_infinite(self) -> bool { // Getting clever with transmutation can result in incorrect answers on some FPUs @@ -574,7 +574,7 @@ impl f64 { /// ``` #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] pub const fn is_finite(self) -> bool { // There's no need to handle NaN separately: if self is NaN, @@ -602,7 +602,7 @@ impl f64 { /// [subnormal]: https://en.wikipedia.org/wiki/Denormal_number #[must_use] #[stable(feature = "is_subnormal", since = "1.53.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] pub const fn is_subnormal(self) -> bool { matches!(self.classify(), FpCategory::Subnormal) @@ -629,7 +629,7 @@ impl f64 { /// [subnormal]: https://en.wikipedia.org/wiki/Denormal_number #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] pub const fn is_normal(self) -> bool { matches!(self.classify(), FpCategory::Normal) @@ -649,43 +649,20 @@ impl f64 { /// assert_eq!(inf.classify(), FpCategory::Infinite); /// ``` #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] pub const fn classify(self) -> FpCategory { - // A previous implementation tried to only use bitmask-based checks, - // using f64::to_bits to transmute the float to its bit repr and match on that. - // If we only cared about being "technically" correct, that's an entirely legit - // implementation. - // - // Unfortunately, there is hardware out there that does not correctly implement the IEEE - // float semantics Rust relies on: x87 uses a too-large exponent, and some hardware flushes - // subnormals to zero. These are platforms bugs, and Rust will misbehave on such hardware, - // but we can at least try to make things seem as sane as possible by being careful here. - // - // FIXME(jubilee): Using x87 operations is never necessary in order to function - // on x86 processors for Rust-to-Rust calls, so this issue should not happen. - // Code generation should be adjusted to use non-C calling conventions, avoiding this. - // - // Thus, a value may compare unequal to infinity, despite having a "full" exponent mask. - // And it may not be NaN, as it can simply be an "overextended" finite value. - if self.is_nan() { - FpCategory::Nan - } else { - // However, std can't simply compare to zero to check for zero, either, - // as correctness requires avoiding equality tests that may be Subnormal == -0.0 - // because it may be wrong under "denormals are zero" and "flush to zero" modes. - // Most of std's targets don't use those, but they are used for thumbv7neon. - // So, this does use bitpattern matching for the rest. On x87, due to the incorrect - // float codegen on this hardware, this doesn't actually return a right answer for NaN - // because it cannot correctly discern between a floating point NaN, and some normal - // floating point numbers truncated from an x87 FPU -- but we took care of NaN above, so - // we are fine. - let b = self.to_bits(); - match (b & Self::MAN_MASK, b & Self::EXP_MASK) { - (0, Self::EXP_MASK) => FpCategory::Infinite, - (0, 0) => FpCategory::Zero, - (_, 0) => FpCategory::Subnormal, - _ => FpCategory::Normal, - } + // We used to have complicated logic here that avoids the simple bit-based tests to work + // around buggy codegen for x87 targets (see + // https://github.com/rust-lang/rust/issues/114479). However, some LLVM versions later, none + // of our tests is able to find any difference between the complicated and the naive + // version, so now we are back to the naive version. + let b = self.to_bits(); + match (b & Self::MAN_MASK, b & Self::EXP_MASK) { + (0, Self::EXP_MASK) => FpCategory::Infinite, + (_, Self::EXP_MASK) => FpCategory::Nan, + (0, 0) => FpCategory::Zero, + (_, 0) => FpCategory::Subnormal, + _ => FpCategory::Normal, } } @@ -708,7 +685,7 @@ impl f64 { /// ``` #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] pub const fn is_sign_positive(self) -> bool { !self.is_sign_negative() @@ -742,7 +719,7 @@ impl f64 { /// ``` #[must_use] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_float_classify", issue = "72505")] + #[rustc_const_stable(feature = "const_float_classify", since = "1.83.0")] #[inline] pub const fn is_sign_negative(self) -> bool { // IEEE754 says: isSignMinus(x) is true if and only if x has negative sign. isSignMinus @@ -868,8 +845,9 @@ impl f64 { /// ``` #[must_use = "this returns the result of the operation, without modifying the original"] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn recip(self) -> f64 { + pub const fn recip(self) -> f64 { 1.0 / self } @@ -885,8 +863,9 @@ impl f64 { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn to_degrees(self) -> f64 { + pub const fn to_degrees(self) -> f64 { // The division here is correctly rounded with respect to the true // value of 180/π. (This differs from f32, where a constant must be // used to ensure a correctly rounded result.) @@ -905,8 +884,9 @@ impl f64 { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn to_radians(self) -> f64 { + pub const fn to_radians(self) -> f64 { const RADS_PER_DEG: f64 = consts::PI / 180.0; self * RADS_PER_DEG } @@ -926,8 +906,9 @@ impl f64 { /// ``` #[must_use = "this returns the result of the comparison, without modifying either input"] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn max(self, other: f64) -> f64 { + pub const fn max(self, other: f64) -> f64 { intrinsics::maxnumf64(self, other) } @@ -946,8 +927,9 @@ impl f64 { /// ``` #[must_use = "this returns the result of the comparison, without modifying either input"] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn min(self, other: f64) -> f64 { + pub const fn min(self, other: f64) -> f64 { intrinsics::minnumf64(self, other) } @@ -974,7 +956,7 @@ impl f64 { #[must_use = "this returns the result of the comparison, without modifying either input"] #[unstable(feature = "float_minimum_maximum", issue = "91079")] #[inline] - pub fn maximum(self, other: f64) -> f64 { + pub const fn maximum(self, other: f64) -> f64 { if self > other { self } else if other > self { @@ -1009,7 +991,7 @@ impl f64 { #[must_use = "this returns the result of the comparison, without modifying either input"] #[unstable(feature = "float_minimum_maximum", issue = "91079")] #[inline] - pub fn minimum(self, other: f64) -> f64 { + pub const fn minimum(self, other: f64) -> f64 { if self < other { self } else if other < self { @@ -1111,7 +1093,7 @@ impl f64 { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[stable(feature = "float_bits_conv", since = "1.20.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[inline] pub const fn to_bits(self) -> u64 { // SAFETY: `u64` is a plain old datatype so we can always transmute to it. @@ -1155,7 +1137,7 @@ impl f64 { /// assert_eq!(v, 12.5); /// ``` #[stable(feature = "float_bits_conv", since = "1.20.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[must_use] #[inline] pub const fn from_bits(v: u64) -> Self { @@ -1179,7 +1161,7 @@ impl f64 { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[stable(feature = "float_to_from_bytes", since = "1.40.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[inline] pub const fn to_be_bytes(self) -> [u8; 8] { self.to_bits().to_be_bytes() @@ -1200,7 +1182,7 @@ impl f64 { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[stable(feature = "float_to_from_bytes", since = "1.40.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[inline] pub const fn to_le_bytes(self) -> [u8; 8] { self.to_bits().to_le_bytes() @@ -1234,7 +1216,7 @@ impl f64 { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[stable(feature = "float_to_from_bytes", since = "1.40.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[inline] pub const fn to_ne_bytes(self) -> [u8; 8] { self.to_bits().to_ne_bytes() @@ -1252,7 +1234,7 @@ impl f64 { /// assert_eq!(value, 12.5); /// ``` #[stable(feature = "float_to_from_bytes", since = "1.40.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[must_use] #[inline] pub const fn from_be_bytes(bytes: [u8; 8]) -> Self { @@ -1271,7 +1253,7 @@ impl f64 { /// assert_eq!(value, 12.5); /// ``` #[stable(feature = "float_to_from_bytes", since = "1.40.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[must_use] #[inline] pub const fn from_le_bytes(bytes: [u8; 8]) -> Self { @@ -1301,7 +1283,7 @@ impl f64 { /// assert_eq!(value, 12.5); /// ``` #[stable(feature = "float_to_from_bytes", since = "1.40.0")] - #[rustc_const_unstable(feature = "const_float_bits_conv", issue = "72447")] + #[rustc_const_stable(feature = "const_float_bits_conv", since = "1.83.0")] #[must_use] #[inline] pub const fn from_ne_bytes(bytes: [u8; 8]) -> Self { @@ -1424,9 +1406,19 @@ impl f64 { /// ``` #[must_use = "method returns a new number and does not mutate the original value"] #[stable(feature = "clamp", since = "1.50.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn clamp(mut self, min: f64, max: f64) -> f64 { - assert!(min <= max, "min > max, or either was NaN. min = {min:?}, max = {max:?}"); + pub const fn clamp(mut self, min: f64, max: f64) -> f64 { + const fn assert_at_const(min: f64, max: f64) { + // Note that we cannot format in constant expressions. + assert!(min <= max, "min > max, or either was NaN"); + } + #[inline] // inline to avoid codegen regression + fn assert_at_rt(min: f64, max: f64) { + assert!(min <= max, "min > max, or either was NaN. min = {min:?}, max = {max:?}"); + } + // FIXME(const-hack): We would prefer to have streamlined panics when formatters become const-friendly. + intrinsics::const_eval_select((min, max), assert_at_const, assert_at_rt); if self < min { self = min; } diff --git a/library/core/src/num/flt2dec/decoder.rs b/library/core/src/num/flt2dec/decoder.rs index 5763860540aa4..40b3aae24a536 100644 --- a/library/core/src/num/flt2dec/decoder.rs +++ b/library/core/src/num/flt2dec/decoder.rs @@ -1,7 +1,7 @@ //! Decodes a floating-point value into individual parts and error ranges. -use crate::num::dec2flt::float::RawFloat; use crate::num::FpCategory; +use crate::num::dec2flt::float::RawFloat; /// Decoded unsigned finite value, such that: /// diff --git a/library/core/src/num/flt2dec/mod.rs b/library/core/src/num/flt2dec/mod.rs index 7d923a2652f28..d6413fadc3381 100644 --- a/library/core/src/num/flt2dec/mod.rs +++ b/library/core/src/num/flt2dec/mod.rs @@ -122,7 +122,7 @@ functions. issue = "none" )] -pub use self::decoder::{decode, DecodableFloat, Decoded, FullDecoded}; +pub use self::decoder::{DecodableFloat, Decoded, FullDecoded, decode}; use super::fmt::{Formatted, Part}; use crate::mem::MaybeUninit; diff --git a/library/core/src/num/flt2dec/strategy/dragon.rs b/library/core/src/num/flt2dec/strategy/dragon.rs index f8db6370653ab..e801f07b3bc0e 100644 --- a/library/core/src/num/flt2dec/strategy/dragon.rs +++ b/library/core/src/num/flt2dec/strategy/dragon.rs @@ -8,7 +8,7 @@ use crate::cmp::Ordering; use crate::mem::MaybeUninit; use crate::num::bignum::{Big32x40 as Big, Digit32 as Digit}; use crate::num::flt2dec::estimator::estimate_scaling_factor; -use crate::num::flt2dec::{round_up, Decoded, MAX_SIG_DIGITS}; +use crate::num::flt2dec::{Decoded, MAX_SIG_DIGITS, round_up}; static POW10: [Digit; 10] = [1, 10, 100, 1000, 10000, 100000, 1000000, 10000000, 100000000, 1000000000]; diff --git a/library/core/src/num/flt2dec/strategy/grisu.rs b/library/core/src/num/flt2dec/strategy/grisu.rs index b9f0d114c6a14..bdf544a4133bb 100644 --- a/library/core/src/num/flt2dec/strategy/grisu.rs +++ b/library/core/src/num/flt2dec/strategy/grisu.rs @@ -7,7 +7,7 @@ use crate::mem::MaybeUninit; use crate::num::diy_float::Fp; -use crate::num::flt2dec::{round_up, Decoded, MAX_SIG_DIGITS}; +use crate::num::flt2dec::{Decoded, MAX_SIG_DIGITS, round_up}; // see the comments in `format_shortest_opt` for the rationale. #[doc(hidden)] diff --git a/library/core/src/num/int_macros.rs b/library/core/src/num/int_macros.rs index 4ff7f97542d93..ee687e3774617 100644 --- a/library/core/src/num/int_macros.rs +++ b/library/core/src/num/int_macros.rs @@ -449,7 +449,7 @@ macro_rules! int_impl { #[inline] pub const fn checked_add(self, rhs: Self) -> Option { let (a, b) = self.overflowing_add(rhs); - if unlikely!(b) { None } else { Some(a) } + if intrinsics::unlikely(b) { None } else { Some(a) } } /// Strict integer addition. Computes `self + rhs`, panicking @@ -546,7 +546,7 @@ macro_rules! int_impl { #[inline] pub const fn checked_add_unsigned(self, rhs: $UnsignedT) -> Option { let (a, b) = self.overflowing_add_unsigned(rhs); - if unlikely!(b) { None } else { Some(a) } + if intrinsics::unlikely(b) { None } else { Some(a) } } /// Strict addition with an unsigned integer. Computes `self + rhs`, @@ -602,7 +602,7 @@ macro_rules! int_impl { #[inline] pub const fn checked_sub(self, rhs: Self) -> Option { let (a, b) = self.overflowing_sub(rhs); - if unlikely!(b) { None } else { Some(a) } + if intrinsics::unlikely(b) { None } else { Some(a) } } /// Strict integer subtraction. Computes `self - rhs`, panicking if @@ -699,7 +699,7 @@ macro_rules! int_impl { #[inline] pub const fn checked_sub_unsigned(self, rhs: $UnsignedT) -> Option { let (a, b) = self.overflowing_sub_unsigned(rhs); - if unlikely!(b) { None } else { Some(a) } + if intrinsics::unlikely(b) { None } else { Some(a) } } /// Strict subtraction with an unsigned integer. Computes `self - rhs`, @@ -755,7 +755,7 @@ macro_rules! int_impl { #[inline] pub const fn checked_mul(self, rhs: Self) -> Option { let (a, b) = self.overflowing_mul(rhs); - if unlikely!(b) { None } else { Some(a) } + if intrinsics::unlikely(b) { None } else { Some(a) } } /// Strict integer multiplication. Computes `self * rhs`, panicking if @@ -852,7 +852,7 @@ macro_rules! int_impl { without modifying the original"] #[inline] pub const fn checked_div(self, rhs: Self) -> Option { - if unlikely!(rhs == 0 || ((self == Self::MIN) && (rhs == -1))) { + if intrinsics::unlikely(rhs == 0 || ((self == Self::MIN) && (rhs == -1))) { None } else { // SAFETY: div by zero and by INT_MIN have been checked above @@ -927,7 +927,7 @@ macro_rules! int_impl { #[inline] pub const fn checked_div_euclid(self, rhs: Self) -> Option { // Using `&` helps LLVM see that it is the same check made in division. - if unlikely!(rhs == 0 || ((self == Self::MIN) & (rhs == -1))) { + if intrinsics::unlikely(rhs == 0 || ((self == Self::MIN) & (rhs == -1))) { None } else { Some(self.div_euclid(rhs)) @@ -1000,7 +1000,7 @@ macro_rules! int_impl { without modifying the original"] #[inline] pub const fn checked_rem(self, rhs: Self) -> Option { - if unlikely!(rhs == 0 || ((self == Self::MIN) && (rhs == -1))) { + if intrinsics::unlikely(rhs == 0 || ((self == Self::MIN) && (rhs == -1))) { None } else { // SAFETY: div by zero and by INT_MIN have been checked above @@ -1074,7 +1074,7 @@ macro_rules! int_impl { #[inline] pub const fn checked_rem_euclid(self, rhs: Self) -> Option { // Using `&` helps LLVM see that it is the same check made in division. - if unlikely!(rhs == 0 || ((self == Self::MIN) & (rhs == -1))) { + if intrinsics::unlikely(rhs == 0 || ((self == Self::MIN) & (rhs == -1))) { None } else { Some(self.rem_euclid(rhs)) @@ -1145,7 +1145,7 @@ macro_rules! int_impl { #[inline] pub const fn checked_neg(self) -> Option { let (a, b) = self.overflowing_neg(); - if unlikely!(b) { None } else { Some(a) } + if intrinsics::unlikely(b) { None } else { Some(a) } } /// Unchecked negation. Computes `-self`, assuming overflow cannot occur. @@ -1164,7 +1164,7 @@ macro_rules! int_impl { )] #[must_use = "this returns the result of the operation, \ without modifying the original"] - #[rustc_const_unstable(feature = "unchecked_neg", issue = "85122")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "unchecked_neg", issue = "85122"))] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[requires(self != $SelfT::MIN)] @@ -1232,8 +1232,7 @@ macro_rules! int_impl { /// ``` #[stable(feature = "wrapping", since = "1.7.0")] #[rustc_const_stable(feature = "const_checked_int_methods", since = "1.47.0")] - // We could always go back to wrapping - #[rustc_allow_const_fn_unstable(unchecked_shifts)] + #[cfg_attr(bootstrap, rustc_allow_const_fn_unstable(unchecked_shifts))] #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline] @@ -1299,7 +1298,7 @@ macro_rules! int_impl { )] #[must_use = "this returns the result of the operation, \ without modifying the original"] - #[rustc_const_unstable(feature = "unchecked_shifts", issue = "85122")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "unchecked_shifts", issue = "85122"))] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[requires(rhs < <$ActualT>::BITS)] @@ -1359,8 +1358,7 @@ macro_rules! int_impl { /// ``` #[stable(feature = "wrapping", since = "1.7.0")] #[rustc_const_stable(feature = "const_checked_int_methods", since = "1.47.0")] - // We could always go back to wrapping - #[rustc_allow_const_fn_unstable(unchecked_shifts)] + #[cfg_attr(bootstrap, rustc_allow_const_fn_unstable(unchecked_shifts))] #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline] @@ -1426,7 +1424,7 @@ macro_rules! int_impl { )] #[must_use = "this returns the result of the operation, \ without modifying the original"] - #[rustc_const_unstable(feature = "unchecked_shifts", issue = "85122")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "unchecked_shifts", issue = "85122"))] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[requires(rhs < <$ActualT>::BITS)] // i.e. requires the right hand side of the shift (rhs) to be less than the number of bits in the type. This prevents undefined behavior. @@ -1636,11 +1634,10 @@ macro_rules! int_impl { /// /// Basic usage: /// ``` - /// #![feature(isqrt)] #[doc = concat!("assert_eq!(10", stringify!($SelfT), ".checked_isqrt(), Some(3));")] /// ``` - #[unstable(feature = "isqrt", issue = "116226")] - #[rustc_const_unstable(feature = "isqrt", issue = "116226")] + #[stable(feature = "isqrt", since = "CURRENT_RUSTC_VERSION")] + #[rustc_const_stable(feature = "isqrt", since = "CURRENT_RUSTC_VERSION")] #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline] @@ -2159,8 +2156,8 @@ macro_rules! int_impl { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline(always)] - #[rustc_allow_const_fn_unstable(unchecked_shifts)] #[ensures(|result| *result == self << (rhs & (Self::BITS - 1)))] + #[cfg_attr(bootstrap, rustc_allow_const_fn_unstable(unchecked_shifts))] pub const fn wrapping_shl(self, rhs: u32) -> Self { // SAFETY: the masking by the bitsize of the type ensures that we do not shift // out of bounds @@ -2190,8 +2187,8 @@ macro_rules! int_impl { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline(always)] - #[rustc_allow_const_fn_unstable(unchecked_shifts)] #[ensures(|result| *result == self >> (rhs & (Self::BITS - 1)))] + #[cfg_attr(bootstrap, rustc_allow_const_fn_unstable(unchecked_shifts))] pub const fn wrapping_shr(self, rhs: u32) -> Self { // SAFETY: the masking by the bitsize of the type ensures that we do not shift // out of bounds @@ -2573,7 +2570,7 @@ macro_rules! int_impl { without modifying the original"] pub const fn overflowing_div(self, rhs: Self) -> (Self, bool) { // Using `&` helps LLVM see that it is the same check made in division. - if unlikely!((self == Self::MIN) & (rhs == -1)) { + if intrinsics::unlikely((self == Self::MIN) & (rhs == -1)) { (self, true) } else { (self / rhs, false) @@ -2604,7 +2601,7 @@ macro_rules! int_impl { without modifying the original"] pub const fn overflowing_div_euclid(self, rhs: Self) -> (Self, bool) { // Using `&` helps LLVM see that it is the same check made in division. - if unlikely!((self == Self::MIN) & (rhs == -1)) { + if intrinsics::unlikely((self == Self::MIN) & (rhs == -1)) { (self, true) } else { (self.div_euclid(rhs), false) @@ -2634,7 +2631,7 @@ macro_rules! int_impl { #[must_use = "this returns the result of the operation, \ without modifying the original"] pub const fn overflowing_rem(self, rhs: Self) -> (Self, bool) { - if unlikely!(rhs == -1) { + if intrinsics::unlikely(rhs == -1) { (0, self == Self::MIN) } else { (self % rhs, false) @@ -2666,7 +2663,7 @@ macro_rules! int_impl { #[inline] #[track_caller] pub const fn overflowing_rem_euclid(self, rhs: Self) -> (Self, bool) { - if unlikely!(rhs == -1) { + if intrinsics::unlikely(rhs == -1) { (0, self == Self::MIN) } else { (self.rem_euclid(rhs), false) @@ -2695,7 +2692,7 @@ macro_rules! int_impl { without modifying the original"] #[allow(unused_attributes)] pub const fn overflowing_neg(self) -> (Self, bool) { - if unlikely!(self == Self::MIN) { + if intrinsics::unlikely(self == Self::MIN) { (Self::MIN, true) } else { (-self, false) @@ -2889,11 +2886,10 @@ macro_rules! int_impl { /// /// Basic usage: /// ``` - /// #![feature(isqrt)] #[doc = concat!("assert_eq!(10", stringify!($SelfT), ".isqrt(), 3);")] /// ``` - #[unstable(feature = "isqrt", issue = "116226")] - #[rustc_const_unstable(feature = "isqrt", issue = "116226")] + #[stable(feature = "isqrt", since = "CURRENT_RUSTC_VERSION")] + #[rustc_const_stable(feature = "isqrt", since = "CURRENT_RUSTC_VERSION")] #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline] @@ -3032,8 +3028,16 @@ macro_rules! int_impl { pub const fn div_floor(self, rhs: Self) -> Self { let d = self / rhs; let r = self % rhs; - if (r > 0 && rhs < 0) || (r < 0 && rhs > 0) { - d - 1 + + // If the remainder is non-zero, we need to subtract one if the + // signs of self and rhs differ, as this means we rounded upwards + // instead of downwards. We do this branchlessly by creating a mask + // which is all-ones iff the signs differ, and 0 otherwise. Then by + // adding this mask (which corresponds to the signed value -1), we + // get our correction. + let correction = (self ^ rhs) >> (Self::BITS - 1); + if r != 0 { + d + correction } else { d } @@ -3068,8 +3072,12 @@ macro_rules! int_impl { pub const fn div_ceil(self, rhs: Self) -> Self { let d = self / rhs; let r = self % rhs; - if (r > 0 && rhs > 0) || (r < 0 && rhs < 0) { - d + 1 + + // When remainder is non-zero we have a.div_ceil(b) == 1 + a.div_floor(b), + // so we can re-use the algorithm from div_floor, just adding 1. + let correction = 1 + ((self ^ rhs) >> (Self::BITS - 1)); + if r != 0 { + d + correction } else { d } @@ -3178,44 +3186,6 @@ macro_rules! int_impl { } } - /// Calculates the middle point of `self` and `rhs`. - /// - /// `midpoint(a, b)` is `(a + b) >> 1` as if it were performed in a - /// sufficiently-large signed integral type. This implies that the result is - /// always rounded towards negative infinity and that no overflow will ever occur. - /// - /// # Examples - /// - /// ``` - /// #![feature(num_midpoint)] - #[doc = concat!("assert_eq!(0", stringify!($SelfT), ".midpoint(4), 2);")] - #[doc = concat!("assert_eq!(0", stringify!($SelfT), ".midpoint(-1), -1);")] - #[doc = concat!("assert_eq!((-1", stringify!($SelfT), ").midpoint(0), -1);")] - /// ``` - #[unstable(feature = "num_midpoint", issue = "110840")] - #[rustc_const_unstable(feature = "const_num_midpoint", issue = "110840")] - #[rustc_allow_const_fn_unstable(const_num_midpoint)] - #[must_use = "this returns the result of the operation, \ - without modifying the original"] - #[inline] - pub const fn midpoint(self, rhs: Self) -> Self { - const U: $UnsignedT = <$SelfT>::MIN.unsigned_abs(); - - // Map an $SelfT to an $UnsignedT - // ex: i8 [-128; 127] to [0; 255] - const fn map(a: $SelfT) -> $UnsignedT { - (a as $UnsignedT) ^ U - } - - // Map an $UnsignedT to an $SelfT - // ex: u8 [0; 255] to [-128; 127] - const fn demap(a: $UnsignedT) -> $SelfT { - (a ^ U) as $SelfT - } - - demap(<$UnsignedT>::midpoint(map(self), map(rhs))) - } - /// Returns the logarithm of the number with respect to an arbitrary base, /// rounded down. /// diff --git a/library/core/src/num/mod.rs b/library/core/src/num/mod.rs index b123d998a8466..765052e9cbaa6 100644 --- a/library/core/src/num/mod.rs +++ b/library/core/src/num/mod.rs @@ -10,7 +10,7 @@ use safety::{requires, ensures}; #[cfg(kani)] use crate::kani; -// Used because the `?` operator is not allowed in a const context. +// FIXME(const-hack): Used because the `?` operator is not allowed in a const context. macro_rules! try_opt { ($e:expr) => { match $e { @@ -20,10 +20,13 @@ macro_rules! try_opt { }; } -#[allow_internal_unstable(const_likely)] -macro_rules! unlikely { - ($e: expr) => { - intrinsics::unlikely($e) +// Use this when the generated code should differ between signed and unsigned types. +macro_rules! sign_dependent_expr { + (signed ? if signed { $signed_case:expr } if unsigned { $unsigned_case:expr } ) => { + $signed_case + }; + (unsigned ? if signed { $signed_case:expr } if unsigned { $unsigned_case:expr } ) => { + $unsigned_case }; } @@ -69,9 +72,9 @@ pub use nonzero::NonZero; )] pub use nonzero::ZeroablePrimitive; #[stable(feature = "signed_nonzero", since = "1.34.0")] -pub use nonzero::{NonZeroI128, NonZeroI16, NonZeroI32, NonZeroI64, NonZeroI8, NonZeroIsize}; +pub use nonzero::{NonZeroI8, NonZeroI16, NonZeroI32, NonZeroI64, NonZeroI128, NonZeroIsize}; #[stable(feature = "nonzero", since = "1.28.0")] -pub use nonzero::{NonZeroU128, NonZeroU16, NonZeroU32, NonZeroU64, NonZeroU8, NonZeroUsize}; +pub use nonzero::{NonZeroU8, NonZeroU16, NonZeroU32, NonZeroU64, NonZeroU128, NonZeroUsize}; #[stable(feature = "saturating_int_impl", since = "1.74.0")] pub use saturating::Saturating; #[stable(feature = "rust1", since = "1.0.0")] @@ -125,6 +128,37 @@ macro_rules! midpoint_impl { ((self ^ rhs) >> 1) + (self & rhs) } }; + ($SelfT:ty, signed) => { + /// Calculates the middle point of `self` and `rhs`. + /// + /// `midpoint(a, b)` is `(a + b) / 2` as if it were performed in a + /// sufficiently-large signed integral type. This implies that the result is + /// always rounded towards zero and that no overflow will ever occur. + /// + /// # Examples + /// + /// ``` + /// #![feature(num_midpoint)] + #[doc = concat!("assert_eq!(0", stringify!($SelfT), ".midpoint(4), 2);")] + #[doc = concat!("assert_eq!((-1", stringify!($SelfT), ").midpoint(2), 0);")] + #[doc = concat!("assert_eq!((-7", stringify!($SelfT), ").midpoint(0), -3);")] + #[doc = concat!("assert_eq!(0", stringify!($SelfT), ".midpoint(-7), -3);")] + #[doc = concat!("assert_eq!(0", stringify!($SelfT), ".midpoint(7), 3);")] + /// ``` + #[unstable(feature = "num_midpoint", issue = "110840")] + #[rustc_const_unstable(feature = "const_num_midpoint", issue = "110840")] + #[must_use = "this returns the result of the operation, \ + without modifying the original"] + #[inline] + pub const fn midpoint(self, rhs: Self) -> Self { + // Use the well known branchless algorithm from Hacker's Delight to compute + // `(a + b) / 2` without overflowing: `((a ^ b) >> 1) + (a & b)`. + let t = ((self ^ rhs) >> 1) + (self & rhs); + // Except that it fails for integers whose sum is an odd negative number as + // their floor is one less than their average. So we adjust the result. + t + (if t < 0 { 1 } else { 0 } & (self ^ rhs)) + } + }; ($SelfT:ty, $WideT:ty, unsigned) => { /// Calculates the middle point of `self` and `rhs`. /// @@ -148,6 +182,32 @@ macro_rules! midpoint_impl { ((self as $WideT + rhs as $WideT) / 2) as $SelfT } }; + ($SelfT:ty, $WideT:ty, signed) => { + /// Calculates the middle point of `self` and `rhs`. + /// + /// `midpoint(a, b)` is `(a + b) / 2` as if it were performed in a + /// sufficiently-large signed integral type. This implies that the result is + /// always rounded towards zero and that no overflow will ever occur. + /// + /// # Examples + /// + /// ``` + /// #![feature(num_midpoint)] + #[doc = concat!("assert_eq!(0", stringify!($SelfT), ".midpoint(4), 2);")] + #[doc = concat!("assert_eq!((-1", stringify!($SelfT), ").midpoint(2), 0);")] + #[doc = concat!("assert_eq!((-7", stringify!($SelfT), ").midpoint(0), -3);")] + #[doc = concat!("assert_eq!(0", stringify!($SelfT), ".midpoint(-7), -3);")] + #[doc = concat!("assert_eq!(0", stringify!($SelfT), ".midpoint(7), 3);")] + /// ``` + #[unstable(feature = "num_midpoint", issue = "110840")] + #[rustc_const_unstable(feature = "const_num_midpoint", issue = "110840")] + #[must_use = "this returns the result of the operation, \ + without modifying the original"] + #[inline] + pub const fn midpoint(self, rhs: $SelfT) -> $SelfT { + ((self as $WideT + rhs as $WideT) / 2) as $SelfT + } + }; } macro_rules! widening_impl { @@ -301,6 +361,7 @@ impl i8 { from_xe_bytes_doc = "", bound_condition = "", } + midpoint_impl! { i8, i16, signed } } impl i16 { @@ -324,6 +385,7 @@ impl i16 { from_xe_bytes_doc = "", bound_condition = "", } + midpoint_impl! { i16, i32, signed } } impl i32 { @@ -347,6 +409,7 @@ impl i32 { from_xe_bytes_doc = "", bound_condition = "", } + midpoint_impl! { i32, i64, signed } } impl i64 { @@ -370,6 +433,7 @@ impl i64 { from_xe_bytes_doc = "", bound_condition = "", } + midpoint_impl! { i64, signed } } impl i128 { @@ -395,6 +459,7 @@ impl i128 { from_xe_bytes_doc = "", bound_condition = "", } + midpoint_impl! { i128, signed } } #[cfg(target_pointer_width = "16")] @@ -419,6 +484,7 @@ impl isize { from_xe_bytes_doc = usize_isize_from_xe_bytes_doc!(), bound_condition = " on 16-bit targets", } + midpoint_impl! { isize, i32, signed } } #[cfg(target_pointer_width = "32")] @@ -443,6 +509,7 @@ impl isize { from_xe_bytes_doc = usize_isize_from_xe_bytes_doc!(), bound_condition = " on 32-bit targets", } + midpoint_impl! { isize, i64, signed } } #[cfg(target_pointer_width = "64")] @@ -467,6 +534,7 @@ impl isize { from_xe_bytes_doc = usize_isize_from_xe_bytes_doc!(), bound_condition = " on 64-bit targets", } + midpoint_impl! { isize, signed } } /// If the 6th bit is set ascii is lower case. @@ -618,8 +686,9 @@ impl u8 { /// /// [`to_ascii_uppercase`]: Self::to_ascii_uppercase #[stable(feature = "ascii_methods_on_intrinsics", since = "1.23.0")] + #[rustc_const_stable(feature = "const_make_ascii", since = "CURRENT_RUSTC_VERSION")] #[inline] - pub fn make_ascii_uppercase(&mut self) { + pub const fn make_ascii_uppercase(&mut self) { *self = self.to_ascii_uppercase(); } @@ -643,8 +712,9 @@ impl u8 { /// /// [`to_ascii_lowercase`]: Self::to_ascii_lowercase #[stable(feature = "ascii_methods_on_intrinsics", since = "1.23.0")] + #[rustc_const_stable(feature = "const_make_ascii", since = "CURRENT_RUSTC_VERSION")] #[inline] - pub fn make_ascii_lowercase(&mut self) { + pub const fn make_ascii_lowercase(&mut self) { *self = self.to_ascii_lowercase(); } @@ -1389,7 +1459,7 @@ from_str_radix_int_impl! { isize i8 i16 i32 i64 i128 usize u8 u16 u32 u64 u128 } #[doc(hidden)] #[inline(always)] #[unstable(issue = "none", feature = "std_internals")] -#[rustc_const_stable(feature = "const_int_from_str", since = "1.82.0")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_from_str", since = "1.82.0"))] pub const fn can_not_overflow(radix: u32, is_signed_ty: bool, digits: &[u8]) -> bool { radix <= 16 && digits.len() <= mem::size_of::() * 2 - is_signed_ty as usize } @@ -1408,21 +1478,32 @@ fn from_str_radix_panic_rt(radix: u32) -> ! { #[cfg_attr(feature = "panic_immediate_abort", inline)] #[cold] #[track_caller] +#[rustc_allow_const_fn_unstable(const_eval_select)] const fn from_str_radix_panic(radix: u32) { // The only difference between these two functions is their panic message. intrinsics::const_eval_select((radix,), from_str_radix_panic_ct, from_str_radix_panic_rt); } macro_rules! from_str_radix { - ($($int_ty:ty)+) => {$( + ($signedness:ident $($int_ty:ty)+) => {$( impl $int_ty { /// Converts a string slice in a given base to an integer. /// - /// The string is expected to be an optional `+` sign - /// followed by digits. - /// Leading and trailing whitespace represent an error. - /// Digits are a subset of these characters, depending on `radix`: + /// The string is expected to be an optional + #[doc = sign_dependent_expr!{ + $signedness ? + if signed { + " `+` or `-` " + } + if unsigned { + " `+` " + } + }] + /// sign followed by only digits. Leading and trailing non-digit characters (including + /// whitespace) represent an error. Underscores (which are accepted in rust literals) + /// also represent an error. /// + /// Digits are a subset of these characters, depending on `radix`: /// * `0-9` /// * `a-z` /// * `A-Z` @@ -1434,10 +1515,13 @@ macro_rules! from_str_radix { /// # Examples /// /// Basic usage: - /// /// ``` #[doc = concat!("assert_eq!(", stringify!($int_ty), "::from_str_radix(\"A\", 16), Ok(10));")] /// ``` + /// Trailing space returns error: + /// ``` + #[doc = concat!("assert!(", stringify!($int_ty), "::from_str_radix(\"1 \", 10).is_err());")] + /// ``` #[stable(feature = "rust1", since = "1.0.0")] #[rustc_const_stable(feature = "const_int_from_str", since = "1.82.0")] pub const fn from_str_radix(src: &str, radix: u32) -> Result<$int_ty, ParseIntError> { @@ -1539,20 +1623,31 @@ macro_rules! from_str_radix { )+} } -from_str_radix! { i8 u8 i16 u16 i32 u32 i64 u64 i128 u128 } +from_str_radix! { unsigned u8 u16 u32 u64 u128 } +from_str_radix! { signed i8 i16 i32 i64 i128 } // Re-use the relevant implementation of from_str_radix for isize and usize to avoid outputting two // identical functions. macro_rules! from_str_radix_size_impl { - ($($t:ident $size:ty),*) => {$( + ($($signedness:ident $t:ident $size:ty),*) => {$( impl $size { /// Converts a string slice in a given base to an integer. /// - /// The string is expected to be an optional `+` sign - /// followed by digits. - /// Leading and trailing whitespace represent an error. - /// Digits are a subset of these characters, depending on `radix`: + /// The string is expected to be an optional + #[doc = sign_dependent_expr!{ + $signedness ? + if signed { + " `+` or `-` " + } + if unsigned { + " `+` " + } + }] + /// sign followed by only digits. Leading and trailing non-digit characters (including + /// whitespace) represent an error. Underscores (which are accepted in rust literals) + /// also represent an error. /// + /// Digits are a subset of these characters, depending on `radix`: /// * `0-9` /// * `a-z` /// * `A-Z` @@ -1564,10 +1659,13 @@ macro_rules! from_str_radix_size_impl { /// # Examples /// /// Basic usage: - /// /// ``` #[doc = concat!("assert_eq!(", stringify!($size), "::from_str_radix(\"A\", 16), Ok(10));")] /// ``` + /// Trailing space returns error: + /// ``` + #[doc = concat!("assert!(", stringify!($size), "::from_str_radix(\"1 \", 10).is_err());")] + /// ``` #[stable(feature = "rust1", since = "1.0.0")] #[rustc_const_stable(feature = "const_int_from_str", since = "1.82.0")] pub const fn from_str_radix(src: &str, radix: u32) -> Result<$size, ParseIntError> { @@ -1580,11 +1678,11 @@ macro_rules! from_str_radix_size_impl { } #[cfg(target_pointer_width = "16")] -from_str_radix_size_impl! { i16 isize, u16 usize } +from_str_radix_size_impl! { signed i16 isize, unsigned u16 usize } #[cfg(target_pointer_width = "32")] -from_str_radix_size_impl! { i32 isize, u32 usize } +from_str_radix_size_impl! { signed i32 isize, unsigned u32 usize } #[cfg(target_pointer_width = "64")] -from_str_radix_size_impl! { i64 isize, u64 usize } +from_str_radix_size_impl! { signed i64 isize, unsigned u64 usize } #[cfg(kani)] #[unstable(feature = "kani", issue = "none")] @@ -1657,6 +1755,51 @@ mod verify { } } } + + /// A macro to generate Kani proof harnesses for the `carrying_mul` method, + /// + /// The macro creates multiple harnesses for different ranges of input values, + /// allowing testing of both small and large inputs. + /// + /// # Parameters: + /// - `$type`: The integer type (e.g., u8, u16) for which the `carrying_mul` function is being tested. + /// - `$wide_type`: A wider type to simulate the multiplication (e.g., u16 for u8, u32 for u16). + /// - `$harness_name`: The name of the Kani harness to be generated. + /// - `$min`: The minimum value for the range of inputs for `lhs`, `rhs`, and `carry_in`. + /// - `$max`: The maximum value for the range of inputs for `lhs`, `rhs`, and `carry_in`. + macro_rules! generate_carrying_mul_intervals { + ($type:ty, $wide_type:ty, $($harness_name:ident, $min:expr, $max:expr),+) => { + $( + #[kani::proof] + pub fn $harness_name() { + let lhs: $type = kani::any::<$type>(); + let rhs: $type = kani::any::<$type>(); + let carry_in: $type = kani::any::<$type>(); + + kani::assume(lhs >= $min && lhs <= $max); + kani::assume(rhs >= $min && rhs <= $max); + kani::assume(carry_in >= $min && carry_in <= $max); + + // Perform the carrying multiplication + let (result, carry_out) = lhs.carrying_mul(rhs, carry_in); + + // Manually compute the expected result and carry using wider type + let wide_result = (lhs as $wide_type) + .wrapping_mul(rhs as $wide_type) + .wrapping_add(carry_in as $wide_type); + + let expected_result = wide_result as $type; + let expected_carry = (wide_result >> <$type>::BITS) as $type; + + // Assert the result and carry are correct + assert_eq!(result, expected_result); + assert_eq!(carry_out, expected_carry); + } + )+ + } + } + + // Part 2 : Nested unsafe functions Generation Macros --> https://github.com/verify-rust-std/blob/main/doc/src/challenges/0011-floats-ints.md @@ -1741,7 +1884,7 @@ mod verify { generate_unchecked_neg_harness!(i128, checked_unchecked_neg_i128); generate_unchecked_neg_harness!(isize, checked_unchecked_neg_isize); - // unchecked_mul proofs + // `unchecked_mul` proofs // // Target types: // i{8,16,32,64,128,size} and u{8,16,32,64,128,size} -- 12 types in total, with different interval checks for each. @@ -1892,8 +2035,37 @@ mod verify { generate_unchecked_math_harness!(u128, unchecked_sub, checked_unchecked_sub_u128); generate_unchecked_math_harness!(usize, unchecked_sub, checked_unchecked_sub_usize); + + // Part_2 `carrying_mul` proofs + // + // ====================== u8 Harnesses ====================== + /// Kani proof harness for `carrying_mul` on `u8` type with full range of values. + generate_carrying_mul_intervals!(u8, u16, + carrying_mul_u8_full_range, 0u8, u8::MAX + ); + + // ====================== u16 Harnesses ====================== + /// Kani proof harness for `carrying_mul` on `u16` type with full range of values. + generate_carrying_mul_intervals!(u16, u32, + carrying_mul_u16_full_range, 0u16, u16::MAX + ); + + // ====================== u32 Harnesses ====================== + generate_carrying_mul_intervals!(u32, u64, + carrying_mul_u32_small, 0u32, 10u32, + carrying_mul_u32_large, u32::MAX - 10u32, u32::MAX, + carrying_mul_u32_mid_edge, (u32::MAX / 2) - 10u32, (u32::MAX / 2) + 10u32 + ); + + // ====================== u64 Harnesses ====================== + generate_carrying_mul_intervals!(u64, u128, + carrying_mul_u64_small, 0u64, 10u64, + carrying_mul_u64_large, u64::MAX - 10u64, u64::MAX, + carrying_mul_u64_mid_edge, (u64::MAX / 2) - 10u64, (u64::MAX / 2) + 10u64 + ); + - // Part 2 : Proof harnesses + // Part_2 `widening_mul` proofs // ====================== u8 Harnesses ====================== generate_widening_mul_intervals!(u8, u16, widening_mul_u8, 0u8, u8::MAX); @@ -1919,7 +2091,7 @@ mod verify { widening_mul_u64_mid_edge, (u64::MAX / 2) - 10u64, (u64::MAX / 2) + 10u64 ); - // `wrapping_shl` proofs + // Part_2 `wrapping_shl` proofs // // Target types: // i{8,16,32,64,128,size} and u{8,16,32,64,128,size} -- 12 types in total @@ -1944,7 +2116,7 @@ mod verify { generate_wrapping_shift_harness!(u128, wrapping_shl, checked_wrapping_shl_u128); generate_wrapping_shift_harness!(usize, wrapping_shl, checked_wrapping_shl_usize); - // `wrapping_shr` proofs + // Part_2 `wrapping_shr` proofs // // Target types: // i{8,16,32,64,128,size} and u{8,16,32,64,128,size} -- 12 types in total @@ -1967,4 +2139,5 @@ mod verify { generate_wrapping_shift_harness!(u64, wrapping_shr, checked_wrapping_shr_u64); generate_wrapping_shift_harness!(u128, wrapping_shr, checked_wrapping_shr_u128); generate_wrapping_shift_harness!(usize, wrapping_shr, checked_wrapping_shr_usize); + } diff --git a/library/core/src/num/nonzero.rs b/library/core/src/num/nonzero.rs index 8b888f12da0b1..f04c83693ef63 100644 --- a/library/core/src/num/nonzero.rs +++ b/library/core/src/num/nonzero.rs @@ -355,7 +355,7 @@ where } /// Creates a non-zero without checking whether the value is non-zero. - /// This results in undefined behaviour if the value is zero. + /// This results in undefined behavior if the value is zero. /// /// # Safety /// @@ -952,9 +952,9 @@ macro_rules! nonzero_integer { /// Multiplies two non-zero integers together, /// assuming overflow cannot occur. - /// Overflow is unchecked, and it is undefined behaviour to overflow + /// Overflow is unchecked, and it is undefined behavior to overflow /// *even if the result would wrap to a non-zero value*. - /// The behaviour is undefined as soon as + /// The behavior is undefined as soon as #[doc = sign_dependent_expr!{ $signedness ? if signed { @@ -1323,9 +1323,9 @@ macro_rules! nonzero_integer_signedness_dependent_methods { /// Adds an unsigned integer to a non-zero value, /// assuming overflow cannot occur. - /// Overflow is unchecked, and it is undefined behaviour to overflow + /// Overflow is unchecked, and it is undefined behavior to overflow /// *even if the result would wrap to a non-zero value*. - /// The behaviour is undefined as soon as + /// The behavior is undefined as soon as #[doc = concat!("`self + rhs > ", stringify!($Int), "::MAX`.")] /// /// # Examples @@ -1475,7 +1475,6 @@ macro_rules! nonzero_integer_signedness_dependent_methods { /// ``` #[unstable(feature = "num_midpoint", issue = "110840")] #[rustc_const_unstable(feature = "const_num_midpoint", issue = "110840")] - #[rustc_allow_const_fn_unstable(const_num_midpoint)] #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline] @@ -1527,7 +1526,6 @@ macro_rules! nonzero_integer_signedness_dependent_methods { /// /// Basic usage: /// ``` - /// #![feature(isqrt)] /// # use std::num::NonZero; /// # /// # fn main() { test().unwrap(); } @@ -1539,8 +1537,8 @@ macro_rules! nonzero_integer_signedness_dependent_methods { /// # Some(()) /// # } /// ``` - #[unstable(feature = "isqrt", issue = "116226")] - #[rustc_const_unstable(feature = "isqrt", issue = "116226")] + #[stable(feature = "isqrt", since = "CURRENT_RUSTC_VERSION")] + #[rustc_const_stable(feature = "isqrt", since = "CURRENT_RUSTC_VERSION")] #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline] @@ -1599,7 +1597,7 @@ macro_rules! nonzero_integer_signedness_dependent_methods { /// Computes the absolute value of self. #[doc = concat!("See [`", stringify!($Int), "::abs`]")] - /// for documentation on overflow behaviour. + /// for documentation on overflow behavior. /// /// # Example /// @@ -1878,7 +1876,7 @@ macro_rules! nonzero_integer_signedness_dependent_methods { /// Negates self, overflowing if this is equal to the minimum value. /// #[doc = concat!("See [`", stringify!($Int), "::overflowing_neg`]")] - /// for documentation on overflow behaviour. + /// for documentation on overflow behavior. /// /// # Example /// @@ -1943,7 +1941,7 @@ macro_rules! nonzero_integer_signedness_dependent_methods { /// of the type. /// #[doc = concat!("See [`", stringify!($Int), "::wrapping_neg`]")] - /// for documentation on overflow behaviour. + /// for documentation on overflow behavior. /// /// # Example /// @@ -1972,16 +1970,6 @@ macro_rules! nonzero_integer_signedness_dependent_methods { }; } -// Use this when the generated code should differ between signed and unsigned types. -macro_rules! sign_dependent_expr { - (signed ? if signed { $signed_case:expr } if unsigned { $unsigned_case:expr } ) => { - $signed_case - }; - (unsigned ? if signed { $signed_case:expr } if unsigned { $unsigned_case:expr } ) => { - $unsigned_case - }; -} - nonzero_integer! { Self = NonZeroU8, Primitive = unsigned u8, diff --git a/library/core/src/num/uint_macros.rs b/library/core/src/num/uint_macros.rs index 3220be1d86f30..da5fded23643c 100644 --- a/library/core/src/num/uint_macros.rs +++ b/library/core/src/num/uint_macros.rs @@ -491,7 +491,7 @@ macro_rules! uint_impl { // Per , // LLVM is happy to re-form the intrinsic later if useful. - if unlikely!(intrinsics::add_with_overflow(self, rhs).1) { + if intrinsics::unlikely(intrinsics::add_with_overflow(self, rhs).1) { None } else { // SAFETY: Just checked it doesn't overflow @@ -594,7 +594,7 @@ macro_rules! uint_impl { #[inline] pub const fn checked_add_signed(self, rhs: $SignedT) -> Option { let (a, b) = self.overflowing_add_signed(rhs); - if unlikely!(b) { None } else { Some(a) } + if intrinsics::unlikely(b) { None } else { Some(a) } } /// Strict addition with a signed integer. Computes `self + rhs`, @@ -847,7 +847,7 @@ macro_rules! uint_impl { #[inline] pub const fn checked_mul(self, rhs: Self) -> Option { let (a, b) = self.overflowing_mul(rhs); - if unlikely!(b) { None } else { Some(a) } + if intrinsics::unlikely(b) { None } else { Some(a) } } /// Strict integer multiplication. Computes `self * rhs`, panicking if @@ -943,7 +943,7 @@ macro_rules! uint_impl { without modifying the original"] #[inline] pub const fn checked_div(self, rhs: Self) -> Option { - if unlikely!(rhs == 0) { + if intrinsics::unlikely(rhs == 0) { None } else { // SAFETY: div by zero has been checked above and unsigned types have no other @@ -1004,7 +1004,7 @@ macro_rules! uint_impl { without modifying the original"] #[inline] pub const fn checked_div_euclid(self, rhs: Self) -> Option { - if unlikely!(rhs == 0) { + if intrinsics::unlikely(rhs == 0) { None } else { Some(self.div_euclid(rhs)) @@ -1064,7 +1064,7 @@ macro_rules! uint_impl { without modifying the original"] #[inline] pub const fn checked_rem(self, rhs: Self) -> Option { - if unlikely!(rhs == 0) { + if intrinsics::unlikely(rhs == 0) { None } else { // SAFETY: div by zero has been checked above and unsigned types have no other @@ -1126,7 +1126,7 @@ macro_rules! uint_impl { without modifying the original"] #[inline] pub const fn checked_rem_euclid(self, rhs: Self) -> Option { - if unlikely!(rhs == 0) { + if intrinsics::unlikely(rhs == 0) { None } else { Some(self.rem_euclid(rhs)) @@ -1365,7 +1365,7 @@ macro_rules! uint_impl { #[inline] pub const fn checked_neg(self) -> Option { let (a, b) = self.overflowing_neg(); - if unlikely!(b) { None } else { Some(a) } + if intrinsics::unlikely(b) { None } else { Some(a) } } /// Strict negation. Computes `-self`, panicking unless `self == @@ -1419,8 +1419,7 @@ macro_rules! uint_impl { /// ``` #[stable(feature = "wrapping", since = "1.7.0")] #[rustc_const_stable(feature = "const_checked_int_methods", since = "1.47.0")] - // We could always go back to wrapping - #[rustc_allow_const_fn_unstable(unchecked_shifts)] + #[cfg_attr(bootstrap, rustc_allow_const_fn_unstable(unchecked_shifts))] #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline] @@ -1486,7 +1485,7 @@ macro_rules! uint_impl { )] #[must_use = "this returns the result of the operation, \ without modifying the original"] - #[rustc_const_unstable(feature = "unchecked_shifts", issue = "85122")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "unchecked_shifts", issue = "85122"))] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[requires(rhs < <$ActualT>::BITS)] @@ -1546,8 +1545,7 @@ macro_rules! uint_impl { /// ``` #[stable(feature = "wrapping", since = "1.7.0")] #[rustc_const_stable(feature = "const_checked_int_methods", since = "1.47.0")] - // We could always go back to wrapping - #[rustc_allow_const_fn_unstable(unchecked_shifts)] + #[cfg_attr(bootstrap, rustc_allow_const_fn_unstable(unchecked_shifts))] #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline] @@ -1613,7 +1611,7 @@ macro_rules! uint_impl { )] #[must_use = "this returns the result of the operation, \ without modifying the original"] - #[rustc_const_unstable(feature = "unchecked_shifts", issue = "85122")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "unchecked_shifts", issue = "85122"))] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[requires(rhs < <$ActualT>::BITS)]// i.e. requires the right hand side of the shift (rhs) to be less than the number of bits in the type. This prevents undefined behavior. @@ -2137,7 +2135,7 @@ macro_rules! uint_impl { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline(always)] - #[rustc_allow_const_fn_unstable(unchecked_shifts)] + #[cfg_attr(bootstrap, rustc_allow_const_fn_unstable(unchecked_shifts))] #[ensures(|result| *result == self << (rhs & (Self::BITS - 1)))] pub const fn wrapping_shl(self, rhs: u32) -> Self { // SAFETY: the masking by the bitsize of the type ensures that we do not shift @@ -2171,7 +2169,7 @@ macro_rules! uint_impl { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline(always)] - #[rustc_allow_const_fn_unstable(unchecked_shifts)] + #[cfg_attr(bootstrap, rustc_allow_const_fn_unstable(unchecked_shifts))] #[ensures(|result| *result == self >> (rhs & (Self::BITS - 1)))] pub const fn wrapping_shr(self, rhs: u32) -> Self { // SAFETY: the masking by the bitsize of the type ensures that we do not shift @@ -2760,11 +2758,10 @@ macro_rules! uint_impl { /// /// Basic usage: /// ``` - /// #![feature(isqrt)] #[doc = concat!("assert_eq!(10", stringify!($SelfT), ".isqrt(), 3);")] /// ``` - #[unstable(feature = "isqrt", issue = "116226")] - #[rustc_const_unstable(feature = "isqrt", issue = "116226")] + #[stable(feature = "isqrt", since = "CURRENT_RUSTC_VERSION")] + #[rustc_const_stable(feature = "isqrt", since = "CURRENT_RUSTC_VERSION")] #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline] @@ -3016,7 +3013,7 @@ macro_rules! uint_impl { // overflow cases it instead ends up returning the maximum value // of the type, and can return 0 for 0. #[inline] - #[rustc_const_stable(feature = "const_int_pow", since = "1.50.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_int_pow", since = "1.50.0"))] const fn one_less_than_next_power_of_two(self) -> Self { if self <= 1 { return 0; } @@ -3093,7 +3090,7 @@ macro_rules! uint_impl { /// ``` #[inline] #[unstable(feature = "wrapping_next_power_of_two", issue = "32463", - reason = "needs decision on wrapping behaviour")] + reason = "needs decision on wrapping behavior")] #[rustc_const_unstable(feature = "wrapping_next_power_of_two", issue = "32463")] #[must_use = "this returns the result of the operation, \ without modifying the original"] diff --git a/library/core/src/num/wrapping.rs b/library/core/src/num/wrapping.rs index 1ac6d3161c2f9..1156b389e2867 100644 --- a/library/core/src/num/wrapping.rs +++ b/library/core/src/num/wrapping.rs @@ -1043,7 +1043,7 @@ macro_rules! wrapping_int_impl_unsigned { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[unstable(feature = "wrapping_next_power_of_two", issue = "32463", - reason = "needs decision on wrapping behaviour")] + reason = "needs decision on wrapping behavior")] pub fn next_power_of_two(self) -> Self { Wrapping(self.0.wrapping_next_power_of_two()) } diff --git a/library/core/src/ops/async_function.rs b/library/core/src/ops/async_function.rs index 37fac2b126fac..4b230b15a1e6f 100644 --- a/library/core/src/ops/async_function.rs +++ b/library/core/src/ops/async_function.rs @@ -79,7 +79,10 @@ mod impls { where F: AsyncFn, { - type CallRefFuture<'a> = F::CallRefFuture<'a> where Self: 'a; + type CallRefFuture<'a> + = F::CallRefFuture<'a> + where + Self: 'a; extern "rust-call" fn async_call_mut(&mut self, args: A) -> Self::CallRefFuture<'_> { F::async_call(*self, args) @@ -104,7 +107,10 @@ mod impls { where F: AsyncFnMut, { - type CallRefFuture<'a> = F::CallRefFuture<'a> where Self: 'a; + type CallRefFuture<'a> + = F::CallRefFuture<'a> + where + Self: 'a; extern "rust-call" fn async_call_mut(&mut self, args: A) -> Self::CallRefFuture<'_> { F::async_call_mut(*self, args) diff --git a/library/core/src/ops/control_flow.rs b/library/core/src/ops/control_flow.rs index ab73dc19fcc73..55deabbee8fb5 100644 --- a/library/core/src/ops/control_flow.rs +++ b/library/core/src/ops/control_flow.rs @@ -171,14 +171,13 @@ impl ControlFlow { /// # Examples /// /// ``` - /// #![feature(control_flow_enum)] /// use std::ops::ControlFlow; /// /// assert_eq!(ControlFlow::::Break(3).break_value(), Some(3)); /// assert_eq!(ControlFlow::::Continue(3).break_value(), None); /// ``` #[inline] - #[unstable(feature = "control_flow_enum", reason = "new API", issue = "75744")] + #[stable(feature = "control_flow_enum", since = "1.83.0")] pub fn break_value(self) -> Option { match self { ControlFlow::Continue(..) => None, @@ -189,11 +188,8 @@ impl ControlFlow { /// Maps `ControlFlow` to `ControlFlow` by applying a function /// to the break value in case it exists. #[inline] - #[unstable(feature = "control_flow_enum", reason = "new API", issue = "75744")] - pub fn map_break(self, f: F) -> ControlFlow - where - F: FnOnce(B) -> T, - { + #[stable(feature = "control_flow_enum", since = "1.83.0")] + pub fn map_break(self, f: impl FnOnce(B) -> T) -> ControlFlow { match self { ControlFlow::Continue(x) => ControlFlow::Continue(x), ControlFlow::Break(x) => ControlFlow::Break(f(x)), @@ -206,14 +202,13 @@ impl ControlFlow { /// # Examples /// /// ``` - /// #![feature(control_flow_enum)] /// use std::ops::ControlFlow; /// /// assert_eq!(ControlFlow::::Break(3).continue_value(), None); /// assert_eq!(ControlFlow::::Continue(3).continue_value(), Some(3)); /// ``` #[inline] - #[unstable(feature = "control_flow_enum", reason = "new API", issue = "75744")] + #[stable(feature = "control_flow_enum", since = "1.83.0")] pub fn continue_value(self) -> Option { match self { ControlFlow::Continue(x) => Some(x), @@ -224,11 +219,8 @@ impl ControlFlow { /// Maps `ControlFlow` to `ControlFlow` by applying a function /// to the continue value in case it exists. #[inline] - #[unstable(feature = "control_flow_enum", reason = "new API", issue = "75744")] - pub fn map_continue(self, f: F) -> ControlFlow - where - F: FnOnce(C) -> T, - { + #[stable(feature = "control_flow_enum", since = "1.83.0")] + pub fn map_continue(self, f: impl FnOnce(C) -> T) -> ControlFlow { match self { ControlFlow::Continue(x) => ControlFlow::Continue(f(x)), ControlFlow::Break(x) => ControlFlow::Break(x), diff --git a/library/core/src/ops/deref.rs b/library/core/src/ops/deref.rs index f0d2c761ef35b..1ef9990c00af8 100644 --- a/library/core/src/ops/deref.rs +++ b/library/core/src/ops/deref.rs @@ -15,7 +15,7 @@ /// /// Types that implement `Deref` or `DerefMut` are often called "smart /// pointers" and the mechanism of deref coercion has been specifically designed -/// to facilitate the pointer-like behaviour that name suggests. Often, the +/// to facilitate the pointer-like behavior that name suggests. Often, the /// purpose of a "smart pointer" type is to change the ownership semantics /// of a contained value (for example, [`Rc`][rc] or [`Cow`][cow]) or the /// storage semantics of a contained value (for example, [`Box`][box]). @@ -42,7 +42,7 @@ /// 1. a value of the type transparently behaves like a value of the target /// type; /// 1. the implementation of the deref function is cheap; and -/// 1. users of the type will not be surprised by any deref coercion behaviour. +/// 1. users of the type will not be surprised by any deref coercion behavior. /// /// In general, deref traits **should not** be implemented if: /// @@ -185,7 +185,7 @@ impl Deref for &mut T { /// /// Types that implement `DerefMut` or `Deref` are often called "smart /// pointers" and the mechanism of deref coercion has been specifically designed -/// to facilitate the pointer-like behaviour that name suggests. Often, the +/// to facilitate the pointer-like behavior that name suggests. Often, the /// purpose of a "smart pointer" type is to change the ownership semantics /// of a contained value (for example, [`Rc`][rc] or [`Cow`][cow]) or the /// storage semantics of a contained value (for example, [`Box`][box]). @@ -297,15 +297,21 @@ unsafe impl DerefPure for &mut T {} /// Indicates that a struct can be used as a method receiver, without the /// `arbitrary_self_types` feature. This is implemented by stdlib pointer types like `Box`, /// `Rc`, `&T`, and `Pin

`. -#[lang = "receiver"] -#[unstable(feature = "receiver_trait", issue = "none")] +/// +/// This trait will shortly be removed and replaced with a more generic +/// facility based around the current "arbitrary self types" unstable feature. +/// That new facility will use a replacement trait called `Receiver` which is +/// why this is now named `LegacyReceiver`. +#[cfg_attr(bootstrap, lang = "receiver")] +#[cfg_attr(not(bootstrap), lang = "legacy_receiver")] +#[unstable(feature = "legacy_receiver_trait", issue = "none")] #[doc(hidden)] -pub trait Receiver { +pub trait LegacyReceiver { // Empty. } -#[unstable(feature = "receiver_trait", issue = "none")] -impl Receiver for &T {} +#[unstable(feature = "legacy_receiver_trait", issue = "none")] +impl LegacyReceiver for &T {} -#[unstable(feature = "receiver_trait", issue = "none")] -impl Receiver for &mut T {} +#[unstable(feature = "legacy_receiver_trait", issue = "none")] +impl LegacyReceiver for &mut T {} diff --git a/library/core/src/ops/index_range.rs b/library/core/src/ops/index_range.rs index 64214eae377dd..dce3514a1595b 100644 --- a/library/core/src/ops/index_range.rs +++ b/library/core/src/ops/index_range.rs @@ -45,7 +45,8 @@ impl IndexRange { #[inline] pub const fn len(&self) -> usize { // SAFETY: By invariant, this cannot wrap - unsafe { self.end.unchecked_sub(self.start) } + // Using the intrinsic because a UB check here impedes LLVM optimization. (#131563) + unsafe { crate::intrinsics::unchecked_sub(self.end, self.start) } } /// # Safety @@ -82,7 +83,8 @@ impl IndexRange { let mid = if n <= self.len() { // SAFETY: We just checked that this will be between start and end, // and thus the addition cannot overflow. - unsafe { self.start.unchecked_add(n) } + // Using the intrinsic avoids a superfluous UB check. + unsafe { crate::intrinsics::unchecked_add(self.start, n) } } else { self.end }; @@ -100,8 +102,9 @@ impl IndexRange { pub fn take_suffix(&mut self, n: usize) -> Self { let mid = if n <= self.len() { // SAFETY: We just checked that this will be between start and end, - // and thus the addition cannot overflow. - unsafe { self.end.unchecked_sub(n) } + // and thus the subtraction cannot overflow. + // Using the intrinsic avoids a superfluous UB check. + unsafe { crate::intrinsics::unchecked_sub(self.end, n) } } else { self.start }; diff --git a/library/core/src/ops/mod.rs b/library/core/src/ops/mod.rs index 98d41b71e8eb8..c9f47e5daadd6 100644 --- a/library/core/src/ops/mod.rs +++ b/library/core/src/ops/mod.rs @@ -162,19 +162,19 @@ pub use self::async_function::{AsyncFn, AsyncFnMut, AsyncFnOnce}; pub use self::bit::{BitAnd, BitOr, BitXor, Not, Shl, Shr}; #[stable(feature = "op_assign_traits", since = "1.8.0")] pub use self::bit::{BitAndAssign, BitOrAssign, BitXorAssign, ShlAssign, ShrAssign}; -#[unstable(feature = "control_flow_enum", reason = "new API", issue = "75744")] +#[stable(feature = "control_flow_enum_type", since = "1.55.0")] pub use self::control_flow::ControlFlow; #[unstable(feature = "coroutine_trait", issue = "43122")] pub use self::coroutine::{Coroutine, CoroutineState}; #[unstable(feature = "deref_pure_trait", issue = "87121")] pub use self::deref::DerefPure; -#[unstable(feature = "receiver_trait", issue = "none")] -pub use self::deref::Receiver; +#[unstable(feature = "legacy_receiver_trait", issue = "none")] +pub use self::deref::LegacyReceiver; #[stable(feature = "rust1", since = "1.0.0")] pub use self::deref::{Deref, DerefMut}; -pub(crate) use self::drop::fallback_surface_drop; #[stable(feature = "rust1", since = "1.0.0")] pub use self::drop::Drop; +pub(crate) use self::drop::fallback_surface_drop; #[stable(feature = "rust1", since = "1.0.0")] pub use self::function::{Fn, FnMut, FnOnce}; #[stable(feature = "rust1", since = "1.0.0")] diff --git a/library/core/src/ops/unsize.rs b/library/core/src/ops/unsize.rs index b51f12580ea4f..d2a07197f6f6a 100644 --- a/library/core/src/ops/unsize.rs +++ b/library/core/src/ops/unsize.rs @@ -68,8 +68,8 @@ impl, U: ?Sized> CoerceUnsized<*const U> for *mut T {} #[unstable(feature = "coerce_unsized", issue = "18598")] impl, U: ?Sized> CoerceUnsized<*const U> for *const T {} -/// `DispatchFromDyn` is used in the implementation of object safety checks (specifically allowing -/// arbitrary self types), to guarantee that a method's receiver type can be dispatched on. +/// `DispatchFromDyn` is used in the implementation of dyn-compatibility[^1] checks (specifically +/// allowing arbitrary self types), to guarantee that a method's receiver type can be dispatched on. /// /// Note: `DispatchFromDyn` was briefly named `CoerceSized` (and had a slightly different /// interpretation). @@ -80,7 +80,7 @@ impl, U: ?Sized> CoerceUnsized<*const U> for *const T {} /// type). The compiler must generate an implicit conversion from the trait object/wide pointer to /// the concrete reference/narrow pointer. Implementing `DispatchFromDyn` indicates that that /// conversion is allowed and thus that the type implementing `DispatchFromDyn` is safe to use as -/// the self type in an object-safe method. (in the above example, the compiler will require +/// the self type in an dyn-compatible method. (in the above example, the compiler will require /// `DispatchFromDyn` is implemented for `&'a U`). /// /// `DispatchFromDyn` does not specify the conversion from wide pointer to narrow pointer; the @@ -112,6 +112,8 @@ impl, U: ?Sized> CoerceUnsized<*const U> for *const T {} /// T: Unsize, /// {} /// ``` +/// +/// [^1]: Formerly known as *object safety*. #[unstable(feature = "dispatch_from_dyn", issue = "none")] #[lang = "dispatch_from_dyn"] pub trait DispatchFromDyn { diff --git a/library/core/src/option.rs b/library/core/src/option.rs index 212e4f0215463..2aa4f1723680f 100644 --- a/library/core/src/option.rs +++ b/library/core/src/option.rs @@ -150,7 +150,7 @@ //! It is further guaranteed that, for the cases above, one can //! [`mem::transmute`] from all valid values of `T` to `Option` and //! from `Some::(_)` to `T` (but transmuting `None::` to `T` -//! is undefined behaviour). +//! is undefined behavior). //! //! # Method overview //! @@ -723,7 +723,7 @@ impl Option { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_option", issue = "67441")] + #[rustc_const_stable(feature = "const_option", since = "1.83.0")] pub const fn as_mut(&mut self) -> Option<&mut T> { match *self { Some(ref mut x) => Some(x), @@ -739,6 +739,7 @@ impl Option { #[stable(feature = "pin", since = "1.33.0")] #[rustc_const_unstable(feature = "const_option_ext", issue = "91930")] pub const fn as_pin_ref(self: Pin<&Self>) -> Option> { + // FIXME(const-hack): use `map` once that is possible match Pin::get_ref(self).as_ref() { // SAFETY: `x` is guaranteed to be pinned because it comes from `self` // which is pinned. @@ -758,6 +759,7 @@ impl Option { // SAFETY: `get_unchecked_mut` is never used to move the `Option` inside `self`. // `x` is guaranteed to be pinned because it comes from `self` which is pinned. unsafe { + // FIXME(const-hack): use `map` once that is possible match Pin::get_unchecked_mut(self).as_mut() { Some(x) => Some(Pin::new_unchecked(x)), None => None, @@ -921,7 +923,9 @@ impl Option { #[inline] #[track_caller] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_option", issue = "67441")] + #[cfg_attr(not(test), rustc_diagnostic_item = "option_expect")] + #[rustc_allow_const_fn_unstable(const_precise_live_drops)] + #[rustc_const_stable(feature = "const_option", since = "1.83.0")] pub const fn expect(self, msg: &str) -> T { match self { Some(val) => val, @@ -958,7 +962,9 @@ impl Option { #[inline(always)] #[track_caller] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_option", issue = "67441")] + #[cfg_attr(not(test), rustc_diagnostic_item = "option_unwrap")] + #[rustc_allow_const_fn_unstable(const_precise_live_drops)] + #[rustc_const_stable(feature = "const_option", since = "1.83.0")] pub const fn unwrap(self) -> T { match self { Some(val) => val, @@ -1065,7 +1071,8 @@ impl Option { #[inline] #[track_caller] #[stable(feature = "option_result_unwrap_unchecked", since = "1.58.0")] - #[rustc_const_unstable(feature = "const_option_ext", issue = "91930")] + #[rustc_allow_const_fn_unstable(const_precise_live_drops)] + #[rustc_const_stable(feature = "const_option", since = "1.83.0")] pub const unsafe fn unwrap_unchecked(self) -> T { match self { Some(val) => val, @@ -1290,10 +1297,7 @@ impl Option { where T: Deref, { - match self.as_ref() { - Some(t) => Some(t.deref()), - None => None, - } + self.as_ref().map(|t| t.deref()) } /// Converts from `Option` (or `&mut Option`) to `Option<&mut T::Target>`. @@ -1316,10 +1320,7 @@ impl Option { where T: DerefMut, { - match self.as_mut() { - Some(t) => Some(t.deref_mut()), - None => None, - } + self.as_mut().map(|t| t.deref_mut()) } ///////////////////////////////////////////////////////////////////////// @@ -1338,9 +1339,8 @@ impl Option { /// assert_eq!(x.iter().next(), None); /// ``` #[inline] - #[rustc_const_unstable(feature = "const_option", issue = "67441")] #[stable(feature = "rust1", since = "1.0.0")] - pub const fn iter(&self) -> Iter<'_, T> { + pub fn iter(&self) -> Iter<'_, T> { Iter { inner: Item { opt: self.as_ref() } } } @@ -1633,13 +1633,7 @@ impl Option { #[inline] #[stable(feature = "option_entry", since = "1.20.0")] pub fn get_or_insert(&mut self, value: T) -> &mut T { - if let None = *self { - *self = Some(value); - } - - // SAFETY: a `None` variant for `self` would have been replaced by a `Some` - // variant in the code above. - unsafe { self.as_mut().unwrap_unchecked() } + self.get_or_insert_with(|| value) } /// Inserts the default value into the option if it is [`None`], then @@ -1648,8 +1642,6 @@ impl Option { /// # Examples /// /// ``` - /// #![feature(option_get_or_insert_default)] - /// /// let mut x = None; /// /// { @@ -1662,7 +1654,7 @@ impl Option { /// assert_eq!(x, Some(7)); /// ``` #[inline] - #[unstable(feature = "option_get_or_insert_default", issue = "82901")] + #[stable(feature = "option_get_or_insert_default", since = "1.83.0")] pub fn get_or_insert_default(&mut self) -> &mut T where T: Default, @@ -1723,9 +1715,9 @@ impl Option { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_option", issue = "67441")] + #[rustc_const_stable(feature = "const_option", since = "1.83.0")] pub const fn take(&mut self) -> Option { - // FIXME replace `mem::replace` by `mem::take` when the latter is const ready + // FIXME(const-hack) replace `mem::replace` by `mem::take` when the latter is const ready mem::replace(self, None) } @@ -1780,8 +1772,8 @@ impl Option { /// assert_eq!(old, None); /// ``` #[inline] - #[rustc_const_unstable(feature = "const_option", issue = "67441")] #[stable(feature = "option_replace", since = "1.31.0")] + #[rustc_const_stable(feature = "const_option", since = "1.83.0")] pub const fn replace(&mut self, value: T) -> Option { mem::replace(self, Some(value)) } @@ -1889,12 +1881,12 @@ impl Option<&T> { /// ``` #[must_use = "`self` will be dropped if the result is not used"] #[stable(feature = "copied", since = "1.35.0")] - #[rustc_const_unstable(feature = "const_option", issue = "67441")] + #[rustc_const_stable(feature = "const_option", since = "1.83.0")] pub const fn copied(self) -> Option where T: Copy, { - // FIXME: this implementation, which sidesteps using `Option::map` since it's not const + // FIXME(const-hack): this implementation, which sidesteps using `Option::map` since it's not const // ready yet, should be reverted when possible to avoid code repetition match self { Some(&v) => Some(v), @@ -1942,7 +1934,7 @@ impl Option<&mut T> { /// ``` #[must_use = "`self` will be dropped if the result is not used"] #[stable(feature = "copied", since = "1.35.0")] - #[rustc_const_unstable(feature = "const_option_ext", issue = "91930")] + #[rustc_const_stable(feature = "const_option", since = "1.83.0")] pub const fn copied(self) -> Option where T: Copy, @@ -1997,7 +1989,8 @@ impl Option> { /// ``` #[inline] #[stable(feature = "transpose_result", since = "1.33.0")] - #[rustc_const_unstable(feature = "const_option", issue = "67441")] + #[rustc_allow_const_fn_unstable(const_precise_live_drops)] + #[rustc_const_stable(feature = "const_option", since = "1.83.0")] pub const fn transpose(self) -> Result, E> { match self { Some(Ok(x)) => Ok(Some(x)), @@ -2020,7 +2013,6 @@ const fn unwrap_failed() -> ! { #[cfg_attr(feature = "panic_immediate_abort", inline)] #[cold] #[track_caller] -#[rustc_const_unstable(feature = "const_option", issue = "67441")] const fn expect_failed(msg: &str) -> ! { panic_display(&msg) } @@ -2545,11 +2537,37 @@ impl Option> { /// ``` #[inline] #[stable(feature = "option_flattening", since = "1.40.0")] - #[rustc_const_unstable(feature = "const_option", issue = "67441")] + #[rustc_allow_const_fn_unstable(const_precise_live_drops)] + #[rustc_const_stable(feature = "const_option", since = "1.83.0")] pub const fn flatten(self) -> Option { + // FIXME(const-hack): could be written with `and_then` match self { Some(inner) => inner, None => None, } } } + +impl [Option; N] { + /// Transposes a `[Option; N]` into a `Option<[T; N]>`. + /// + /// # Examples + /// + /// ``` + /// #![feature(option_array_transpose)] + /// # use std::option::Option; + /// + /// let data = [Some(0); 1000]; + /// let data: Option<[u8; 1000]> = data.transpose(); + /// assert_eq!(data, Some([0; 1000])); + /// + /// let data = [Some(0), None]; + /// let data: Option<[u8; 2]> = data.transpose(); + /// assert_eq!(data, None); + /// ``` + #[inline] + #[unstable(feature = "option_array_transpose", issue = "130828")] + pub fn transpose(self) -> Option<[T; N]> { + self.try_map(core::convert::identity) + } +} diff --git a/library/core/src/panic.rs b/library/core/src/panic.rs index 6c5236ed99ce8..c95a000561c35 100644 --- a/library/core/src/panic.rs +++ b/library/core/src/panic.rs @@ -140,6 +140,31 @@ pub macro unreachable_2021 { ), } +/// Invokes a closure, aborting if the closure unwinds. +/// +/// When compiled with aborting panics, this function is effectively a no-op. +/// With unwinding panics, an unwind results in another call into the panic +/// hook followed by a process abort. +/// +/// # Notes +/// +/// Instead of using this function, code should attempt to support unwinding. +/// Implementing [`Drop`] allows you to restore invariants uniformly in both +/// return and unwind paths. +/// +/// If an unwind can lead to logical issues but not soundness issues, you +/// should allow the unwind. Opting out of [`UnwindSafe`] indicates to your +/// consumers that they need to consider correctness in the face of unwinds. +/// +/// If an unwind would be unsound, then this function should be used in order +/// to prevent unwinds. However, note that `extern "C" fn` will automatically +/// convert unwinds to aborts, so using this function isn't necessary for FFI. +#[unstable(feature = "abort_unwind", issue = "130338")] +#[rustc_nounwind] +pub fn abort_unwind R, R>(f: F) -> R { + f() +} + /// An internal trait used by std to pass data from std to `panic_unwind` and /// other panic runtimes. Not intended to be stabilized any time soon, do not /// use. diff --git a/library/core/src/panic/location.rs b/library/core/src/panic/location.rs index e2a842046a96d..1ad5c07d15cd0 100644 --- a/library/core/src/panic/location.rs +++ b/library/core/src/panic/location.rs @@ -44,7 +44,7 @@ impl<'a> Location<'a> { /// /// # Examples /// - /// ```standalone + /// ```standalone_crate /// use std::panic::Location; /// /// /// Returns the [`Location`] at which it is called. diff --git a/library/core/src/panic/panic_info.rs b/library/core/src/panic/panic_info.rs index e4d0c897b65ca..1d950eb362504 100644 --- a/library/core/src/panic/panic_info.rs +++ b/library/core/src/panic/panic_info.rs @@ -12,7 +12,7 @@ use crate::panic::Location; #[stable(feature = "panic_hooks", since = "1.10.0")] #[derive(Debug)] pub struct PanicInfo<'a> { - message: fmt::Arguments<'a>, + message: &'a fmt::Arguments<'a>, location: &'a Location<'a>, can_unwind: bool, force_no_backtrace: bool, @@ -26,13 +26,13 @@ pub struct PanicInfo<'a> { /// See [`PanicInfo::message`]. #[stable(feature = "panic_info_message", since = "1.81.0")] pub struct PanicMessage<'a> { - message: fmt::Arguments<'a>, + message: &'a fmt::Arguments<'a>, } impl<'a> PanicInfo<'a> { #[inline] pub(crate) fn new( - message: fmt::Arguments<'a>, + message: &'a fmt::Arguments<'a>, location: &'a Location<'a>, can_unwind: bool, force_no_backtrace: bool, @@ -146,7 +146,7 @@ impl Display for PanicInfo<'_> { formatter.write_str("panicked at ")?; self.location.fmt(formatter)?; formatter.write_str(":\n")?; - formatter.write_fmt(self.message)?; + formatter.write_fmt(*self.message)?; Ok(()) } } @@ -168,6 +168,7 @@ impl<'a> PanicMessage<'a> { #[rustc_const_unstable(feature = "const_arguments_as_str", issue = "103900")] #[must_use] #[inline] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] pub const fn as_str(&self) -> Option<&'static str> { self.message.as_str() } @@ -177,7 +178,7 @@ impl<'a> PanicMessage<'a> { impl Display for PanicMessage<'_> { #[inline] fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - formatter.write_fmt(self.message) + formatter.write_fmt(*self.message) } } @@ -185,6 +186,6 @@ impl Display for PanicMessage<'_> { impl fmt::Debug for PanicMessage<'_> { #[inline] fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - formatter.write_fmt(self.message) + formatter.write_fmt(*self.message) } } diff --git a/library/core/src/panicking.rs b/library/core/src/panicking.rs index e4a623040871a..9071d6719a30e 100644 --- a/library/core/src/panicking.rs +++ b/library/core/src/panicking.rs @@ -50,7 +50,8 @@ const _: () = assert!(cfg!(panic = "abort"), "panic_immediate_abort requires -C #[track_caller] #[lang = "panic_fmt"] // needed for const-evaluated panics #[rustc_do_not_const_check] // hooked by const-eval -#[rustc_const_unstable(feature = "panic_internals", issue = "none")] +#[cfg_attr(bootstrap, rustc_const_unstable(feature = "panic_internals", issue = "none"))] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] // must follow stable const rules since it is exposed to stable pub const fn panic_fmt(fmt: fmt::Arguments<'_>) -> ! { if cfg!(feature = "panic_immediate_abort") { super::intrinsics::abort() @@ -64,7 +65,7 @@ pub const fn panic_fmt(fmt: fmt::Arguments<'_>) -> ! { } let pi = PanicInfo::new( - fmt, + &fmt, Location::caller(), /* can_unwind */ true, /* force_no_backtrace */ false, @@ -84,7 +85,9 @@ pub const fn panic_fmt(fmt: fmt::Arguments<'_>) -> ! { // and unwinds anyway, we will hit the "unwinding out of nounwind function" guard, // which causes a "panic in a function that cannot unwind". #[rustc_nounwind] -#[rustc_const_unstable(feature = "panic_internals", issue = "none")] +#[cfg_attr(bootstrap, rustc_const_unstable(feature = "panic_internals", issue = "none"))] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] // must follow stable const rules since it is exposed to stable +#[rustc_allow_const_fn_unstable(const_eval_select)] pub const fn panic_nounwind_fmt(fmt: fmt::Arguments<'_>, force_no_backtrace: bool) -> ! { #[inline] // this should always be inlined into `panic_nounwind_fmt` #[track_caller] @@ -102,7 +105,7 @@ pub const fn panic_nounwind_fmt(fmt: fmt::Arguments<'_>, force_no_backtrace: boo // PanicInfo with the `can_unwind` flag set to false forces an abort. let pi = PanicInfo::new( - fmt, + &fmt, Location::caller(), /* can_unwind */ false, force_no_backtrace, @@ -131,7 +134,8 @@ pub const fn panic_nounwind_fmt(fmt: fmt::Arguments<'_>, force_no_backtrace: boo #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never), cold)] #[cfg_attr(feature = "panic_immediate_abort", inline)] #[track_caller] -#[rustc_const_unstable(feature = "panic_internals", issue = "none")] +#[cfg_attr(bootstrap, rustc_const_unstable(feature = "panic_internals", issue = "none"))] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] // must follow stable const rules since it is exposed to stable #[lang = "panic"] // used by lints and miri for panics pub const fn panic(expr: &'static str) -> ! { // Use Arguments::new_const instead of format_args!("{expr}") to potentially @@ -169,7 +173,8 @@ macro_rules! panic_const { #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never), cold)] #[cfg_attr(feature = "panic_immediate_abort", inline)] #[track_caller] - #[rustc_const_unstable(feature = "panic_internals", issue = "none")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "panic_internals", issue = "none"))] + #[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] // must follow stable const rules since it is exposed to stable #[lang = stringify!($lang)] pub const fn $lang() -> ! { // Use Arguments::new_const instead of format_args!("{expr}") to potentially @@ -216,7 +221,8 @@ panic_const! { #[cfg_attr(feature = "panic_immediate_abort", inline)] #[lang = "panic_nounwind"] // needed by codegen for non-unwinding panics #[rustc_nounwind] -#[rustc_const_unstable(feature = "panic_internals", issue = "none")] +#[cfg_attr(bootstrap, rustc_const_unstable(feature = "panic_internals", issue = "none"))] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] // must follow stable const rules since it is exposed to stable pub const fn panic_nounwind(expr: &'static str) -> ! { panic_nounwind_fmt(fmt::Arguments::new_const(&[expr]), /* force_no_backtrace */ false); } @@ -232,7 +238,8 @@ pub fn panic_nounwind_nobacktrace(expr: &'static str) -> ! { #[track_caller] #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never), cold)] #[cfg_attr(feature = "panic_immediate_abort", inline)] -#[rustc_const_unstable(feature = "panic_internals", issue = "none")] +#[cfg_attr(bootstrap, rustc_const_unstable(feature = "panic_internals", issue = "none"))] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] // must follow stable const rules since it is exposed to stable pub const fn panic_explicit() -> ! { panic_display(&"explicit panic"); } @@ -249,7 +256,8 @@ pub fn unreachable_display(x: &T) -> ! { #[inline] #[track_caller] #[rustc_diagnostic_item = "panic_str_2015"] -#[rustc_const_unstable(feature = "panic_internals", issue = "none")] +#[cfg_attr(bootstrap, rustc_const_unstable(feature = "panic_internals", issue = "none"))] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] // must follow stable const rules since it is exposed to stable pub const fn panic_str_2015(expr: &str) -> ! { panic_display(&expr); } @@ -259,7 +267,8 @@ pub const fn panic_str_2015(expr: &str) -> ! { #[rustc_do_not_const_check] // hooked by const-eval // enforce a &&str argument in const-check and hook this by const-eval #[rustc_const_panic_str] -#[rustc_const_unstable(feature = "panic_internals", issue = "none")] +#[cfg_attr(bootstrap, rustc_const_unstable(feature = "panic_internals", issue = "none"))] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] // must follow stable const rules since it is exposed to stable pub const fn panic_display(x: &T) -> ! { panic_fmt(format_args!("{}", *x)); } @@ -327,8 +336,9 @@ fn panic_in_cleanup() -> ! { } /// This function is used instead of panic_fmt in const eval. -#[lang = "const_panic_fmt"] -#[rustc_const_unstable(feature = "panic_internals", issue = "none")] +#[lang = "const_panic_fmt"] // needed by const-eval machine to replace calls to `panic_fmt` lang item +#[cfg_attr(bootstrap, rustc_const_unstable(feature = "panic_internals", issue = "none"))] +#[cfg_attr(not(bootstrap), rustc_const_stable_indirect)] // must follow stable const rules since it is exposed to stable pub const fn const_panic_fmt(fmt: fmt::Arguments<'_>) -> ! { if let Some(msg) = fmt.as_str() { // The panic_display function is hooked by const eval. diff --git a/library/core/src/pin.rs b/library/core/src/pin.rs index 9c13662e08e8f..254b306fcaafe 100644 --- a/library/core/src/pin.rs +++ b/library/core/src/pin.rs @@ -921,7 +921,7 @@ #![stable(feature = "pin", since = "1.33.0")] use crate::hash::{Hash, Hasher}; -use crate::ops::{CoerceUnsized, Deref, DerefMut, DerefPure, DispatchFromDyn, Receiver}; +use crate::ops::{CoerceUnsized, Deref, DerefMut, DerefPure, DispatchFromDyn, LegacyReceiver}; #[allow(unused_imports)] use crate::{ cell::{RefCell, UnsafeCell}, @@ -1186,7 +1186,7 @@ impl> Pin { /// let mut pinned: Pin<&mut u8> = Pin::new(&mut val); /// ``` #[inline(always)] - #[rustc_const_unstable(feature = "const_pin", issue = "76654")] + #[rustc_const_stable(feature = "const_pin", since = "CURRENT_RUSTC_VERSION")] #[stable(feature = "pin", since = "1.33.0")] pub const fn new(pointer: Ptr) -> Pin { // SAFETY: the value pointed to is `Unpin`, and so has no requirements @@ -1214,7 +1214,7 @@ impl> Pin { /// assert_eq!(*r, 5); /// ``` #[inline(always)] - #[rustc_const_unstable(feature = "const_pin", issue = "76654")] + #[rustc_const_unstable(feature = "const_pin_2", issue = "76654")] #[stable(feature = "pin_into_inner", since = "1.39.0")] pub const fn into_inner(pin: Pin) -> Ptr { pin.__pointer @@ -1351,7 +1351,7 @@ impl Pin { /// [`pin` module docs]: self #[lang = "new_unchecked"] #[inline(always)] - #[rustc_const_unstable(feature = "const_pin", issue = "76654")] + #[rustc_const_stable(feature = "const_pin", since = "CURRENT_RUSTC_VERSION")] #[stable(feature = "pin", since = "1.33.0")] pub const unsafe fn new_unchecked(pointer: Ptr) -> Pin { Pin { __pointer: pointer } @@ -1422,7 +1422,7 @@ impl Pin { /// move in the future, and this method does not enable the pointee to move. "Malicious" /// implementations of `Ptr::DerefMut` are likewise ruled out by the contract of /// `Pin::new_unchecked`. - #[unstable(feature = "pin_deref_mut", issue = "86918")] + #[stable(feature = "pin_deref_mut", since = "CURRENT_RUSTC_VERSION")] #[must_use = "`self` will be dropped if the result is not used"] #[inline(always)] pub fn as_deref_mut(self: Pin<&mut Pin>) -> Pin<&mut Ptr::Target> { @@ -1503,7 +1503,7 @@ impl Pin { /// If the underlying data is [`Unpin`], [`Pin::into_inner`] should be used /// instead. #[inline(always)] - #[rustc_const_unstable(feature = "const_pin", issue = "76654")] + #[rustc_const_unstable(feature = "const_pin_2", issue = "76654")] #[stable(feature = "pin_into_inner", since = "1.39.0")] pub const unsafe fn into_inner_unchecked(pin: Pin) -> Ptr { pin.__pointer @@ -1559,7 +1559,7 @@ impl<'a, T: ?Sized> Pin<&'a T> { /// ["pinning projections"]: self#projections-and-structural-pinning #[inline(always)] #[must_use] - #[rustc_const_unstable(feature = "const_pin", issue = "76654")] + #[rustc_const_stable(feature = "const_pin", since = "CURRENT_RUSTC_VERSION")] #[stable(feature = "pin", since = "1.33.0")] pub const fn get_ref(self) -> &'a T { self.__pointer @@ -1570,7 +1570,7 @@ impl<'a, T: ?Sized> Pin<&'a mut T> { /// Converts this `Pin<&mut T>` into a `Pin<&T>` with the same lifetime. #[inline(always)] #[must_use = "`self` will be dropped if the result is not used"] - #[rustc_const_unstable(feature = "const_pin", issue = "76654")] + #[rustc_const_stable(feature = "const_pin", since = "CURRENT_RUSTC_VERSION")] #[stable(feature = "pin", since = "1.33.0")] pub const fn into_ref(self) -> Pin<&'a T> { Pin { __pointer: self.__pointer } @@ -1588,7 +1588,7 @@ impl<'a, T: ?Sized> Pin<&'a mut T> { #[inline(always)] #[must_use = "`self` will be dropped if the result is not used"] #[stable(feature = "pin", since = "1.33.0")] - #[rustc_const_unstable(feature = "const_pin", issue = "76654")] + #[rustc_const_stable(feature = "const_pin", since = "CURRENT_RUSTC_VERSION")] pub const fn get_mut(self) -> &'a mut T where T: Unpin, @@ -1609,7 +1609,7 @@ impl<'a, T: ?Sized> Pin<&'a mut T> { #[inline(always)] #[must_use = "`self` will be dropped if the result is not used"] #[stable(feature = "pin", since = "1.33.0")] - #[rustc_const_unstable(feature = "const_pin", issue = "76654")] + #[rustc_const_stable(feature = "const_pin", since = "CURRENT_RUSTC_VERSION")] pub const unsafe fn get_unchecked_mut(self) -> &'a mut T { self.__pointer } @@ -1652,7 +1652,7 @@ impl Pin<&'static T> { /// This is safe because `T` is borrowed immutably for the `'static` lifetime, which /// never ends. #[stable(feature = "pin_static_ref", since = "1.61.0")] - #[rustc_const_unstable(feature = "const_pin", issue = "76654")] + #[rustc_const_stable(feature = "const_pin", since = "CURRENT_RUSTC_VERSION")] pub const fn static_ref(r: &'static T) -> Pin<&'static T> { // SAFETY: The 'static borrow guarantees the data will not be // moved/invalidated until it gets dropped (which is never). @@ -1666,7 +1666,7 @@ impl Pin<&'static mut T> { /// This is safe because `T` is borrowed for the `'static` lifetime, which /// never ends. #[stable(feature = "pin_static_ref", since = "1.61.0")] - #[rustc_const_unstable(feature = "const_pin", issue = "76654")] + #[rustc_const_stable(feature = "const_pin", since = "CURRENT_RUSTC_VERSION")] pub const fn static_mut(r: &'static mut T) -> Pin<&'static mut T> { // SAFETY: The 'static borrow guarantees the data will not be // moved/invalidated until it gets dropped (which is never). @@ -1692,8 +1692,8 @@ impl> DerefMut for Pin { #[unstable(feature = "deref_pure_trait", issue = "87121")] unsafe impl DerefPure for Pin {} -#[unstable(feature = "receiver_trait", issue = "none")] -impl Receiver for Pin {} +#[unstable(feature = "legacy_receiver_trait", issue = "none")] +impl LegacyReceiver for Pin {} #[stable(feature = "pin", since = "1.33.0")] impl fmt::Debug for Pin { diff --git a/library/core/src/primitive.rs b/library/core/src/primitive.rs index 81a72118614dd..b5f97b898878f 100644 --- a/library/core/src/primitive.rs +++ b/library/core/src/primitive.rs @@ -46,7 +46,7 @@ pub use f32; #[stable(feature = "core_primitive", since = "1.43.0")] pub use f64; #[stable(feature = "core_primitive", since = "1.43.0")] -pub use i128; +pub use i8; #[stable(feature = "core_primitive", since = "1.43.0")] pub use i16; #[stable(feature = "core_primitive", since = "1.43.0")] @@ -54,13 +54,13 @@ pub use i32; #[stable(feature = "core_primitive", since = "1.43.0")] pub use i64; #[stable(feature = "core_primitive", since = "1.43.0")] -pub use i8; +pub use i128; #[stable(feature = "core_primitive", since = "1.43.0")] pub use isize; #[stable(feature = "core_primitive", since = "1.43.0")] pub use str; #[stable(feature = "core_primitive", since = "1.43.0")] -pub use u128; +pub use u8; #[stable(feature = "core_primitive", since = "1.43.0")] pub use u16; #[stable(feature = "core_primitive", since = "1.43.0")] @@ -68,6 +68,6 @@ pub use u32; #[stable(feature = "core_primitive", since = "1.43.0")] pub use u64; #[stable(feature = "core_primitive", since = "1.43.0")] -pub use u8; +pub use u128; #[stable(feature = "core_primitive", since = "1.43.0")] pub use usize; diff --git a/library/core/src/primitive_docs.rs b/library/core/src/primitive_docs.rs index 5451e45f6c817..4a6fca5085c42 100644 --- a/library/core/src/primitive_docs.rs +++ b/library/core/src/primitive_docs.rs @@ -100,7 +100,7 @@ mod prim_bool {} /// /// Both match arms must produce values of type [`u32`], but since `break` never produces a value /// at all we know it can never produce a value which isn't a [`u32`]. This illustrates another -/// behaviour of the `!` type - expressions with type `!` will coerce into any other type. +/// behavior of the `!` type - expressions with type `!` will coerce into any other type. /// /// [`u32`]: prim@u32 /// [`exit`]: ../std/process/fn.exit.html @@ -134,7 +134,7 @@ mod prim_bool {} /// /// Since the [`Err`] variant contains a `!`, it can never occur. If the `exhaustive_patterns` /// feature is present this means we can exhaustively match on [`Result`] by just taking the -/// [`Ok`] variant. This illustrates another behaviour of `!` - it can be used to "delete" certain +/// [`Ok`] variant. This illustrates another behavior of `!` - it can be used to "delete" certain /// enum variants from generic types like `Result`. /// /// ## Infinite loops @@ -351,7 +351,7 @@ mod prim_never {} /// ``` /// /// ```no_run -/// // Undefined behaviour +/// // Undefined behavior /// let _ = unsafe { char::from_u32_unchecked(0x110000) }; /// ``` /// @@ -505,9 +505,11 @@ impl () {} /// /// *[See also the `std::ptr` module](ptr).* /// -/// Working with raw pointers in Rust is uncommon, typically limited to a few patterns. -/// Raw pointers can be unaligned or [`null`]. However, when a raw pointer is -/// dereferenced (using the `*` operator), it must be non-null and aligned. +/// Working with raw pointers in Rust is uncommon, typically limited to a few patterns. Raw pointers +/// can be out-of-bounds, unaligned, or [`null`]. However, when loading from or storing to a raw +/// pointer, it must be [valid] for the given access and aligned. When using a field expression, +/// tuple index expression, or array/slice index expression on a raw pointer, it follows the rules +/// of [in-bounds pointer arithmetic][`offset`]. /// /// Storing through a raw pointer using `*ptr = data` calls `drop` on the old value, so /// [`write`] must be used if the type has drop glue and memory is not already @@ -566,7 +568,7 @@ impl () {} /// Instead of coercing a reference to a raw pointer, you can use the macros /// [`ptr::addr_of!`] (for `*const T`) and [`ptr::addr_of_mut!`] (for `*mut T`). /// These macros allow you to create raw pointers to fields to which you cannot -/// create a reference (without causing undefined behaviour), such as an +/// create a reference (without causing undefined behavior), such as an /// unaligned field. This might be necessary if packed structs or uninitialized /// memory is involved. /// @@ -613,6 +615,7 @@ impl () {} /// [`offset`]: pointer::offset /// [`into_raw`]: ../std/boxed/struct.Box.html#method.into_raw /// [`write`]: ptr::write +/// [valid]: ptr#safety #[stable(feature = "rust1", since = "1.0.0")] mod prim_pointer {} @@ -862,6 +865,27 @@ mod prim_array {} /// assert_eq!(x, &[1, 7, 3]); /// ``` /// +/// It is possible to slice empty subranges of slices by using empty ranges (including `slice.len()..slice.len()`): +/// ``` +/// let x = [1, 2, 3]; +/// let empty = &x[0..0]; // subslice before the first element +/// assert_eq!(empty, &[]); +/// let empty = &x[..0]; // same as &x[0..0] +/// assert_eq!(empty, &[]); +/// let empty = &x[1..1]; // empty subslice in the middle +/// assert_eq!(empty, &[]); +/// let empty = &x[3..3]; // subslice after the last element +/// assert_eq!(empty, &[]); +/// let empty = &x[3..]; // same as &x[3..3] +/// assert_eq!(empty, &[]); +/// ``` +/// +/// It is not allowed to use subranges that start with lower bound bigger than `slice.len()`: +/// ```should_panic +/// let x = vec![1, 2, 3]; +/// let _ = &x[4..4]; +/// ``` +/// /// As slices store the length of the sequence they refer to, they have twice /// the size of pointers to [`Sized`](marker/trait.Sized.html) types. /// Also see the reference on @@ -1251,7 +1275,7 @@ mod prim_f16 {} /// - **Unchanged NaN propagation**: The quiet bit and payload are copied from any input operand /// that is a NaN. If the inputs and outputs do not have the same size (i.e., for `as` casts), the /// same rules as for "quieting NaN propagation" apply, with one caveat: if the output is smaller -/// than the input, droppig the low-order bits may result in a payload of 0; a payload of 0 is not +/// than the input, dropping the low-order bits may result in a payload of 0; a payload of 0 is not /// possible with a signaling NaN (the all-0 significand encodes an infinity) so unchanged NaN /// propagation cannot occur with some inputs. /// - **Target-specific NaN**: The quiet bit is set and the payload is picked from a target-specific @@ -1429,7 +1453,7 @@ mod prim_usize {} /// &[bool] can only point to an allocation containing the integer values `1` /// ([`true`](../std/keyword.true.html)) or `0` ([`false`](../std/keyword.false.html)), but /// creating a &[bool] that points to an allocation containing -/// the value `3` causes undefined behaviour. +/// the value `3` causes undefined behavior. /// In fact, [Option]\<&T> has the same memory representation as a /// nullable but aligned pointer, and can be passed across FFI boundaries as such. /// @@ -1588,6 +1612,9 @@ mod prim_ref {} /// pointers, make your type [`Option`](core::option#options-and-pointers-nullable-pointers) /// with your required signature. /// +/// Note that FFI requires additional care to ensure that the ABI for both sides of the call match. +/// The exact requirements are not currently documented. +/// /// ### Safety /// /// Plain function pointers are obtained by casting either plain functions, or closures that don't @@ -1726,8 +1753,13 @@ mod prim_ref {} /// is also used rarely. So, most likely you do not have to worry about ABI compatibility. /// /// But assuming such circumstances, what are the rules? For this section, we are only considering -/// the ABI of direct Rust-to-Rust calls, not linking in general -- once functions are imported via -/// `extern` blocks, there are more things to consider that we do not go into here. +/// the ABI of direct Rust-to-Rust calls (with both definition and callsite visible to the +/// Rust compiler), not linking in general -- once functions are imported via `extern` blocks, there +/// are more things to consider that we do not go into here. Note that this also applies to +/// passing/calling functions across language boundaries via function pointers. +/// +/// **Nothing in this section should be taken as a guarantee for non-Rust-to-Rust calls, even with +/// types from `core::ffi` or `libc`**. /// /// For two signatures to be considered *ABI-compatible*, they must use a compatible ABI string, /// must take the same number of arguments, the individual argument types and the return types must @@ -1760,7 +1792,11 @@ mod prim_ref {} /// unique field that doesn't have size 0 and alignment 1 (if there is such a field). /// - `i32` is ABI-compatible with `NonZero`, and similar for all other integer types. /// - If `T` is guaranteed to be subject to the [null pointer -/// optimization](option/index.html#representation), then `T` and `Option` are ABI-compatible. +/// optimization](option/index.html#representation), and `E` is an enum satisfying the following +/// requirements, then `T` and `E` are ABI-compatible. Such an enum `E` is called "option-like". +/// - The enum `E` has exactly two variants. +/// - One variant has exactly one field, of type `T`. +/// - All fields of the other variant are zero-sized with 1-byte alignment. /// /// Furthermore, ABI compatibility satisfies the following general properties: /// diff --git a/library/core/src/ptr/alignment.rs b/library/core/src/ptr/alignment.rs index fd1b4f9a45d54..9ae3b7e6ea889 100644 --- a/library/core/src/ptr/alignment.rs +++ b/library/core/src/ptr/alignment.rs @@ -1,4 +1,4 @@ -use safety::{ensures, requires}; +use safety::{ensures, invariant, requires}; use crate::num::NonZero; use crate::ub_checks::assert_unsafe_precondition; use crate::{cmp, fmt, hash, mem, num}; @@ -6,6 +6,9 @@ use crate::{cmp, fmt, hash, mem, num}; #[cfg(kani)] use crate::kani; +#[cfg(kani)] +use crate::ub_checks::Invariant; + /// A type storing a `usize` which is a power of two, and thus /// represents a possible alignment in the Rust abstract machine. /// @@ -14,6 +17,7 @@ use crate::kani; #[unstable(feature = "ptr_alignment_type", issue = "102070")] #[derive(Copy, Clone, PartialEq, Eq)] #[repr(transparent)] +#[invariant(self.as_usize().is_power_of_two())] pub struct Alignment(AlignmentEnum); // Alignment is `repr(usize)`, but via extra steps. @@ -45,7 +49,7 @@ impl Alignment { /// This provides the same numerical value as [`mem::align_of`], /// but in an `Alignment` instead of a `usize`. #[unstable(feature = "ptr_alignment_type", issue = "102070")] - #[rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070"))] #[inline] #[requires(mem::align_of::().is_power_of_two())] #[ensures(|result| result.as_usize().is_power_of_two())] @@ -59,7 +63,7 @@ impl Alignment { /// /// Note that `0` is not a power of two, nor a valid alignment. #[unstable(feature = "ptr_alignment_type", issue = "102070")] - #[rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070"))] #[inline] #[ensures(|result| align.is_power_of_two() == result.is_some())] #[ensures(|result| result.is_none() || result.unwrap().as_usize() == align)] @@ -81,7 +85,7 @@ impl Alignment { /// Equivalently, it must be `1 << exp` for some `exp` in `0..usize::BITS`. /// It must *not* be zero. #[unstable(feature = "ptr_alignment_type", issue = "102070")] - #[rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070"))] #[inline] #[requires(align > 0 && (align & (align - 1)) == 0)] #[ensures(|result| result.as_usize() == align)] @@ -100,7 +104,7 @@ impl Alignment { /// Returns the alignment as a [`usize`]. #[unstable(feature = "ptr_alignment_type", issue = "102070")] - #[rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070"))] #[inline] #[ensures(|result| result.is_power_of_two())] pub const fn as_usize(self) -> usize { @@ -109,7 +113,7 @@ impl Alignment { /// Returns the alignment as a [NonZero]<[usize]>. #[unstable(feature = "ptr_alignment_type", issue = "102070")] - #[rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070"))] #[inline] #[ensures(|result| result.get().is_power_of_two())] #[ensures(|result| result.get() == self.as_usize())] @@ -132,7 +136,7 @@ impl Alignment { /// assert_eq!(Alignment::new(1024).unwrap().log2(), 10); /// ``` #[unstable(feature = "ptr_alignment_type", issue = "102070")] - #[rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070"))] #[inline] #[requires(self.as_usize().is_power_of_two())] #[ensures(|result| (*result as usize) < mem::size_of::() * 8)] @@ -165,7 +169,7 @@ impl Alignment { /// assert_ne!(one.mask(Alignment::of::().mask()), one); /// ``` #[unstable(feature = "ptr_alignment_type", issue = "102070")] - #[rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "ptr_alignment_type", issue = "102070"))] #[inline] #[ensures(|result| *result > 0)] #[ensures(|result| *result == !(self.as_usize() -1))] @@ -174,6 +178,11 @@ impl Alignment { // SAFETY: The alignment is always nonzero, and therefore decrementing won't overflow. !(unsafe { self.as_usize().unchecked_sub(1) }) } + + // FIXME(const-hack) Remove me once `Ord::max` is usable in const + pub(crate) const fn max(a: Self, b: Self) -> Self { + if a.as_usize() > b.as_usize() { a } else { b } + } } #[unstable(feature = "ptr_alignment_type", issue = "102070")] @@ -219,7 +228,6 @@ impl From for usize { } } -#[rustc_const_unstable(feature = "const_alloc_layout", issue = "67521")] #[unstable(feature = "ptr_alignment_type", issue = "102070")] impl cmp::Ord for Alignment { #[inline] @@ -228,7 +236,6 @@ impl cmp::Ord for Alignment { } } -#[rustc_const_unstable(feature = "const_alloc_layout", issue = "67521")] #[unstable(feature = "ptr_alignment_type", issue = "102070")] impl cmp::PartialOrd for Alignment { #[inline] @@ -256,6 +263,7 @@ impl Default for Alignment { #[cfg(target_pointer_width = "16")] #[derive(Copy, Clone, PartialEq, Eq)] #[repr(u16)] +#[cfg_attr(kani, derive(kani::Arbitrary))] enum AlignmentEnum { _Align1Shl0 = 1 << 0, _Align1Shl1 = 1 << 1, @@ -278,6 +286,7 @@ enum AlignmentEnum { #[cfg(target_pointer_width = "32")] #[derive(Copy, Clone, PartialEq, Eq)] #[repr(u32)] +#[cfg_attr(kani, derive(kani::Arbitrary))] enum AlignmentEnum { _Align1Shl0 = 1 << 0, _Align1Shl1 = 1 << 1, @@ -316,6 +325,7 @@ enum AlignmentEnum { #[cfg(target_pointer_width = "64")] #[derive(Copy, Clone, PartialEq, Eq)] #[repr(u64)] +#[cfg_attr(kani, derive(kani::Arbitrary))] enum AlignmentEnum { _Align1Shl0 = 1 << 0, _Align1Shl1 = 1 << 1, @@ -390,8 +400,9 @@ mod verify { impl kani::Arbitrary for Alignment { fn any() -> Self { - let align = kani::any_where(|a: &usize| a.is_power_of_two()); - unsafe { mem::transmute::(align) } + let obj = Self { 0: kani::any() }; + kani::assume(obj.is_safe()); + obj } } diff --git a/library/core/src/ptr/const_ptr.rs b/library/core/src/ptr/const_ptr.rs index 3b635e2a4aa9e..57a7c0fc0925c 100644 --- a/library/core/src/ptr/const_ptr.rs +++ b/library/core/src/ptr/const_ptr.rs @@ -39,16 +39,19 @@ impl *const T { } #[inline] + #[rustc_const_unstable(feature = "const_ptr_is_null", issue = "74939")] const fn const_impl(ptr: *const u8) -> bool { - // Compare via a cast to a thin pointer, so fat pointers are only - // considering their "data" part for null-ness. match (ptr).guaranteed_eq(null_mut()) { - None => false, Some(res) => res, + // To remain maximally convervative, we stop execution when we don't + // know whether the pointer is null or not. + // We can *not* return `false` here, that would be unsound in `NonNull::new`! + None => panic!("null-ness of this pointer cannot be determined in const context"), } } - #[allow(unused_unsafe)] + // Compare via a cast to a thin pointer, so fat pointers are only + // considering their "data" part for null-ness. const_eval_select((self as *const u8,), const_impl, runtime_impl) } @@ -61,21 +64,22 @@ impl *const T { self as _ } - /// Uses the pointer value in a new pointer of another type. + /// Uses the address value in a new pointer of another type. /// - /// In case `meta` is a (fat) pointer to an unsized type, this operation - /// will ignore the pointer part, whereas for (thin) pointers to sized - /// types, this has the same effect as a simple cast. + /// This operation will ignore the address part of its `meta` operand and discard existing + /// metadata of `self`. For pointers to a sized types (thin pointers), this has the same effect + /// as a simple cast. For pointers to an unsized type (fat pointers) this recombines the address + /// with new metadata such as slice lengths or `dyn`-vtable. /// - /// The resulting pointer will have provenance of `self`, i.e., for a fat - /// pointer, this operation is semantically the same as creating a new - /// fat pointer with the data pointer value of `self` but the metadata of - /// `meta`. + /// The resulting pointer will have provenance of `self`. This operation is semantically the + /// same as creating a new pointer with the data pointer value of `self` but the metadata of + /// `meta`, being fat or thin depending on the `meta` operand. /// /// # Examples /// - /// This function is primarily useful for allowing byte-wise pointer - /// arithmetic on potentially fat pointers: + /// This function is primarily useful for enabling pointer arithmetic on potentially fat + /// pointers. The pointer is cast to a sized pointee to utilize offset operations and then + /// recombined with its own original metadata. /// /// ``` /// #![feature(set_ptr_value)] @@ -89,8 +93,28 @@ impl *const T { /// println!("{:?}", &*ptr); // will print "3" /// } /// ``` + /// + /// # *Incorrect* usage + /// + /// The provenance from pointers is *not* combined. The result must only be used to refer to the + /// address allowed by `self`. + /// + /// ```rust,no_run + /// #![feature(set_ptr_value)] + /// let x = 0u32; + /// let y = 1u32; + /// + /// let x = (&x) as *const u32; + /// let y = (&y) as *const u32; + /// + /// let offset = (x as usize - y as usize) / 4; + /// let bad = x.wrapping_add(offset).with_metadata_of(y); + /// + /// // This dereference is UB. The pointer only has provenance for `x` but points to `y`. + /// println!("{:?}", unsafe { &*bad }); + /// ``` #[unstable(feature = "set_ptr_value", issue = "75091")] - #[rustc_const_unstable(feature = "set_ptr_value", issue = "75091")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "ptr_metadata_const", since = "1.83.0"))] #[must_use = "returns a new pointer rather than modifying its argument"] #[inline] pub const fn with_metadata_of(self, meta: *const U) -> *const U @@ -114,10 +138,11 @@ impl *const T { /// Gets the "address" portion of the pointer. /// - /// This is similar to `self as usize`, which semantically discards *provenance* and - /// *address-space* information. However, unlike `self as usize`, casting the returned address - /// back to a pointer yields a [pointer without provenance][without_provenance], which is undefined behavior to dereference. To - /// properly restore the lost information and obtain a dereferenceable pointer, use + /// This is similar to `self as usize`, except that the [provenance][crate::ptr#provenance] of + /// the pointer is discarded and not [exposed][crate::ptr#exposed-provenance]. This means that + /// casting the returned address back to a pointer yields a [pointer without + /// provenance][without_provenance], which is undefined behavior to dereference. To properly + /// restore the lost information and obtain a dereferenceable pointer, use /// [`with_addr`][pointer::with_addr] or [`map_addr`][pointer::map_addr]. /// /// If using those APIs is not possible because there is no way to preserve a pointer with the @@ -132,90 +157,81 @@ impl *const T { /// perform a change of representation to produce a value containing only the address /// portion of the pointer. What that means is up to the platform to define. /// - /// This API and its claimed semantics are part of the Strict Provenance experiment, and as such - /// might change in the future (including possibly weakening this so it becomes wholly - /// equivalent to `self as usize`). See the [module documentation][crate::ptr] for details. + /// This is a [Strict Provenance][crate::ptr#strict-provenance] API. #[must_use] #[inline(always)] - #[unstable(feature = "strict_provenance", issue = "95228")] + #[stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] pub fn addr(self) -> usize { - // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic. + // A pointer-to-integer transmute currently has exactly the right semantics: it returns the + // address without exposing the provenance. Note that this is *not* a stable guarantee about + // transmute semantics, it relies on sysroot crates having special status. // SAFETY: Pointer-to-integer transmutes are valid (if you are okay with losing the // provenance). unsafe { mem::transmute(self.cast::<()>()) } } - /// Exposes the "provenance" part of the pointer for future use in - /// [`with_exposed_provenance`][] and returns the "address" portion. + /// Exposes the ["provenance"][crate::ptr#provenance] part of the pointer for future use in + /// [`with_exposed_provenance`] and returns the "address" portion. /// - /// This is equivalent to `self as usize`, which semantically discards *provenance* and - /// *address-space* information. Furthermore, this (like the `as` cast) has the implicit - /// side-effect of marking the provenance as 'exposed', so on platforms that support it you can - /// later call [`with_exposed_provenance`][] to reconstitute the original pointer including its - /// provenance. (Reconstructing address space information, if required, is your responsibility.) + /// This is equivalent to `self as usize`, which semantically discards provenance information. + /// Furthermore, this (like the `as` cast) has the implicit side-effect of marking the + /// provenance as 'exposed', so on platforms that support it you can later call + /// [`with_exposed_provenance`] to reconstitute the original pointer including its provenance. /// - /// Using this method means that code is *not* following [Strict - /// Provenance][super#strict-provenance] rules. Supporting - /// [`with_exposed_provenance`][] complicates specification and reasoning and may not be supported by - /// tools that help you to stay conformant with the Rust memory model, so it is recommended to - /// use [`addr`][pointer::addr] wherever possible. + /// Due to its inherent ambiguity, [`with_exposed_provenance`] may not be supported by tools + /// that help you to stay conformant with the Rust memory model. It is recommended to use + /// [Strict Provenance][crate::ptr#strict-provenance] APIs such as [`with_addr`][pointer::with_addr] + /// wherever possible, in which case [`addr`][pointer::addr] should be used instead of `expose_provenance`. /// /// On most platforms this will produce a value with the same bytes as the original pointer, /// because all the bytes are dedicated to describing the address. Platforms which need to store /// additional information in the pointer may not support this operation, since the 'expose' - /// side-effect which is required for [`with_exposed_provenance`][] to work is typically not + /// side-effect which is required for [`with_exposed_provenance`] to work is typically not /// available. /// - /// It is unclear whether this method can be given a satisfying unambiguous specification. This - /// API and its claimed semantics are part of [Exposed Provenance][super#exposed-provenance]. + /// This is an [Exposed Provenance][crate::ptr#exposed-provenance] API. /// /// [`with_exposed_provenance`]: with_exposed_provenance #[must_use] #[inline(always)] - #[unstable(feature = "exposed_provenance", issue = "95228")] + #[stable(feature = "exposed_provenance", since = "CURRENT_RUSTC_VERSION")] pub fn expose_provenance(self) -> usize { - // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic. self.cast::<()>() as usize } - /// Creates a new pointer with the given address. + /// Creates a new pointer with the given address and the [provenance][crate::ptr#provenance] of + /// `self`. /// - /// This performs the same operation as an `addr as ptr` cast, but copies - /// the *address-space* and *provenance* of `self` to the new pointer. - /// This allows us to dynamically preserve and propagate this important - /// information in a way that is otherwise impossible with a unary cast. + /// This is similar to a `addr as *const T` cast, but copies + /// the *provenance* of `self` to the new pointer. + /// This avoids the inherent ambiguity of the unary cast. /// /// This is equivalent to using [`wrapping_offset`][pointer::wrapping_offset] to offset /// `self` to the given address, and therefore has all the same capabilities and restrictions. /// - /// This API and its claimed semantics are part of the Strict Provenance experiment, - /// see the [module documentation][crate::ptr] for details. + /// This is a [Strict Provenance][crate::ptr#strict-provenance] API. #[must_use] #[inline] - #[unstable(feature = "strict_provenance", issue = "95228")] + #[stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] pub fn with_addr(self, addr: usize) -> Self { - // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic. - // - // In the mean-time, this operation is defined to be "as if" it was - // a wrapping_offset, so we can emulate it as such. This should properly - // restore pointer provenance even under today's compiler. + // This should probably be an intrinsic to avoid doing any sort of arithmetic, but + // meanwhile, we can implement it with `wrapping_offset`, which preserves the pointer's + // provenance. let self_addr = self.addr() as isize; let dest_addr = addr as isize; let offset = dest_addr.wrapping_sub(self_addr); - - // This is the canonical desugaring of this operation self.wrapping_byte_offset(offset) } - /// Creates a new pointer by mapping `self`'s address to a new one. + /// Creates a new pointer by mapping `self`'s address to a new one, preserving the + /// [provenance][crate::ptr#provenance] of `self`. /// /// This is a convenience for [`with_addr`][pointer::with_addr], see that method for details. /// - /// This API and its claimed semantics are part of the Strict Provenance experiment, - /// see the [module documentation][crate::ptr] for details. + /// This is a [Strict Provenance][crate::ptr#strict-provenance] API. #[must_use] #[inline] - #[unstable(feature = "strict_provenance", issue = "95228")] + #[stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] pub fn map_addr(self, f: impl FnOnce(usize) -> usize) -> Self { self.with_addr(f(self.addr())) } @@ -268,7 +284,7 @@ impl *const T { /// } /// ``` #[stable(feature = "ptr_as_ref", since = "1.9.0")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "const_ptr_is_null", issue = "74939")] #[inline] pub const unsafe fn as_ref<'a>(self) -> Option<&'a T> { // SAFETY: the caller must guarantee that `self` is valid @@ -300,7 +316,7 @@ impl *const T { /// ``` // FIXME: mention it in the docs for `as_ref` and `as_uninit_ref` once stabilized. #[unstable(feature = "ptr_as_ref_unchecked", issue = "122034")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "ptr_as_ref_unchecked", issue = "122034")] #[inline] #[must_use] pub const unsafe fn as_ref_unchecked<'a>(self) -> &'a T { @@ -334,7 +350,7 @@ impl *const T { /// ``` #[inline] #[unstable(feature = "ptr_as_uninit", issue = "75402")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "ptr_as_uninit", issue = "75402")] pub const unsafe fn as_uninit_ref<'a>(self) -> Option<&'a MaybeUninit> where T: Sized, @@ -344,7 +360,7 @@ impl *const T { if self.is_null() { None } else { Some(unsafe { &*(self as *const MaybeUninit) }) } } - /// Adds an offset to a pointer. + /// Adds a signed offset to a pointer. /// /// `count` is in units of T; e.g., a `count` of 3 represents a pointer /// offset of `3 * size_of::()` bytes. @@ -353,9 +369,10 @@ impl *const T { /// /// If any of the following conditions are violated, the result is Undefined Behavior: /// - /// * The computed offset, `count * size_of::()` bytes, must not overflow `isize`. + /// * The offset in bytes, `count * size_of::()`, computed on mathematical integers (without + /// "wrapping around"), must fit in an `isize`. /// - /// * If the computed offset is non-zero, then `self` must be derived from a pointer to some + /// * If the computed offset is non-zero, then `self` must be [derived from][crate::ptr#provenance] a pointer to some /// [allocated object], and the entire memory range between `self` and the result must be in /// bounds of that allocated object. In particular, this range must not "wrap around" the edge /// of the address space. @@ -392,11 +409,42 @@ impl *const T { where T: Sized, { + #[inline] + #[rustc_allow_const_fn_unstable(const_eval_select)] + const fn runtime_offset_nowrap(this: *const (), count: isize, size: usize) -> bool { + #[inline] + fn runtime(this: *const (), count: isize, size: usize) -> bool { + // We know `size <= isize::MAX` so the `as` cast here is not lossy. + let Some(byte_offset) = count.checked_mul(size as isize) else { + return false; + }; + let (_, overflow) = this.addr().overflowing_add_signed(byte_offset); + !overflow + } + + const fn comptime(_: *const (), _: isize, _: usize) -> bool { + true + } + + // We can use const_eval_select here because this is only for UB checks. + intrinsics::const_eval_select((this, count, size), comptime, runtime) + } + + ub_checks::assert_unsafe_precondition!( + check_language_ub, + "ptr::offset requires the address calculation to not overflow", + ( + this: *const () = self as *const (), + count: isize = count, + size: usize = size_of::(), + ) => runtime_offset_nowrap(this, count, size) + ); + // SAFETY: the caller must uphold the safety contract for `offset`. unsafe { intrinsics::offset(self, count) } } - /// Calculates the offset from a pointer in bytes. + /// Adds a signed offset in bytes to a pointer. /// /// `count` is in units of **bytes**. /// @@ -410,14 +458,13 @@ impl *const T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn byte_offset(self, count: isize) -> Self { // SAFETY: the caller must uphold the safety contract for `offset`. unsafe { self.cast::().offset(count).with_metadata_of(self) } } - /// Calculates the offset from a pointer using wrapping arithmetic. + /// Adds a signed offset to a pointer using wrapping arithmetic. /// /// `count` is in units of T; e.g., a `count` of 3 represents a pointer /// offset of `3 * size_of::()` bytes. @@ -479,7 +526,7 @@ impl *const T { unsafe { intrinsics::arith_offset(self, count) } } - /// Calculates the offset from a pointer in bytes using wrapping arithmetic. + /// Adds a signed offset in bytes to a pointer using wrapping arithmetic. /// /// `count` is in units of **bytes**. /// @@ -493,7 +540,6 @@ impl *const T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] pub const fn wrapping_byte_offset(self, count: isize) -> Self { self.cast::().wrapping_offset(count).with_metadata_of(self) } @@ -508,7 +554,7 @@ impl *const T { /// ## Examples /// /// ``` - /// #![feature(ptr_mask, strict_provenance)] + /// #![feature(ptr_mask)] /// let v = 17_u32; /// let ptr: *const u32 = &v; /// @@ -536,7 +582,7 @@ impl *const T { intrinsics::ptr_mask(self.cast::<()>(), mask).with_metadata_of(self) } - /// Calculates the distance between two pointers. The returned value is in + /// Calculates the distance between two pointers within the same allocation. The returned value is in /// units of T: the distance in bytes divided by `mem::size_of::()`. /// /// This is equivalent to `(self as isize - origin as isize) / (mem::size_of::() as isize)`, @@ -559,7 +605,7 @@ impl *const T { /// * `self` and `origin` must either /// /// * point to the same address, or - /// * both be *derived from* a pointer to the same [allocated object], and the memory range between + /// * both be [derived from][crate::ptr#provenance] a pointer to the same [allocated object], and the memory range between /// the two pointers must be in bounds of that object. (See below for an example.) /// /// * The distance between the pointers, in bytes, must be an exact multiple @@ -631,7 +677,7 @@ impl *const T { unsafe { intrinsics::ptr_offset_from(self, origin) } } - /// Calculates the distance between two pointers. The returned value is in + /// Calculates the distance between two pointers within the same allocation. The returned value is in /// units of **bytes**. /// /// This is purely a convenience for casting to a `u8` pointer and @@ -643,14 +689,13 @@ impl *const T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn byte_offset_from(self, origin: *const U) -> isize { // SAFETY: the caller must uphold the safety contract for `offset_from`. unsafe { self.cast::().offset_from(origin.cast::()) } } - /// Calculates the distance between two pointers, *where it's known that + /// Calculates the distance between two pointers within the same allocation, *where it's known that /// `self` is equal to or greater than `origin`*. The returned value is in /// units of T: the distance in bytes is divided by `mem::size_of::()`. /// @@ -661,7 +706,7 @@ impl *const T { /// but it provides slightly more information to the optimizer, which can /// sometimes allow it to optimize slightly better with some backends. /// - /// This method can be though of as recovering the `count` that was passed + /// This method can be thought of as recovering the `count` that was passed /// to [`add`](#method.add) (or, with the parameters in the other order, /// to [`sub`](#method.sub)). The following are all equivalent, assuming /// that their safety preconditions are met: @@ -718,6 +763,7 @@ impl *const T { where T: Sized, { + #[rustc_allow_const_fn_unstable(const_eval_select)] const fn runtime_ptr_ge(this: *const (), origin: *const ()) -> bool { fn runtime(this: *const (), origin: *const ()) -> bool { this >= origin @@ -726,7 +772,6 @@ impl *const T { true } - #[allow(unused_unsafe)] intrinsics::const_eval_select((this, origin), comptime, runtime) } @@ -745,6 +790,25 @@ impl *const T { unsafe { intrinsics::ptr_offset_from_unsigned(self, origin) } } + /// Calculates the distance between two pointers within the same allocation, *where it's known that + /// `self` is equal to or greater than `origin`*. The returned value is in + /// units of **bytes**. + /// + /// This is purely a convenience for casting to a `u8` pointer and + /// using [`sub_ptr`][pointer::sub_ptr] on it. See that method for + /// documentation and safety requirements. + /// + /// For non-`Sized` pointees this operation considers only the data pointers, + /// ignoring the metadata. + #[unstable(feature = "ptr_sub_ptr", issue = "95892")] + #[rustc_const_unstable(feature = "const_ptr_sub_ptr", issue = "95892")] + #[inline] + #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces + pub const unsafe fn byte_sub_ptr(self, origin: *const U) -> usize { + // SAFETY: the caller must uphold the safety contract for `sub_ptr`. + unsafe { self.cast::().sub_ptr(origin.cast::()) } + } + /// Returns whether two pointers are guaranteed to be equal. /// /// At runtime this function behaves like `Some(self == other)`. @@ -805,7 +869,11 @@ impl *const T { } } - /// Adds an offset to a pointer (convenience for `.offset(count as isize)`). + /// Adds an unsigned offset to a pointer. + /// + /// This can only move the pointer forward (or not move it). If you need to move forward or + /// backward depending on the value, then you might want [`offset`](#method.offset) instead + /// which takes a signed offset. /// /// `count` is in units of T; e.g., a `count` of 3 represents a pointer /// offset of `3 * size_of::()` bytes. @@ -814,9 +882,10 @@ impl *const T { /// /// If any of the following conditions are violated, the result is Undefined Behavior: /// - /// * The computed offset, `count * size_of::()` bytes, must not overflow `isize`. + /// * The offset in bytes, `count * size_of::()`, computed on mathematical integers (without + /// "wrapping around"), must fit in an `isize`. /// - /// * If the computed offset is non-zero, then `self` must be derived from a pointer to some + /// * If the computed offset is non-zero, then `self` must be [derived from][crate::ptr#provenance] a pointer to some /// [allocated object], and the entire memory range between `self` and the result must be in /// bounds of that allocated object. In particular, this range must not "wrap around" the edge /// of the address space. @@ -853,11 +922,42 @@ impl *const T { where T: Sized, { + #[cfg(debug_assertions)] + #[inline] + #[rustc_allow_const_fn_unstable(const_eval_select)] + const fn runtime_add_nowrap(this: *const (), count: usize, size: usize) -> bool { + #[inline] + fn runtime(this: *const (), count: usize, size: usize) -> bool { + let Some(byte_offset) = count.checked_mul(size) else { + return false; + }; + let (_, overflow) = this.addr().overflowing_add(byte_offset); + byte_offset <= (isize::MAX as usize) && !overflow + } + + const fn comptime(_: *const (), _: usize, _: usize) -> bool { + true + } + + intrinsics::const_eval_select((this, count, size), comptime, runtime) + } + + #[cfg(debug_assertions)] // Expensive, and doesn't catch much in the wild. + ub_checks::assert_unsafe_precondition!( + check_language_ub, + "ptr::add requires that the address calculation does not overflow", + ( + this: *const () = self as *const (), + count: usize = count, + size: usize = size_of::(), + ) => runtime_add_nowrap(this, count, size) + ); + // SAFETY: the caller must uphold the safety contract for `offset`. unsafe { intrinsics::offset(self, count) } } - /// Calculates the offset from a pointer in bytes (convenience for `.byte_offset(count as isize)`). + /// Adds an unsigned offset in bytes to a pointer. /// /// `count` is in units of bytes. /// @@ -871,15 +971,17 @@ impl *const T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn byte_add(self, count: usize) -> Self { // SAFETY: the caller must uphold the safety contract for `add`. unsafe { self.cast::().add(count).with_metadata_of(self) } } - /// Subtracts an offset from a pointer (convenience for - /// `.offset((count as isize).wrapping_neg())`). + /// Subtracts an unsigned offset from a pointer. + /// + /// This can only move the pointer backward (or not move it). If you need to move forward or + /// backward depending on the value, then you might want [`offset`](#method.offset) instead + /// which takes a signed offset. /// /// `count` is in units of T; e.g., a `count` of 3 represents a pointer /// offset of `3 * size_of::()` bytes. @@ -888,9 +990,10 @@ impl *const T { /// /// If any of the following conditions are violated, the result is Undefined Behavior: /// - /// * The computed offset, `count * size_of::()` bytes, must not overflow `isize`. + /// * The offset in bytes, `count * size_of::()`, computed on mathematical integers (without + /// "wrapping around"), must fit in an `isize`. /// - /// * If the computed offset is non-zero, then `self` must be derived from a pointer to some + /// * If the computed offset is non-zero, then `self` must be [derived from][crate::ptr#provenance] a pointer to some /// [allocated object], and the entire memory range between `self` and the result must be in /// bounds of that allocated object. In particular, this range must not "wrap around" the edge /// of the address space. @@ -921,13 +1024,43 @@ impl *const T { #[stable(feature = "pointer_methods", since = "1.26.0")] #[must_use = "returns a new pointer rather than modifying its argument"] #[rustc_const_stable(feature = "const_ptr_offset", since = "1.61.0")] - #[rustc_allow_const_fn_unstable(unchecked_neg)] + #[cfg_attr(bootstrap, rustc_allow_const_fn_unstable(unchecked_neg))] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn sub(self, count: usize) -> Self where T: Sized, { + #[cfg(debug_assertions)] + #[inline] + #[rustc_allow_const_fn_unstable(const_eval_select)] + const fn runtime_sub_nowrap(this: *const (), count: usize, size: usize) -> bool { + #[inline] + fn runtime(this: *const (), count: usize, size: usize) -> bool { + let Some(byte_offset) = count.checked_mul(size) else { + return false; + }; + byte_offset <= (isize::MAX as usize) && this.addr() >= byte_offset + } + + const fn comptime(_: *const (), _: usize, _: usize) -> bool { + true + } + + intrinsics::const_eval_select((this, count, size), comptime, runtime) + } + + #[cfg(debug_assertions)] // Expensive, and doesn't catch much in the wild. + ub_checks::assert_unsafe_precondition!( + check_language_ub, + "ptr::sub requires that the address calculation does not overflow", + ( + this: *const () = self as *const (), + count: usize = count, + size: usize = size_of::(), + ) => runtime_sub_nowrap(this, count, size) + ); + if T::IS_ZST { // Pointer arithmetic does nothing when the pointee is a ZST. self @@ -935,12 +1068,11 @@ impl *const T { // SAFETY: the caller must uphold the safety contract for `offset`. // Because the pointee is *not* a ZST, that means that `count` is // at most `isize::MAX`, and thus the negation cannot overflow. - unsafe { self.offset((count as isize).unchecked_neg()) } + unsafe { intrinsics::offset(self, intrinsics::unchecked_sub(0, count as isize)) } } } - /// Calculates the offset from a pointer in bytes (convenience for - /// `.byte_offset((count as isize).wrapping_neg())`). + /// Subtracts an unsigned offset in bytes from a pointer. /// /// `count` is in units of bytes. /// @@ -954,15 +1086,13 @@ impl *const T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn byte_sub(self, count: usize) -> Self { // SAFETY: the caller must uphold the safety contract for `sub`. unsafe { self.cast::().sub(count).with_metadata_of(self) } } - /// Calculates the offset from a pointer using wrapping arithmetic. - /// (convenience for `.wrapping_offset(count as isize)`) + /// Adds an unsigned offset to a pointer using wrapping arithmetic. /// /// `count` is in units of T; e.g., a `count` of 3 represents a pointer /// offset of `3 * size_of::()` bytes. @@ -1023,8 +1153,7 @@ impl *const T { self.wrapping_offset(count as isize) } - /// Calculates the offset from a pointer in bytes using wrapping arithmetic. - /// (convenience for `.wrapping_byte_offset(count as isize)`) + /// Adds an unsigned offset in bytes to a pointer using wrapping arithmetic. /// /// `count` is in units of bytes. /// @@ -1037,13 +1166,11 @@ impl *const T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] pub const fn wrapping_byte_add(self, count: usize) -> Self { self.cast::().wrapping_add(count).with_metadata_of(self) } - /// Calculates the offset from a pointer using wrapping arithmetic. - /// (convenience for `.wrapping_offset((count as isize).wrapping_neg())`) + /// Subtracts an unsigned offset from a pointer using wrapping arithmetic. /// /// `count` is in units of T; e.g., a `count` of 3 represents a pointer /// offset of `3 * size_of::()` bytes. @@ -1104,8 +1231,7 @@ impl *const T { self.wrapping_offset((count as isize).wrapping_neg()) } - /// Calculates the offset from a pointer in bytes using wrapping arithmetic. - /// (convenience for `.wrapping_offset((count as isize).wrapping_neg())`) + /// Subtracts an unsigned offset in bytes from a pointer using wrapping arithmetic. /// /// `count` is in units of bytes. /// @@ -1118,7 +1244,6 @@ impl *const T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] pub const fn wrapping_byte_sub(self, count: usize) -> Self { self.cast::().wrapping_sub(count).with_metadata_of(self) } @@ -1190,7 +1315,7 @@ impl *const T { /// See [`ptr::copy`] for safety concerns and examples. /// /// [`ptr::copy`]: crate::ptr::copy() - #[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] + #[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")] #[stable(feature = "pointer_methods", since = "1.26.0")] #[inline] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces @@ -1210,7 +1335,7 @@ impl *const T { /// See [`ptr::copy_nonoverlapping`] for safety concerns and examples. /// /// [`ptr::copy_nonoverlapping`]: crate::ptr::copy_nonoverlapping() - #[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] + #[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")] #[stable(feature = "pointer_methods", since = "1.26.0")] #[inline] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces @@ -1521,6 +1646,7 @@ impl *const T { } #[inline] + #[rustc_const_unstable(feature = "const_pointer_is_aligned", issue = "104203")] const fn const_impl(ptr: *const (), align: usize) -> bool { // We can't use the address of `self` in a `const fn`, so we use `align_offset` instead. ptr.align_offset(align) == 0 @@ -1552,7 +1678,6 @@ impl *const [T] { #[inline] #[stable(feature = "slice_ptr_len", since = "1.79.0")] #[rustc_const_stable(feature = "const_slice_ptr_len", since = "1.79.0")] - #[rustc_allow_const_fn_unstable(ptr_metadata)] pub const fn len(self) -> usize { metadata(self) } @@ -1662,7 +1787,7 @@ impl *const [T] { /// [allocated object]: crate::ptr#allocated-object #[inline] #[unstable(feature = "ptr_as_uninit", issue = "75402")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "ptr_as_uninit", issue = "75402")] pub const unsafe fn as_uninit_slice<'a>(self) -> Option<&'a [MaybeUninit]> { if self.is_null() { None diff --git a/library/core/src/ptr/metadata.rs b/library/core/src/ptr/metadata.rs index ccc9f8754f00c..5f20cb2ee7206 100644 --- a/library/core/src/ptr/metadata.rs +++ b/library/core/src/ptr/metadata.rs @@ -92,7 +92,7 @@ pub trait Thin = Pointee; /// /// assert_eq!(std::ptr::metadata("foo"), 3_usize); /// ``` -#[rustc_const_unstable(feature = "ptr_metadata", issue = "81513")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "ptr_metadata_const", since = "1.83.0"))] #[inline] pub const fn metadata(ptr: *const T) -> ::Metadata { ptr_metadata(ptr) @@ -106,7 +106,7 @@ pub const fn metadata(ptr: *const T) -> ::Metadata { /// /// [`slice::from_raw_parts`]: crate::slice::from_raw_parts #[unstable(feature = "ptr_metadata", issue = "81513")] -#[rustc_const_unstable(feature = "ptr_metadata", issue = "81513")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "ptr_metadata_const", since = "1.83.0"))] #[inline] pub const fn from_raw_parts( data_pointer: *const impl Thin, @@ -120,7 +120,7 @@ pub const fn from_raw_parts( /// /// See the documentation of [`from_raw_parts`] for more details. #[unstable(feature = "ptr_metadata", issue = "81513")] -#[rustc_const_unstable(feature = "ptr_metadata", issue = "81513")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "ptr_metadata_const", since = "1.83.0"))] #[inline] pub const fn from_raw_parts_mut( data_pointer: *mut impl Thin, @@ -187,14 +187,14 @@ impl DynMetadata { // Consider a reference like `&(i32, dyn Send)`: the vtable will only store the size of the // `Send` part! // SAFETY: DynMetadata always contains a valid vtable pointer - return unsafe { crate::intrinsics::vtable_size(self.vtable_ptr() as *const ()) }; + unsafe { crate::intrinsics::vtable_size(self.vtable_ptr() as *const ()) } } /// Returns the alignment of the type associated with this vtable. #[inline] pub fn align_of(self) -> usize { // SAFETY: DynMetadata always contains a valid vtable pointer - return unsafe { crate::intrinsics::vtable_align(self.vtable_ptr() as *const ()) }; + unsafe { crate::intrinsics::vtable_align(self.vtable_ptr() as *const ()) } } /// Returns the size and alignment together as a `Layout` diff --git a/library/core/src/ptr/mod.rs b/library/core/src/ptr/mod.rs index 2f92443cf1656..2bcea0b8fdb36 100644 --- a/library/core/src/ptr/mod.rs +++ b/library/core/src/ptr/mod.rs @@ -18,10 +18,11 @@ //! * For operations of [size zero][zst], *every* pointer is valid, including the [null] pointer. //! The following points are only concerned with non-zero-sized accesses. //! * A [null] pointer is *never* valid. -//! * For a pointer to be valid, it is necessary, but not always sufficient, that the pointer -//! be *dereferenceable*: the memory range of the given size starting at the pointer must all be -//! within the bounds of a single allocated object. Note that in Rust, -//! every (stack-allocated) variable is considered a separate allocated object. +//! * For a pointer to be valid, it is necessary, but not always sufficient, that the pointer be +//! *dereferenceable*. The [provenance] of the pointer is used to determine which [allocated +//! object] it is derived from; a pointer is dereferenceable if the memory range of the given size +//! starting at the pointer is entirely contained within the bounds of that allocated object. Note +//! that in Rust, every (stack-allocated) variable is considered a separate allocated object. //! * All accesses performed by functions in this module are *non-atomic* in the sense //! of [atomic operations] used to synchronize between threads. This means it is //! undefined behavior to perform two concurrent accesses to the same location from different @@ -130,123 +131,130 @@ //! //! [`null()`]: null //! -//! # Strict Provenance -//! -//! **The following text is non-normative, insufficiently formal, and is an extremely strict -//! interpretation of provenance. It's ok if your code doesn't strictly conform to it.** -//! -//! [Strict Provenance][] is an experimental set of APIs that help tools that try -//! to validate the memory-safety of your program's execution. Notably this includes [Miri][] -//! and [CHERI][], which can detect when you access out of bounds memory or otherwise violate -//! Rust's memory model. -//! -//! Provenance must exist in some form for any programming -//! language compiled for modern computer architectures, but specifying a model for provenance -//! in a way that is useful to both compilers and programmers is an ongoing challenge. -//! The [Strict Provenance][] experiment seeks to explore the question: *what if we just said you -//! couldn't do all the nasty operations that make provenance so messy?* -//! -//! What APIs would have to be removed? What APIs would have to be added? How much would code -//! have to change, and is it worse or better now? Would any patterns become truly inexpressible? -//! Could we carve out special exceptions for those patterns? Should we? -//! -//! A secondary goal of this project is to see if we can disambiguate the many functions of -//! pointer<->integer casts enough for the definition of `usize` to be loosened so that it -//! isn't *pointer*-sized but address-space/offset/allocation-sized (we'll probably continue -//! to conflate these notions). This would potentially make it possible to more efficiently -//! target platforms where pointers are larger than offsets, such as CHERI and maybe some -//! segmented architectures. -//! -//! ## Provenance -//! -//! **This section is *non-normative* and is part of the [Strict Provenance][] experiment.** +//! # Provenance //! //! Pointers are not *simply* an "integer" or "address". For instance, it's uncontroversial -//! to say that a Use After Free is clearly Undefined Behaviour, even if you "get lucky" +//! to say that a Use After Free is clearly Undefined Behavior, even if you "get lucky" //! and the freed memory gets reallocated before your read/write (in fact this is the //! worst-case scenario, UAFs would be much less concerning if this didn't happen!). -//! To rationalize this claim, pointers need to somehow be *more* than just their addresses: -//! they must have provenance. +//! As another example, consider that [`wrapping_offset`] is documented to "remember" +//! the allocated object that the original pointer points to, even if it is offset far +//! outside the memory range occupied by that allocated object. +//! To rationalize claims like this, pointers need to somehow be *more* than just their addresses: +//! they must have **provenance**. //! -//! When an allocation is created, that allocation has a unique Original Pointer. For alloc -//! APIs this is literally the pointer the call returns, and for local variables and statics, -//! this is the name of the variable/static. This is mildly overloading the term "pointer" -//! for the sake of brevity/exposition. +//! A pointer value in Rust semantically contains the following information: //! -//! The Original Pointer for an allocation is guaranteed to have unique access to the entire -//! allocation and *only* that allocation. In this sense, an allocation can be thought of -//! as a "sandbox" that cannot be broken into or out of. *Provenance* is the permission -//! to access an allocation's sandbox and has both a *spatial* and *temporal* component: +//! * The **address** it points to, which can be represented by a `usize`. +//! * The **provenance** it has, defining the memory it has permission to access. Provenance can be +//! absent, in which case the pointer does not have permission to access any memory. //! -//! * Spatial: A range of bytes that the pointer is allowed to access. -//! * Temporal: The lifetime (of the allocation) that access to these bytes is tied to. +//! The exact structure of provenance is not yet specified, but the permission defined by a +//! pointer's provenance have a *spatial* component, a *temporal* component, and a *mutability* +//! component: //! -//! Spatial provenance makes sure you don't go beyond your sandbox, while temporal provenance -//! makes sure that you can't "get lucky" after your permission to access some memory -//! has been revoked (either through deallocations or borrows expiring). +//! * Spatial: The set of memory addresses that the pointer is allowed to access. +//! * Temporal: The timespan during which the pointer is allowed to access those memory addresses. +//! * Mutability: Whether the pointer may only access the memory for reads, or also access it for +//! writes. Note that this can interact with the other components, e.g. a pointer might permit +//! mutation only for a subset of addresses, or only for a subset of its maximal timespan. //! -//! Provenance is implicitly shared with all pointers transitively derived from -//! The Original Pointer through operations like [`offset`], borrowing, and pointer casts. -//! Some operations may *shrink* the derived provenance, limiting how much memory it can -//! access or how long it's valid for (i.e. borrowing a subfield and subslicing). +//! When an [allocated object] is created, it has a unique Original Pointer. For alloc +//! APIs this is literally the pointer the call returns, and for local variables and statics, +//! this is the name of the variable/static. (This is mildly overloading the term "pointer" +//! for the sake of brevity/exposition.) +//! +//! The Original Pointer for an allocated object has provenance that constrains the *spatial* +//! permissions of this pointer to the memory range of the allocation, and the *temporal* +//! permissions to the lifetime of the allocation. Provenance is implicitly inherited by all +//! pointers transitively derived from the Original Pointer through operations like [`offset`], +//! borrowing, and pointer casts. Some operations may *shrink* the permissions of the derived +//! provenance, limiting how much memory it can access or how long it's valid for (i.e. borrowing a +//! subfield and subslicing can shrink the spatial component of provenance, and all borrowing can +//! shrink the temporal component of provenance). However, no operation can ever *grow* the +//! permissions of the derived provenance: even if you "know" there is a larger allocation, you +//! can't derive a pointer with a larger provenance. Similarly, you cannot "recombine" two +//! contiguous provenances back into one (i.e. with a `fn merge(&[T], &[T]) -> &[T]`). +//! +//! A reference to a place always has provenance over at least the memory that place occupies. +//! A reference to a slice always has provenance over at least the range that slice describes. +//! Whether and when exactly the provenance of a reference gets "shrunk" to *exactly* fit +//! the memory it points to is not yet determined. +//! +//! A *shared* reference only ever has provenance that permits reading from memory, +//! and never permits writes, except inside [`UnsafeCell`]. +//! +//! Provenance can affect whether a program has undefined behavior: +//! +//! * It is undefined behavior to access memory through a pointer that does not have provenance over +//! that memory. Note that a pointer "at the end" of its provenance is not actually outside its +//! provenance, it just has 0 bytes it can load/store. Zero-sized accesses do not require any +//! provenance since they access an empty range of memory. +//! +//! * It is undefined behavior to [`offset`] a pointer across a memory range that is not contained +//! in the allocated object it is derived from, or to [`offset_from`] two pointers not derived +//! from the same allocated object. Provenance is used to say what exactly "derived from" even +//! means: the lineage of a pointer is traced back to the Original Pointer it descends from, and +//! that identifies the relevant allocated object. In particular, it's always UB to offset a +//! pointer derived from something that is now deallocated, except if the offset is 0. //! -//! Shrinking provenance cannot be undone: even if you "know" there is a larger allocation, you -//! can't derive a pointer with a larger provenance. Similarly, you cannot "recombine" -//! two contiguous provenances back into one (i.e. with a `fn merge(&[T], &[T]) -> &[T]`). +//! But it *is* still sound to: //! -//! A reference to a value always has provenance over exactly the memory that field occupies. -//! A reference to a slice always has provenance over exactly the range that slice describes. +//! * Create a pointer without provenance from just an address (see [`ptr::dangling`]). Such a +//! pointer cannot be used for memory accesses (except for zero-sized accesses). This can still be +//! useful for sentinel values like `null` *or* to represent a tagged pointer that will never be +//! dereferenceable. In general, it is always sound for an integer to pretend to be a pointer "for +//! fun" as long as you don't use operations on it which require it to be valid (non-zero-sized +//! offset, read, write, etc). //! -//! If an allocation is deallocated, all pointers with provenance to that allocation become -//! invalidated, and effectively lose their provenance. +//! * Forge an allocation of size zero at any sufficiently aligned non-null address. +//! i.e. the usual "ZSTs are fake, do what you want" rules apply. //! -//! The strict provenance experiment is mostly only interested in exploring stricter *spatial* -//! provenance. In this sense it can be thought of as a subset of the more ambitious and -//! formal [Stacked Borrows][] research project, which is what tools like [Miri][] are based on. -//! In particular, Stacked Borrows is necessary to properly describe what borrows are allowed -//! to do and when they become invalidated. This necessarily involves much more complex -//! *temporal* reasoning than simply identifying allocations. Adjusting APIs and code -//! for the strict provenance experiment will also greatly help Stacked Borrows. +//! * [`wrapping_offset`] a pointer outside its provenance. This includes pointers +//! which have "no" provenance. In particular, this makes it sound to do pointer tagging tricks. //! +//! * Compare arbitrary pointers by address. Pointer comparison ignores provenance and addresses +//! *are* just integers, so there is always a coherent answer, even if the pointers are dangling +//! or from different provenances. Note that if you get "lucky" and notice that a pointer at the +//! end of one allocated object is the "same" address as the start of another allocated object, +//! anything you do with that fact is *probably* going to be gibberish. The scope of that +//! gibberish is kept under control by the fact that the two pointers *still* aren't allowed to +//! access the other's allocation (bytes), because they still have different provenance. //! -//! ## Pointer Vs Addresses +//! Note that the full definition of provenance in Rust is not decided yet, as this interacts +//! with the as-yet undecided [aliasing] rules. //! -//! **This section is *non-normative* and is part of the [Strict Provenance][] experiment.** +//! ## Pointers Vs Integers //! -//! One of the largest historical issues with trying to define provenance is that programmers -//! freely convert between pointers and integers. Once you allow for this, it generally becomes -//! impossible to accurately track and preserve provenance information, and you need to appeal -//! to very complex and unreliable heuristics. But of course, converting between pointers and -//! integers is very useful, so what can we do? +//! From this discussion, it becomes very clear that a `usize` *cannot* accurately represent a pointer, +//! and converting from a pointer to a `usize` is generally an operation which *only* extracts the +//! address. Converting this address back into pointer requires somehow answering the question: +//! which provenance should the resulting pointer have? //! -//! Also did you know WASM is actually a "Harvard Architecture"? As in function pointers are -//! handled completely differently from data pointers? And we kind of just shipped Rust on WASM -//! without really addressing the fact that we let you freely convert between function pointers -//! and data pointers, because it mostly Just Works? Let's just put that on the "pointer casts -//! are dubious" pile. +//! Rust provides two ways of dealing with this situation: *Strict Provenance* and *Exposed Provenance*. //! -//! Strict Provenance attempts to square these circles by decoupling Rust's traditional conflation -//! of pointers and `usize` (and `isize`), and defining a pointer to semantically contain the -//! following information: +//! Note that a pointer *can* represent a `usize` (via [`without_provenance`]), so the right type to +//! use in situations where a value is "sometimes a pointer and sometimes a bare `usize`" is a +//! pointer type. //! -//! * The **address-space** it is part of (e.g. "data" vs "code" in WASM). -//! * The **address** it points to, which can be represented by a `usize`. -//! * The **provenance** it has, defining the memory it has permission to access. -//! Provenance can be absent, in which case the pointer does not have permission to access any memory. +//! ## Strict Provenance +//! +//! "Strict Provenance" refers to a set of APIs designed to make working with provenance more +//! explicit. They are intended as substitutes for casting a pointer to an integer and back. //! -//! Under Strict Provenance, a `usize` *cannot* accurately represent a pointer, and converting from -//! a pointer to a `usize` is generally an operation which *only* extracts the address. It is -//! therefore *impossible* to construct a valid pointer from a `usize` because there is no way -//! to restore the address-space and provenance. In other words, pointer-integer-pointer -//! roundtrips are not possible (in the sense that the resulting pointer is not dereferenceable). +//! Entirely avoiding integer-to-pointer casts successfully side-steps the inherent ambiguity of +//! that operation. This benefits compiler optimizations, and it is pretty much a requirement for +//! using tools like [Miri] and architectures like [CHERI] that aim to detect and diagnose pointer +//! misuse. //! -//! The key insight to making this model *at all* viable is the [`with_addr`][] method: +//! The key insight to making programming without integer-to-pointer casts *at all* viable is the +//! [`with_addr`] method: //! //! ```text //! /// Creates a new pointer with the given address. //! /// //! /// This performs the same operation as an `addr as ptr` cast, but copies -//! /// the *address-space* and *provenance* of `self` to the new pointer. +//! /// the *provenance* of `self` to the new pointer. //! /// This allows us to dynamically preserve and propagate this important //! /// information in a way that is otherwise impossible with a unary cast. //! /// @@ -257,23 +265,21 @@ //! //! So you're still able to drop down to the address representation and do whatever //! clever bit tricks you want *as long as* you're able to keep around a pointer -//! into the allocation you care about that can "reconstitute" the other parts of the pointer. +//! into the allocation you care about that can "reconstitute" the provenance. //! Usually this is very easy, because you only are taking a pointer, messing with the address, //! and then immediately converting back to a pointer. To make this use case more ergonomic, -//! we provide the [`map_addr`][] method. +//! we provide the [`map_addr`] method. //! //! To help make it clear that code is "following" Strict Provenance semantics, we also provide an -//! [`addr`][] method which promises that the returned address is not part of a -//! pointer-usize-pointer roundtrip. In the future we may provide a lint for pointer<->integer +//! [`addr`] method which promises that the returned address is not part of a +//! pointer-integer-pointer roundtrip. In the future we may provide a lint for pointer<->integer //! casts to help you audit if your code conforms to strict provenance. //! -//! -//! ## Using Strict Provenance +//! ### Using Strict Provenance //! //! Most code needs no changes to conform to strict provenance, as the only really concerning -//! operation that *wasn't* obviously already Undefined Behaviour is casts from usize to a -//! pointer. For code which *does* cast a `usize` to a pointer, the scope of the change depends -//! on exactly what you're doing. +//! operation is casts from usize to a pointer. For code which *does* cast a `usize` to a pointer, +//! the scope of the change depends on exactly what you're doing. //! //! In general, you just need to make sure that if you want to convert a `usize` address to a //! pointer and then use that pointer to read/write memory, you need to keep around a pointer @@ -284,8 +290,6 @@ //! represent the tagged pointer as an actual pointer and not a `usize`*. For instance: //! //! ``` -//! #![feature(strict_provenance)] -//! //! unsafe { //! // A flag we want to pack into our pointer //! static HAS_DATA: usize = 0x1; @@ -314,122 +318,65 @@ //! be using AtomicPtr instead. If that messes up the way you atomically manipulate pointers, //! we would like to know why, and what needs to be done to fix it.) //! -//! Something more complicated and just generally *evil* like an XOR-List requires more significant -//! changes like allocating all nodes in a pre-allocated Vec or Arena and using a pointer -//! to the whole allocation to reconstitute the XORed addresses. -//! //! Situations where a valid pointer *must* be created from just an address, such as baremetal code -//! accessing a memory-mapped interface at a fixed address, are an open question on how to support. -//! These situations *will* still be allowed, but we might require some kind of "I know what I'm -//! doing" annotation to explain the situation to the compiler. It's also possible they need no -//! special attention at all, because they're generally accessing memory outside the scope of -//! "the abstract machine", or already using "I know what I'm doing" annotations like "volatile". -//! -//! Under [Strict Provenance] it is Undefined Behaviour to: -//! -//! * Access memory through a pointer that does not have provenance over that memory. -//! -//! * [`offset`] a pointer to or from an address it doesn't have provenance over. -//! This means it's always UB to offset a pointer derived from something deallocated, -//! even if the offset is 0. Note that a pointer "one past the end" of its provenance -//! is not actually outside its provenance, it just has 0 bytes it can load/store. -//! -//! But it *is* still sound to: -//! -//! * Create a pointer without provenance from just an address (see [`ptr::dangling`][]). Such a -//! pointer cannot be used for memory accesses (except for zero-sized accesses). This can still be -//! useful for sentinel values like `null` *or* to represent a tagged pointer that will never be -//! dereferenceable. In general, it is always sound for an integer to pretend to be a pointer "for -//! fun" as long as you don't use operations on it which require it to be valid (non-zero-sized -//! offset, read, write, etc). -//! -//! * Forge an allocation of size zero at any sufficiently aligned non-null address. -//! i.e. the usual "ZSTs are fake, do what you want" rules apply *but* this only applies -//! for actual forgery (integers cast to pointers). If you borrow some struct's field -//! that *happens* to be zero-sized, the resulting pointer will have provenance tied to -//! that allocation, and it will still get invalidated if the allocation gets deallocated. -//! In the future we may introduce an API to make such a forged allocation explicit. -//! -//! * [`wrapping_offset`][] a pointer outside its provenance. This includes pointers -//! which have "no" provenance. Unfortunately there may be practical limits on this for a -//! particular platform, and it's an open question as to how to specify this (if at all). -//! Notably, [CHERI][] relies on a compression scheme that can't handle a -//! pointer getting offset "too far" out of bounds. If this happens, the address -//! returned by `addr` will be the value you expect, but the provenance will get invalidated -//! and using it to read/write will fault. The details of this are architecture-specific -//! and based on alignment, but the buffer on either side of the pointer's range is pretty -//! generous (think kilobytes, not bytes). -//! -//! * Compare arbitrary pointers by address. Addresses *are* just integers and so there is -//! always a coherent answer, even if the pointers are dangling or from different -//! address-spaces/provenances. Of course, comparing addresses from different address-spaces -//! is generally going to be *meaningless*, but so is comparing Kilograms to Meters, and Rust -//! doesn't prevent that either. Similarly, if you get "lucky" and notice that a pointer -//! one-past-the-end is the "same" address as the start of an unrelated allocation, anything -//! you do with that fact is *probably* going to be gibberish. The scope of that gibberish -//! is kept under control by the fact that the two pointers *still* aren't allowed to access -//! the other's allocation (bytes), because they still have different provenance. -//! -//! * Perform pointer tagging tricks. This falls out of [`wrapping_offset`] but is worth -//! mentioning in more detail because of the limitations of [CHERI][]. Low-bit tagging -//! is very robust, and often doesn't even go out of bounds because types ensure -//! size >= align (and over-aligning actually gives CHERI more flexibility). Anything -//! more complex than this rapidly enters "extremely platform-specific" territory as -//! certain things may or may not be allowed based on specific supported operations. -//! For instance, ARM explicitly supports high-bit tagging, and so CHERI on ARM inherits -//! that and should support it. +//! accessing a memory-mapped interface at a fixed address, cannot currently be handled with strict +//! provenance APIs and should use [exposed provenance](#exposed-provenance). //! //! ## Exposed Provenance //! -//! **This section is *non-normative* and is an extension to the [Strict Provenance] experiment.** -//! -//! As discussed above, pointer-usize-pointer roundtrips are not possible under [Strict Provenance]. +//! As discussed above, integer-to-pointer casts are not possible with Strict Provenance APIs. //! This is by design: the goal of Strict Provenance is to provide a clear specification that we are -//! confident can be formalized unambiguously and can be subject to precise formal reasoning. +//! confident can be formalized unambiguously and can be subject to precise formal reasoning. +//! Integer-to-pointer casts do not (currently) have such a clear specification. //! -//! However, there exist situations where pointer-usize-pointer roundtrips cannot be avoided, or +//! However, there exist situations where integer-to-pointer casts cannot be avoided, or //! where avoiding them would require major refactoring. Legacy platform APIs also regularly assume -//! that `usize` can capture all the information that makes up a pointer. The goal of Strict -//! Provenance is not to rule out such code; the goal is to put all the *other* pointer-manipulating -//! code onto a more solid foundation. Strict Provenance is about improving the situation where -//! possible (all the code that can be written with Strict Provenance) without making things worse -//! for situations where Strict Provenance is insufficient. -//! -//! For these situations, there is a highly experimental extension to Strict Provenance called -//! *Exposed Provenance*. This extension permits pointer-usize-pointer roundtrips. However, its -//! semantics are on much less solid footing than Strict Provenance, and at this point it is not yet -//! clear where a satisfying unambiguous semantics can be defined for Exposed Provenance. -//! Furthermore, Exposed Provenance will not work (well) with tools like [Miri] and [CHERI]. +//! that `usize` can capture all the information that makes up a pointer. +//! Bare-metal platforms can also require the synthesis of a pointer "out of thin air" without +//! anywhere to obtain proper provenance from. +//! +//! Rust's model for dealing with integer-to-pointer casts is called *Exposed Provenance*. However, +//! the semantics of Exposed Provenance are on much less solid footing than Strict Provenance, and +//! at this point it is not yet clear whether a satisfying unambiguous semantics can be defined for +//! Exposed Provenance. (If that sounds bad, be reassured that other popular languages that provide +//! integer-to-pointer casts are not faring any better.) Furthermore, Exposed Provenance will not +//! work (well) with tools like [Miri] and [CHERI]. //! //! Exposed Provenance is provided by the [`expose_provenance`] and [`with_exposed_provenance`] methods, -//! which are meant to replace `as` casts between pointers and integers. [`expose_provenance`] is a lot like -//! [`addr`], but additionally adds the provenance of the pointer to a global list of 'exposed' -//! provenances. (This list is purely conceptual, it exists for the purpose of specifying Rust but -//! is not materialized in actual executions, except in tools like [Miri].) [`with_exposed_provenance`] -//! can be used to construct a pointer with one of these previously 'exposed' provenances. -//! [`with_exposed_provenance`] takes only `addr: usize` as arguments, so unlike in [`with_addr`] there is -//! no indication of what the correct provenance for the returned pointer is -- and that is exactly -//! what makes pointer-usize-pointer roundtrips so tricky to rigorously specify! There is no -//! algorithm that decides which provenance will be used. You can think of this as "guessing" the -//! right provenance, and the guess will be "maximally in your favor", in the sense that if there is -//! any way to avoid undefined behavior, then that is the guess that will be taken. However, if -//! there is *no* previously 'exposed' provenance that justifies the way the returned pointer will -//! be used, the program has undefined behavior. -//! -//! Using [`expose_provenance`] or [`with_exposed_provenance`] (or the `as` casts) means that code is -//! *not* following Strict Provenance rules. The goal of the Strict Provenance experiment is to -//! determine how far one can get in Rust without the use of [`expose_provenance`] and -//! [`with_exposed_provenance`], and to encourage code to be written with Strict Provenance APIs only. -//! Maximizing the amount of such code is a major win for avoiding specification complexity and to -//! facilitate adoption of tools like [CHERI] and [Miri] that can be a big help in increasing the -//! confidence in (unsafe) Rust code. +//! which are equivalent to `as` casts between pointers and integers. +//! - [`expose_provenance`] is a lot like [`addr`], but additionally adds the provenance of the +//! pointer to a global list of 'exposed' provenances. (This list is purely conceptual, it exists +//! for the purpose of specifying Rust but is not materialized in actual executions, except in +//! tools like [Miri].) +//! Memory which is outside the control of the Rust abstract machine (MMIO registers, for example) +//! is always considered to be exposed, so long as this memory is disjoint from memory that will +//! be used by the abstract machine such as the stack, heap, and statics. +//! - [`with_exposed_provenance`] can be used to construct a pointer with one of these previously +//! 'exposed' provenances. [`with_exposed_provenance`] takes only `addr: usize` as arguments, so +//! unlike in [`with_addr`] there is no indication of what the correct provenance for the returned +//! pointer is -- and that is exactly what makes integer-to-pointer casts so tricky to rigorously +//! specify! The compiler will do its best to pick the right provenance for you, but currently we +//! cannot provide any guarantees about which provenance the resulting pointer will have. Only one +//! thing is clear: if there is *no* previously 'exposed' provenance that justifies the way the +//! returned pointer will be used, the program has undefined behavior. +//! +//! If at all possible, we encourage code to be ported to [Strict Provenance] APIs, thus avoiding +//! the need for Exposed Provenance. Maximizing the amount of such code is a major win for avoiding +//! specification complexity and to facilitate adoption of tools like [CHERI] and [Miri] that can be +//! a big help in increasing the confidence in (unsafe) Rust code. However, we acknowledge that this +//! is not always possible, and offer Exposed Provenance as a way to explicit "opt out" of the +//! well-defined semantics of Strict Provenance, and "opt in" to the unclear semantics of +//! integer-to-pointer casts. //! //! [aliasing]: ../../nomicon/aliasing.html +//! [allocated object]: #allocated-object +//! [provenance]: #provenance //! [book]: ../../book/ch19-01-unsafe-rust.html#dereferencing-a-raw-pointer //! [ub]: ../../reference/behavior-considered-undefined.html //! [zst]: ../../nomicon/exotic-sizes.html#zero-sized-types-zsts //! [atomic operations]: crate::sync::atomic //! [`offset`]: pointer::offset +//! [`offset_from`]: pointer::offset_from //! [`wrapping_offset`]: pointer::wrapping_offset //! [`with_addr`]: pointer::with_addr //! [`map_addr`]: pointer::map_addr @@ -439,8 +386,8 @@ //! [`with_exposed_provenance`]: with_exposed_provenance //! [Miri]: https://github.com/rust-lang/miri //! [CHERI]: https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/ -//! [Strict Provenance]: https://github.com/rust-lang/rust/issues/95228 -//! [Stacked Borrows]: https://plv.mpi-sws.org/rustbelt/stacked-borrows/ +//! [Strict Provenance]: #strict-provenance +//! [`UnsafeCell`]: core::cell::UnsafeCell #![stable(feature = "rust1", since = "1.0.0")] // There are many unsafe functions taking pointers that don't dereference them. @@ -448,7 +395,7 @@ use crate::cmp::Ordering; use crate::marker::FnPtr; -use crate::mem::{self, MaybeUninit}; +use crate::mem::{self, MaybeUninit, SizedTypeProperties}; use crate::{fmt, hash, intrinsics, ub_checks}; #[cfg(kani)] @@ -470,7 +417,7 @@ pub use crate::intrinsics::write_bytes; mod metadata; #[unstable(feature = "ptr_metadata", issue = "81513")] -pub use metadata::{from_raw_parts, from_raw_parts_mut, metadata, DynMetadata, Pointee, Thin}; +pub use metadata::{DynMetadata, Pointee, Thin, from_raw_parts, from_raw_parts_mut, metadata}; mod non_null; #[stable(feature = "nonnull", since = "1.25.0")] @@ -602,7 +549,6 @@ pub unsafe fn drop_in_place(to_drop: *mut T) { #[stable(feature = "rust1", since = "1.0.0")] #[rustc_promotable] #[rustc_const_stable(feature = "const_ptr_null", since = "1.24.0")] -#[rustc_allow_const_fn_unstable(ptr_metadata)] #[rustc_diagnostic_item = "ptr_null"] pub const fn null() -> *const T { from_raw_parts(without_provenance::<()>(0), ()) @@ -628,13 +574,12 @@ pub const fn null() -> *const T { #[stable(feature = "rust1", since = "1.0.0")] #[rustc_promotable] #[rustc_const_stable(feature = "const_ptr_null", since = "1.24.0")] -#[rustc_allow_const_fn_unstable(ptr_metadata)] #[rustc_diagnostic_item = "ptr_null_mut"] pub const fn null_mut() -> *mut T { from_raw_parts_mut(without_provenance_mut::<()>(0), ()) } -/// Creates a pointer with the given address and no provenance. +/// Creates a pointer with the given address and no [provenance][crate::ptr#provenance]. /// /// This is equivalent to `ptr::null().with_addr(addr)`. /// @@ -646,22 +591,21 @@ pub const fn null_mut() -> *mut T { /// This is different from `addr as *const T`, which creates a pointer that picks up a previously /// exposed provenance. See [`with_exposed_provenance`] for more details on that operation. /// -/// This API and its claimed semantics are part of the Strict Provenance experiment, -/// see the [module documentation][crate::ptr] for details. +/// This is a [Strict Provenance][crate::ptr#strict-provenance] API. #[inline(always)] #[must_use] -#[rustc_const_stable(feature = "stable_things_using_strict_provenance", since = "1.61.0")] -#[unstable(feature = "strict_provenance", issue = "95228")] +#[stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] +#[rustc_const_stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] pub const fn without_provenance(addr: usize) -> *const T { - // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic. - // We use transmute rather than a cast so tools like Miri can tell that this - // is *not* the same as with_exposed_provenance. + // An int-to-pointer transmute currently has exactly the intended semantics: it creates a + // pointer without provenance. Note that this is *not* a stable guarantee about transmute + // semantics, it relies on sysroot crates having special status. // SAFETY: every valid integer is also a valid pointer (as long as you don't dereference that // pointer). unsafe { mem::transmute(addr) } } -/// Creates a new pointer that is dangling, but well-aligned. +/// Creates a new pointer that is dangling, but non-null and well-aligned. /// /// This is useful for initializing types which lazily allocate, like /// `Vec::new` does. @@ -672,13 +616,13 @@ pub const fn without_provenance(addr: usize) -> *const T { /// some other means. #[inline(always)] #[must_use] -#[rustc_const_stable(feature = "stable_things_using_strict_provenance", since = "1.61.0")] -#[unstable(feature = "strict_provenance", issue = "95228")] +#[stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] +#[rustc_const_stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] pub const fn dangling() -> *const T { without_provenance(mem::align_of::()) } -/// Creates a pointer with the given address and no provenance. +/// Creates a pointer with the given address and no [provenance][crate::ptr#provenance]. /// /// This is equivalent to `ptr::null_mut().with_addr(addr)`. /// @@ -690,22 +634,21 @@ pub const fn dangling() -> *const T { /// This is different from `addr as *mut T`, which creates a pointer that picks up a previously /// exposed provenance. See [`with_exposed_provenance_mut`] for more details on that operation. /// -/// This API and its claimed semantics are part of the Strict Provenance experiment, -/// see the [module documentation][crate::ptr] for details. +/// This is a [Strict Provenance][crate::ptr#strict-provenance] API. #[inline(always)] #[must_use] -#[rustc_const_stable(feature = "stable_things_using_strict_provenance", since = "1.61.0")] -#[unstable(feature = "strict_provenance", issue = "95228")] +#[stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] +#[rustc_const_stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] pub const fn without_provenance_mut(addr: usize) -> *mut T { - // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic. - // We use transmute rather than a cast so tools like Miri can tell that this - // is *not* the same as with_exposed_provenance. + // An int-to-pointer transmute currently has exactly the intended semantics: it creates a + // pointer without provenance. Note that this is *not* a stable guarantee about transmute + // semantics, it relies on sysroot crates having special status. // SAFETY: every valid integer is also a valid pointer (as long as you don't dereference that // pointer). unsafe { mem::transmute(addr) } } -/// Creates a new pointer that is dangling, but well-aligned. +/// Creates a new pointer that is dangling, but non-null and well-aligned. /// /// This is useful for initializing types which lazily allocate, like /// `Vec::new` does. @@ -716,97 +659,89 @@ pub const fn without_provenance_mut(addr: usize) -> *mut T { /// some other means. #[inline(always)] #[must_use] -#[rustc_const_stable(feature = "stable_things_using_strict_provenance", since = "1.61.0")] -#[unstable(feature = "strict_provenance", issue = "95228")] +#[stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] +#[rustc_const_stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] pub const fn dangling_mut() -> *mut T { without_provenance_mut(mem::align_of::()) } -/// Converts an address back to a pointer, picking up a previously 'exposed' provenance. -/// -/// This is a more rigorously specified alternative to `addr as *const T`. The provenance of the -/// returned pointer is that of *any* pointer that was previously exposed by passing it to -/// [`expose_provenance`][pointer::expose_provenance], or a `ptr as usize` cast. In addition, memory which is -/// outside the control of the Rust abstract machine (MMIO registers, for example) is always -/// considered to be exposed, so long as this memory is disjoint from memory that will be used by -/// the abstract machine such as the stack, heap, and statics. +/// Converts an address back to a pointer, picking up some previously 'exposed' +/// [provenance][crate::ptr#provenance]. /// -/// If there is no 'exposed' provenance that justifies the way this pointer will be used, -/// the program has undefined behavior. In particular, the aliasing rules still apply: pointers -/// and references that have been invalidated due to aliasing accesses cannot be used anymore, -/// even if they have been exposed! +/// This is fully equivalent to `addr as *const T`. The provenance of the returned pointer is that +/// of *some* pointer that was previously exposed by passing it to +/// [`expose_provenance`][pointer::expose_provenance], or a `ptr as usize` cast. In addition, memory +/// which is outside the control of the Rust abstract machine (MMIO registers, for example) is +/// always considered to be accessible with an exposed provenance, so long as this memory is disjoint +/// from memory that will be used by the abstract machine such as the stack, heap, and statics. /// -/// Note that there is no algorithm that decides which provenance will be used. You can think of this -/// as "guessing" the right provenance, and the guess will be "maximally in your favor", in the sense -/// that if there is any way to avoid undefined behavior (while upholding all aliasing requirements), -/// then that is the guess that will be taken. +/// The exact provenance that gets picked is not specified. The compiler will do its best to pick +/// the "right" provenance for you (whatever that may be), but currently we cannot provide any +/// guarantees about which provenance the resulting pointer will have -- and therefore there +/// is no definite specification for which memory the resulting pointer may access. /// -/// On platforms with multiple address spaces, it is your responsibility to ensure that the -/// address makes sense in the address space that this pointer will be used with. +/// If there is *no* previously 'exposed' provenance that justifies the way the returned pointer +/// will be used, the program has undefined behavior. In particular, the aliasing rules still apply: +/// pointers and references that have been invalidated due to aliasing accesses cannot be used +/// anymore, even if they have been exposed! /// -/// Using this function means that code is *not* following [Strict -/// Provenance][self#strict-provenance] rules. "Guessing" a -/// suitable provenance complicates specification and reasoning and may not be supported by -/// tools that help you to stay conformant with the Rust memory model, so it is recommended to -/// use [`with_addr`][pointer::with_addr] wherever possible. +/// Due to its inherent ambiguity, this operation may not be supported by tools that help you to +/// stay conformant with the Rust memory model. It is recommended to use [Strict +/// Provenance][self#strict-provenance] APIs such as [`with_addr`][pointer::with_addr] wherever +/// possible. /// /// On most platforms this will produce a value with the same bytes as the address. Platforms /// which need to store additional information in a pointer may not support this operation, /// since it is generally not possible to actually *compute* which provenance the returned /// pointer has to pick up. /// -/// It is unclear whether this function can be given a satisfying unambiguous specification. This -/// API and its claimed semantics are part of [Exposed Provenance][self#exposed-provenance]. +/// This is an [Exposed Provenance][crate::ptr#exposed-provenance] API. #[must_use] #[inline(always)] -#[unstable(feature = "exposed_provenance", issue = "95228")] +#[stable(feature = "exposed_provenance", since = "CURRENT_RUSTC_VERSION")] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[allow(fuzzy_provenance_casts)] // this *is* the explicit provenance API one should use instead -pub fn with_exposed_provenance(addr: usize) -> *const T -where - T: Sized, -{ - // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic. +pub fn with_exposed_provenance(addr: usize) -> *const T { addr as *const T } -/// Converts an address back to a mutable pointer, picking up a previously 'exposed' provenance. +/// Converts an address back to a mutable pointer, picking up some previously 'exposed' +/// [provenance][crate::ptr#provenance]. /// -/// This is a more rigorously specified alternative to `addr as *mut T`. The provenance of the -/// returned pointer is that of *any* pointer that was previously passed to -/// [`expose_provenance`][pointer::expose_provenance] or a `ptr as usize` cast. If there is no previously -/// 'exposed' provenance that justifies the way this pointer will be used, the program has undefined -/// behavior. Note that there is no algorithm that decides which provenance will be used. You can -/// think of this as "guessing" the right provenance, and the guess will be "maximally in your -/// favor", in the sense that if there is any way to avoid undefined behavior, then that is the -/// guess that will be taken. +/// This is fully equivalent to `addr as *mut T`. The provenance of the returned pointer is that +/// of *some* pointer that was previously exposed by passing it to +/// [`expose_provenance`][pointer::expose_provenance], or a `ptr as usize` cast. In addition, memory +/// which is outside the control of the Rust abstract machine (MMIO registers, for example) is +/// always considered to be accessible with an exposed provenance, so long as this memory is disjoint +/// from memory that will be used by the abstract machine such as the stack, heap, and statics. /// -/// On platforms with multiple address spaces, it is your responsibility to ensure that the -/// address makes sense in the address space that this pointer will be used with. +/// The exact provenance that gets picked is not specified. The compiler will do its best to pick +/// the "right" provenance for you (whatever that may be), but currently we cannot provide any +/// guarantees about which provenance the resulting pointer will have -- and therefore there +/// is no definite specification for which memory the resulting pointer may access. /// -/// Using this function means that code is *not* following [Strict -/// Provenance][self#strict-provenance] rules. "Guessing" a -/// suitable provenance complicates specification and reasoning and may not be supported by -/// tools that help you to stay conformant with the Rust memory model, so it is recommended to -/// use [`with_addr`][pointer::with_addr] wherever possible. +/// If there is *no* previously 'exposed' provenance that justifies the way the returned pointer +/// will be used, the program has undefined behavior. In particular, the aliasing rules still apply: +/// pointers and references that have been invalidated due to aliasing accesses cannot be used +/// anymore, even if they have been exposed! +/// +/// Due to its inherent ambiguity, this operation may not be supported by tools that help you to +/// stay conformant with the Rust memory model. It is recommended to use [Strict +/// Provenance][self#strict-provenance] APIs such as [`with_addr`][pointer::with_addr] wherever +/// possible. /// /// On most platforms this will produce a value with the same bytes as the address. Platforms /// which need to store additional information in a pointer may not support this operation, /// since it is generally not possible to actually *compute* which provenance the returned /// pointer has to pick up. /// -/// It is unclear whether this function can be given a satisfying unambiguous specification. This -/// API and its claimed semantics are part of [Exposed Provenance][self#exposed-provenance]. +/// This is an [Exposed Provenance][crate::ptr#exposed-provenance] API. #[must_use] #[inline(always)] -#[unstable(feature = "exposed_provenance", issue = "95228")] +#[stable(feature = "exposed_provenance", since = "CURRENT_RUSTC_VERSION")] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[allow(fuzzy_provenance_casts)] // this *is* the explicit provenance API one should use instead -pub fn with_exposed_provenance_mut(addr: usize) -> *mut T -where - T: Sized, -{ - // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic. +pub fn with_exposed_provenance_mut(addr: usize) -> *mut T { addr as *mut T } @@ -912,7 +847,6 @@ pub const fn from_ref(r: &T) -> *const T { #[must_use] #[stable(feature = "ptr_from_ref", since = "1.76.0")] #[rustc_const_stable(feature = "ptr_from_ref", since = "1.76.0")] -#[rustc_allow_const_fn_unstable(const_mut_refs)] #[rustc_never_returns_null_ptr] pub const fn from_mut(r: &mut T) -> *mut T { r @@ -952,7 +886,6 @@ pub const fn from_mut(r: &mut T) -> *mut T { #[inline] #[stable(feature = "slice_from_raw_parts", since = "1.42.0")] #[rustc_const_stable(feature = "const_slice_from_raw_parts", since = "1.64.0")] -#[rustc_allow_const_fn_unstable(ptr_metadata)] #[rustc_diagnostic_item = "ptr_slice_from_raw_parts"] pub const fn slice_from_raw_parts(data: *const T, len: usize) -> *const [T] { from_raw_parts(data, len) @@ -998,7 +931,7 @@ pub const fn slice_from_raw_parts(data: *const T, len: usize) -> *const [T] { /// ``` #[inline] #[stable(feature = "slice_from_raw_parts", since = "1.42.0")] -#[rustc_const_unstable(feature = "const_slice_from_raw_parts_mut", issue = "67456")] +#[rustc_const_stable(feature = "const_slice_from_raw_parts_mut", since = "1.83.0")] #[rustc_diagnostic_item = "ptr_slice_from_raw_parts_mut"] pub const fn slice_from_raw_parts_mut(data: *mut T, len: usize) -> *mut [T] { from_raw_parts_mut(data, len) @@ -1030,7 +963,7 @@ pub const fn slice_from_raw_parts_mut(data: *mut T, len: usize) -> *mut [T] { /// /// * Both `x` and `y` must be properly aligned. /// -/// Note that even if `T` has size `0`, the pointers must be non-null and properly aligned. +/// Note that even if `T` has size `0`, the pointers must be properly aligned. /// /// [valid]: self#safety /// @@ -1116,7 +1049,7 @@ pub const unsafe fn swap(x: *mut T, y: *mut T) { /// beginning at `y` with the same size. /// /// Note that even if the effectively copied size (`count * size_of::()`) is `0`, -/// the pointers must be non-null and properly aligned. +/// the pointers must be properly aligned. /// /// [valid]: self#safety /// @@ -1171,10 +1104,12 @@ pub const unsafe fn swap_nonoverlapping(x: *mut T, y: *mut T, count: usize) { size: usize = size_of::(), align: usize = align_of::(), count: usize = count, - ) => - ub_checks::is_aligned_and_not_null(x, align) - && ub_checks::is_aligned_and_not_null(y, align) - && ub_checks::is_nonoverlapping(x, y, size, count) + ) => { + let zero_size = size == 0 || count == 0; + ub_checks::is_aligned_and_not_null(x, align, zero_size) + && ub_checks::is_aligned_and_not_null(y, align, zero_size) + && ub_checks::is_nonoverlapping(x, y, size, count) + } ); // Split up the slice into small power-of-two-sized chunks that LLVM is able @@ -1192,7 +1127,7 @@ pub const unsafe fn swap_nonoverlapping(x: *mut T, y: *mut T, count: usize) { unsafe { swap_nonoverlapping_simple_untyped(x, y, count) } } -/// Same behaviour and safety conditions as [`swap_nonoverlapping`] +/// Same behavior and safety conditions as [`swap_nonoverlapping`] /// /// LLVM can vectorize this (at least it can for the power-of-two-sized types /// `swap_nonoverlapping` tries to use) so no need to manually SIMD it. @@ -1247,7 +1182,7 @@ const unsafe fn swap_nonoverlapping_simple_untyped(x: *mut T, y: *mut T, coun /// /// * `dst` must point to a properly initialized value of type `T`. /// -/// Note that even if `T` has size `0`, the pointer must be non-null and properly aligned. +/// Note that even if `T` has size `0`, the pointer must be properly aligned. /// /// [valid]: self#safety /// @@ -1269,7 +1204,7 @@ const unsafe fn swap_nonoverlapping_simple_untyped(x: *mut T, y: *mut T, coun /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] -#[rustc_const_unstable(feature = "const_replace", issue = "83164")] +#[rustc_const_stable(feature = "const_replace", since = "1.83.0")] #[rustc_diagnostic_item = "ptr_replace"] pub const unsafe fn replace(dst: *mut T, src: T) -> T { // SAFETY: the caller must guarantee that `dst` is valid to be @@ -1283,7 +1218,8 @@ pub const unsafe fn replace(dst: *mut T, src: T) -> T { ( addr: *const () = dst as *const (), align: usize = align_of::(), - ) => ub_checks::is_aligned_and_not_null(addr, align) + is_zst: bool = T::IS_ZST, + ) => ub_checks::is_aligned_and_not_null(addr, align, is_zst) ); mem::replace(&mut *dst, src) } @@ -1303,7 +1239,7 @@ pub const unsafe fn replace(dst: *mut T, src: T) -> T { /// /// * `src` must point to a properly initialized value of type `T`. /// -/// Note that even if `T` has size `0`, the pointer must be non-null and properly aligned. +/// Note that even if `T` has size `0`, the pointer must be properly aligned. /// /// # Examples /// @@ -1435,7 +1371,8 @@ pub const unsafe fn read(src: *const T) -> T { ( addr: *const () = src as *const (), align: usize = align_of::(), - ) => ub_checks::is_aligned_and_not_null(addr, align) + is_zst: bool = T::IS_ZST, + ) => ub_checks::is_aligned_and_not_null(addr, align, is_zst) ); crate::intrinsics::read_via_copy(src) } @@ -1519,11 +1456,6 @@ pub const unsafe fn read(src: *const T) -> T { #[inline] #[stable(feature = "ptr_unaligned", since = "1.17.0")] #[rustc_const_stable(feature = "const_ptr_read", since = "1.71.0")] -#[rustc_allow_const_fn_unstable( - const_mut_refs, - const_maybe_uninit_as_mut_ptr, - const_intrinsic_copy -)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[rustc_diagnostic_item = "ptr_read_unaligned"] pub const unsafe fn read_unaligned(src: *const T) -> T { @@ -1562,7 +1494,7 @@ pub const unsafe fn read_unaligned(src: *const T) -> T { /// * `dst` must be properly aligned. Use [`write_unaligned`] if this is not the /// case. /// -/// Note that even if `T` has size `0`, the pointer must be non-null and properly aligned. +/// Note that even if `T` has size `0`, the pointer must be properly aligned. /// /// [valid]: self#safety /// @@ -1621,7 +1553,7 @@ pub const unsafe fn read_unaligned(src: *const T) -> T { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] -#[rustc_const_unstable(feature = "const_ptr_write", issue = "86302")] +#[rustc_const_stable(feature = "const_ptr_write", since = "1.83.0")] #[rustc_diagnostic_item = "ptr_write"] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn write(dst: *mut T, src: T) { @@ -1644,7 +1576,8 @@ pub const unsafe fn write(dst: *mut T, src: T) { ( addr: *mut () = dst as *mut (), align: usize = align_of::(), - ) => ub_checks::is_aligned_and_not_null(addr, align) + is_zst: bool = T::IS_ZST, + ) => ub_checks::is_aligned_and_not_null(addr, align, is_zst) ); intrinsics::write_via_move(dst, src) } @@ -1729,7 +1662,7 @@ pub const unsafe fn write(dst: *mut T, src: T) { /// ``` #[inline] #[stable(feature = "ptr_unaligned", since = "1.17.0")] -#[rustc_const_unstable(feature = "const_ptr_write", issue = "86302")] +#[rustc_const_stable(feature = "const_ptr_write", since = "1.83.0")] #[rustc_diagnostic_item = "ptr_write_unaligned"] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn write_unaligned(dst: *mut T, src: T) { @@ -1737,7 +1670,7 @@ pub const unsafe fn write_unaligned(dst: *mut T, src: T) { // `dst` cannot overlap `src` because the caller has mutable access // to `dst` while `src` is owned by this function. unsafe { - copy_nonoverlapping(addr_of!(src) as *const u8, dst as *mut u8, mem::size_of::()); + copy_nonoverlapping((&raw const src) as *const u8, dst as *mut u8, mem::size_of::()); // We are calling the intrinsic directly to avoid function calls in the generated code. intrinsics::forget(src); } @@ -1780,7 +1713,7 @@ pub const unsafe fn write_unaligned(dst: *mut T, src: T) { /// However, storing non-[`Copy`] types in volatile memory is almost certainly /// incorrect. /// -/// Note that even if `T` has size `0`, the pointer must be non-null and properly aligned. +/// Note that even if `T` has size `0`, the pointer must be properly aligned. /// /// [valid]: self#safety /// [read-ownership]: read#ownership-of-the-returned-value @@ -1817,7 +1750,8 @@ pub unsafe fn read_volatile(src: *const T) -> T { ( addr: *const () = src as *const (), align: usize = align_of::(), - ) => ub_checks::is_aligned_and_not_null(addr, align) + is_zst: bool = T::IS_ZST, + ) => ub_checks::is_aligned_and_not_null(addr, align, is_zst) ); intrinsics::volatile_load(src) } @@ -1859,7 +1793,7 @@ pub unsafe fn read_volatile(src: *const T) -> T { /// /// * `dst` must be properly aligned. /// -/// Note that even if `T` has size `0`, the pointer must be non-null and properly aligned. +/// Note that even if `T` has size `0`, the pointer must be properly aligned. /// /// [valid]: self#safety /// @@ -1897,7 +1831,8 @@ pub unsafe fn write_volatile(dst: *mut T, src: T) { ( addr: *mut () = dst as *mut (), align: usize = align_of::(), - ) => ub_checks::is_aligned_and_not_null(addr, align) + is_zst: bool = T::IS_ZST, + ) => ub_checks::is_aligned_and_not_null(addr, align, is_zst) ); intrinsics::volatile_store(dst, src); } @@ -1921,7 +1856,9 @@ pub unsafe fn write_volatile(dst: *mut T, src: T) { /// than trying to adapt this to accommodate that change. /// /// Any questions go to @nagisa. +#[allow(ptr_to_integer_transmute_in_consts)] #[lang = "align_offset"] +#[rustc_const_unstable(feature = "const_align_offset", issue = "90962")] #[safety::requires(a.is_power_of_two())] #[safety::ensures(|result| { let stride = mem::size_of::(); @@ -2208,13 +2145,39 @@ pub fn addr_eq(p: *const T, q: *const U) -> bool { /// Compares the *addresses* of the two function pointers for equality. /// -/// Function pointers comparisons can have surprising results since -/// they are never guaranteed to be unique and could vary between different -/// code generation units. Furthermore, different functions could have the -/// same address after being merged together. +/// This is the same as `f == g`, but using this function makes clear that the potentially +/// surprising semantics of function pointer comparison are involved. +/// +/// There are **very few guarantees** about how functions are compiled and they have no intrinsic +/// “identity”; in particular, this comparison: +/// +/// * May return `true` unexpectedly, in cases where functions are equivalent. +/// +/// For example, the following program is likely (but not guaranteed) to print `(true, true)` +/// when compiled with optimization: +/// +/// ``` +/// # #![feature(ptr_fn_addr_eq)] +/// let f: fn(i32) -> i32 = |x| x; +/// let g: fn(i32) -> i32 = |x| x + 0; // different closure, different body +/// let h: fn(u32) -> u32 = |x| x + 0; // different signature too +/// dbg!(std::ptr::fn_addr_eq(f, g), std::ptr::fn_addr_eq(f, h)); // not guaranteed to be equal +/// ``` +/// +/// * May return `false` in any case. +/// +/// This is particularly likely with generic functions but may happen with any function. +/// (From an implementation perspective, this is possible because functions may sometimes be +/// processed more than once by the compiler, resulting in duplicate machine code.) +/// +/// Despite these false positives and false negatives, this comparison can still be useful. +/// Specifically, if +/// +/// * `T` is the same type as `U`, `T` is a [subtype] of `U`, or `U` is a [subtype] of `T`, and +/// * `ptr::fn_addr_eq(f, g)` returns true, +/// +/// then calling `f` and calling `g` will be equivalent. /// -/// This is the same as `f == g` but using this function makes clear -/// that you are aware of these potentially surprising semantics. /// /// # Examples /// @@ -2226,6 +2189,8 @@ pub fn addr_eq(p: *const T, q: *const U) -> bool { /// fn b() { println!("b"); } /// assert!(!ptr::fn_addr_eq(a as fn(), b as fn())); /// ``` +/// +/// [subtype]: https://doc.rust-lang.org/reference/subtyping.html #[unstable(feature = "ptr_fn_addr_eq", issue = "129322")] #[inline(always)] #[must_use = "function pointer comparison produces a value"] @@ -2390,7 +2355,6 @@ impl fmt::Debug for F { /// no difference whether the pointer is null or dangling.) #[stable(feature = "raw_ref_macros", since = "1.51.0")] #[rustc_macro_transparency = "semitransparent"] -#[allow_internal_unstable(raw_ref_op)] pub macro addr_of($place:expr) { &raw const $place } @@ -2481,7 +2445,6 @@ pub macro addr_of($place:expr) { /// makes no difference whether the pointer is null or dangling.) #[stable(feature = "raw_ref_macros", since = "1.51.0")] #[rustc_macro_transparency = "semitransparent"] -#[allow_internal_unstable(raw_ref_op)] pub macro addr_of_mut($place:expr) { &raw mut $place } diff --git a/library/core/src/ptr/mut_ptr.rs b/library/core/src/ptr/mut_ptr.rs index 42975cc927b8e..7aa6a309a06b5 100644 --- a/library/core/src/ptr/mut_ptr.rs +++ b/library/core/src/ptr/mut_ptr.rs @@ -33,22 +33,7 @@ impl *mut T { #[rustc_diagnostic_item = "ptr_is_null"] #[inline] pub const fn is_null(self) -> bool { - #[inline] - fn runtime_impl(ptr: *mut u8) -> bool { - ptr.addr() == 0 - } - - #[inline] - const fn const_impl(ptr: *mut u8) -> bool { - // Compare via a cast to a thin pointer, so fat pointers are only - // considering their "data" part for null-ness. - match (ptr).guaranteed_eq(null_mut()) { - None => false, - Some(res) => res, - } - } - - const_eval_select((self as *mut u8,), const_impl, runtime_impl) + self.cast_const().is_null() } /// Casts to a pointer of another type. @@ -60,21 +45,22 @@ impl *mut T { self as _ } - /// Uses the pointer value in a new pointer of another type. + /// Uses the address value in a new pointer of another type. /// - /// In case `meta` is a (fat) pointer to an unsized type, this operation - /// will ignore the pointer part, whereas for (thin) pointers to sized - /// types, this has the same effect as a simple cast. + /// This operation will ignore the address part of its `meta` operand and discard existing + /// metadata of `self`. For pointers to a sized types (thin pointers), this has the same effect + /// as a simple cast. For pointers to an unsized type (fat pointers) this recombines the address + /// with new metadata such as slice lengths or `dyn`-vtable. /// - /// The resulting pointer will have provenance of `self`, i.e., for a fat - /// pointer, this operation is semantically the same as creating a new - /// fat pointer with the data pointer value of `self` but the metadata of - /// `meta`. + /// The resulting pointer will have provenance of `self`. This operation is semantically the + /// same as creating a new pointer with the data pointer value of `self` but the metadata of + /// `meta`, being fat or thin depending on the `meta` operand. /// /// # Examples /// - /// This function is primarily useful for allowing byte-wise pointer - /// arithmetic on potentially fat pointers: + /// This function is primarily useful for enabling pointer arithmetic on potentially fat + /// pointers. The pointer is cast to a sized pointee to utilize offset operations and then + /// recombined with its own original metadata. /// /// ``` /// #![feature(set_ptr_value)] @@ -88,8 +74,27 @@ impl *mut T { /// println!("{:?}", &*ptr); // will print "3" /// } /// ``` + /// + /// # *Incorrect* usage + /// + /// The provenance from pointers is *not* combined. The result must only be used to refer to the + /// address allowed by `self`. + /// + /// ```rust,no_run + /// #![feature(set_ptr_value)] + /// let mut x = 0u32; + /// let mut y = 1u32; + /// + /// let x = (&mut x) as *mut u32; + /// let y = (&mut y) as *mut u32; + /// + /// let offset = (x as usize - y as usize) / 4; + /// let bad = x.wrapping_add(offset).with_metadata_of(y); + /// + /// // This dereference is UB. The pointer only has provenance for `x` but points to `y`. + /// println!("{:?}", unsafe { &*bad }); #[unstable(feature = "set_ptr_value", issue = "75091")] - #[rustc_const_unstable(feature = "set_ptr_value", issue = "75091")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "ptr_metadata_const", since = "1.83.0"))] #[must_use = "returns a new pointer rather than modifying its argument"] #[inline] pub const fn with_metadata_of(self, meta: *const U) -> *mut U @@ -119,12 +124,12 @@ impl *mut T { /// Gets the "address" portion of the pointer. /// - /// This is similar to `self as usize`, which semantically discards *provenance* and - /// *address-space* information. However, unlike `self as usize`, casting the returned address - /// back to a pointer yields a [pointer without provenance][without_provenance_mut], which is undefined - /// behavior to dereference. To properly restore the lost information and obtain a - /// dereferenceable pointer, use [`with_addr`][pointer::with_addr] or - /// [`map_addr`][pointer::map_addr]. + /// This is similar to `self as usize`, except that the [provenance][crate::ptr#provenance] of + /// the pointer is discarded and not [exposed][crate::ptr#exposed-provenance]. This means that + /// casting the returned address back to a pointer yields a [pointer without + /// provenance][without_provenance_mut], which is undefined behavior to dereference. To properly + /// restore the lost information and obtain a dereferenceable pointer, use + /// [`with_addr`][pointer::with_addr] or [`map_addr`][pointer::map_addr]. /// /// If using those APIs is not possible because there is no way to preserve a pointer with the /// required provenance, then Strict Provenance might not be for you. Use pointer-integer casts @@ -138,89 +143,80 @@ impl *mut T { /// perform a change of representation to produce a value containing only the address /// portion of the pointer. What that means is up to the platform to define. /// - /// This API and its claimed semantics are part of the Strict Provenance experiment, and as such - /// might change in the future (including possibly weakening this so it becomes wholly - /// equivalent to `self as usize`). See the [module documentation][crate::ptr] for details. + /// This is a [Strict Provenance][crate::ptr#strict-provenance] API. #[must_use] #[inline(always)] - #[unstable(feature = "strict_provenance", issue = "95228")] + #[stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] pub fn addr(self) -> usize { - // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic. + // A pointer-to-integer transmute currently has exactly the right semantics: it returns the + // address without exposing the provenance. Note that this is *not* a stable guarantee about + // transmute semantics, it relies on sysroot crates having special status. // SAFETY: Pointer-to-integer transmutes are valid (if you are okay with losing the // provenance). unsafe { mem::transmute(self.cast::<()>()) } } - /// Exposes the "provenance" part of the pointer for future use in - /// [`with_exposed_provenance`][] and returns the "address" portion. + /// Exposes the ["provenance"][crate::ptr#provenance] part of the pointer for future use in + /// [`with_exposed_provenance_mut`] and returns the "address" portion. /// - /// This is equivalent to `self as usize`, which semantically discards *provenance* and - /// *address-space* information. Furthermore, this (like the `as` cast) has the implicit - /// side-effect of marking the provenance as 'exposed', so on platforms that support it you can - /// later call [`with_exposed_provenance_mut`][] to reconstitute the original pointer including its - /// provenance. (Reconstructing address space information, if required, is your responsibility.) + /// This is equivalent to `self as usize`, which semantically discards provenance information. + /// Furthermore, this (like the `as` cast) has the implicit side-effect of marking the + /// provenance as 'exposed', so on platforms that support it you can later call + /// [`with_exposed_provenance_mut`] to reconstitute the original pointer including its provenance. /// - /// Using this method means that code is *not* following [Strict - /// Provenance][super#strict-provenance] rules. Supporting - /// [`with_exposed_provenance_mut`][] complicates specification and reasoning and may not be supported - /// by tools that help you to stay conformant with the Rust memory model, so it is recommended - /// to use [`addr`][pointer::addr] wherever possible. + /// Due to its inherent ambiguity, [`with_exposed_provenance_mut`] may not be supported by tools + /// that help you to stay conformant with the Rust memory model. It is recommended to use + /// [Strict Provenance][crate::ptr#strict-provenance] APIs such as [`with_addr`][pointer::with_addr] + /// wherever possible, in which case [`addr`][pointer::addr] should be used instead of `expose_provenance`. /// /// On most platforms this will produce a value with the same bytes as the original pointer, /// because all the bytes are dedicated to describing the address. Platforms which need to store /// additional information in the pointer may not support this operation, since the 'expose' - /// side-effect which is required for [`with_exposed_provenance_mut`][] to work is typically not + /// side-effect which is required for [`with_exposed_provenance_mut`] to work is typically not /// available. /// - /// It is unclear whether this method can be given a satisfying unambiguous specification. This - /// API and its claimed semantics are part of [Exposed Provenance][super#exposed-provenance]. + /// This is an [Exposed Provenance][crate::ptr#exposed-provenance] API. /// /// [`with_exposed_provenance_mut`]: with_exposed_provenance_mut #[inline(always)] - #[unstable(feature = "exposed_provenance", issue = "95228")] + #[stable(feature = "exposed_provenance", since = "CURRENT_RUSTC_VERSION")] pub fn expose_provenance(self) -> usize { - // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic. self.cast::<()>() as usize } - /// Creates a new pointer with the given address. + /// Creates a new pointer with the given address and the [provenance][crate::ptr#provenance] of + /// `self`. /// - /// This performs the same operation as an `addr as ptr` cast, but copies - /// the *address-space* and *provenance* of `self` to the new pointer. - /// This allows us to dynamically preserve and propagate this important - /// information in a way that is otherwise impossible with a unary cast. + /// This is similar to a `addr as *mut T` cast, but copies + /// the *provenance* of `self` to the new pointer. + /// This avoids the inherent ambiguity of the unary cast. /// /// This is equivalent to using [`wrapping_offset`][pointer::wrapping_offset] to offset /// `self` to the given address, and therefore has all the same capabilities and restrictions. /// - /// This API and its claimed semantics are an extension to the Strict Provenance experiment, - /// see the [module documentation][crate::ptr] for details. + /// This is a [Strict Provenance][crate::ptr#strict-provenance] API. #[must_use] #[inline] - #[unstable(feature = "strict_provenance", issue = "95228")] + #[stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] pub fn with_addr(self, addr: usize) -> Self { - // FIXME(strict_provenance_magic): I am magic and should be a compiler intrinsic. - // - // In the mean-time, this operation is defined to be "as if" it was - // a wrapping_offset, so we can emulate it as such. This should properly - // restore pointer provenance even under today's compiler. + // This should probably be an intrinsic to avoid doing any sort of arithmetic, but + // meanwhile, we can implement it with `wrapping_offset`, which preserves the pointer's + // provenance. let self_addr = self.addr() as isize; let dest_addr = addr as isize; let offset = dest_addr.wrapping_sub(self_addr); - - // This is the canonical desugaring of this operation self.wrapping_byte_offset(offset) } - /// Creates a new pointer by mapping `self`'s address to a new one. + /// Creates a new pointer by mapping `self`'s address to a new one, preserving the original + /// pointer's [provenance][crate::ptr#provenance]. /// /// This is a convenience for [`with_addr`][pointer::with_addr], see that method for details. /// - /// This API and its claimed semantics are part of the Strict Provenance experiment, - /// see the [module documentation][crate::ptr] for details. + /// This is a [Strict Provenance][crate::ptr#strict-provenance] API. #[must_use] #[inline] - #[unstable(feature = "strict_provenance", issue = "95228")] + #[stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] pub fn map_addr(self, f: impl FnOnce(usize) -> usize) -> Self { self.with_addr(f(self.addr())) } @@ -276,7 +272,7 @@ impl *mut T { /// } /// ``` #[stable(feature = "ptr_as_ref", since = "1.9.0")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "const_ptr_is_null", issue = "74939")] #[inline] pub const unsafe fn as_ref<'a>(self) -> Option<&'a T> { // SAFETY: the caller must guarantee that `self` is valid for a @@ -310,7 +306,7 @@ impl *mut T { /// ``` // FIXME: mention it in the docs for `as_ref` and `as_uninit_ref` once stabilized. #[unstable(feature = "ptr_as_ref_unchecked", issue = "122034")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "ptr_as_ref_unchecked", issue = "122034")] #[inline] #[must_use] pub const unsafe fn as_ref_unchecked<'a>(self) -> &'a T { @@ -349,7 +345,7 @@ impl *mut T { /// ``` #[inline] #[unstable(feature = "ptr_as_uninit", issue = "75402")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "ptr_as_uninit", issue = "75402")] pub const unsafe fn as_uninit_ref<'a>(self) -> Option<&'a MaybeUninit> where T: Sized, @@ -359,7 +355,7 @@ impl *mut T { if self.is_null() { None } else { Some(unsafe { &*(self as *const MaybeUninit) }) } } - /// Adds an offset to a pointer. + /// Adds a signed offset to a pointer. /// /// `count` is in units of T; e.g., a `count` of 3 represents a pointer /// offset of `3 * size_of::()` bytes. @@ -368,9 +364,10 @@ impl *mut T { /// /// If any of the following conditions are violated, the result is Undefined Behavior: /// - /// * The computed offset, `count * size_of::()` bytes, must not overflow `isize`. + /// * The offset in bytes, `count * size_of::()`, computed on mathematical integers (without + /// "wrapping around"), must fit in an `isize`. /// - /// * If the computed offset is non-zero, then `self` must be derived from a pointer to some + /// * If the computed offset is non-zero, then `self` must be [derived from][crate::ptr#provenance] a pointer to some /// [allocated object], and the entire memory range between `self` and the result must be in /// bounds of that allocated object. In particular, this range must not "wrap around" the edge /// of the address space. @@ -407,13 +404,45 @@ impl *mut T { where T: Sized, { + #[inline] + #[rustc_allow_const_fn_unstable(const_eval_select)] + const fn runtime_offset_nowrap(this: *const (), count: isize, size: usize) -> bool { + #[inline] + fn runtime(this: *const (), count: isize, size: usize) -> bool { + // `size` is the size of a Rust type, so we know that + // `size <= isize::MAX` and thus `as` cast here is not lossy. + let Some(byte_offset) = count.checked_mul(size as isize) else { + return false; + }; + let (_, overflow) = this.addr().overflowing_add_signed(byte_offset); + !overflow + } + + const fn comptime(_: *const (), _: isize, _: usize) -> bool { + true + } + + // We can use const_eval_select here because this is only for UB checks. + intrinsics::const_eval_select((this, count, size), comptime, runtime) + } + + ub_checks::assert_unsafe_precondition!( + check_language_ub, + "ptr::offset requires the address calculation to not overflow", + ( + this: *const () = self as *const (), + count: isize = count, + size: usize = size_of::(), + ) => runtime_offset_nowrap(this, count, size) + ); + // SAFETY: the caller must uphold the safety contract for `offset`. // The obtained pointer is valid for writes since the caller must // guarantee that it points to the same allocated object as `self`. unsafe { intrinsics::offset(self, count) } } - /// Calculates the offset from a pointer in bytes. + /// Adds a signed offset in bytes to a pointer. /// /// `count` is in units of **bytes**. /// @@ -427,14 +456,14 @@ impl *mut T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn byte_offset(self, count: isize) -> Self { // SAFETY: the caller must uphold the safety contract for `offset`. unsafe { self.cast::().offset(count).with_metadata_of(self) } } - /// Calculates the offset from a pointer using wrapping arithmetic. + /// Adds a signed offset to a pointer using wrapping arithmetic. + /// /// `count` is in units of T; e.g., a `count` of 3 represents a pointer /// offset of `3 * size_of::()` bytes. /// @@ -493,7 +522,7 @@ impl *mut T { unsafe { intrinsics::arith_offset(self, count) as *mut T } } - /// Calculates the offset from a pointer in bytes using wrapping arithmetic. + /// Adds a signed offset in bytes to a pointer using wrapping arithmetic. /// /// `count` is in units of **bytes**. /// @@ -507,7 +536,6 @@ impl *mut T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] pub const fn wrapping_byte_offset(self, count: isize) -> Self { self.cast::().wrapping_offset(count).with_metadata_of(self) } @@ -522,7 +550,7 @@ impl *mut T { /// ## Examples /// /// ``` - /// #![feature(ptr_mask, strict_provenance)] + /// #![feature(ptr_mask)] /// let mut v = 17_u32; /// let ptr: *mut u32 = &mut v; /// @@ -595,7 +623,7 @@ impl *mut T { /// println!("{s:?}"); // It'll print: "[4, 2, 3]". /// ``` #[stable(feature = "ptr_as_ref", since = "1.9.0")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "const_ptr_is_null", issue = "74939")] #[inline] pub const unsafe fn as_mut<'a>(self) -> Option<&'a mut T> { // SAFETY: the caller must guarantee that `self` is be valid for @@ -631,7 +659,7 @@ impl *mut T { /// ``` // FIXME: mention it in the docs for `as_mut` and `as_uninit_mut` once stabilized. #[unstable(feature = "ptr_as_ref_unchecked", issue = "122034")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "ptr_as_ref_unchecked", issue = "122034")] #[inline] #[must_use] pub const unsafe fn as_mut_unchecked<'a>(self) -> &'a mut T { @@ -654,7 +682,7 @@ impl *mut T { /// the pointer is [convertible to a reference](crate::ptr#pointer-to-reference-conversion). #[inline] #[unstable(feature = "ptr_as_uninit", issue = "75402")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "ptr_as_uninit", issue = "75402")] pub const unsafe fn as_uninit_mut<'a>(self) -> Option<&'a mut MaybeUninit> where T: Sized, @@ -718,7 +746,7 @@ impl *mut T { (self as *const T).guaranteed_ne(other as _) } - /// Calculates the distance between two pointers. The returned value is in + /// Calculates the distance between two pointers within the same allocation. The returned value is in /// units of T: the distance in bytes divided by `mem::size_of::()`. /// /// This is equivalent to `(self as isize - origin as isize) / (mem::size_of::() as isize)`, @@ -741,7 +769,7 @@ impl *mut T { /// * `self` and `origin` must either /// /// * point to the same address, or - /// * both be *derived from* a pointer to the same [allocated object], and the memory range between + /// * both be [derived from][crate::ptr#provenance] a pointer to the same [allocated object], and the memory range between /// the two pointers must be in bounds of that object. (See below for an example.) /// /// * The distance between the pointers, in bytes, must be an exact multiple @@ -811,7 +839,7 @@ impl *mut T { unsafe { (self as *const T).offset_from(origin) } } - /// Calculates the distance between two pointers. The returned value is in + /// Calculates the distance between two pointers within the same allocation. The returned value is in /// units of **bytes**. /// /// This is purely a convenience for casting to a `u8` pointer and @@ -823,14 +851,13 @@ impl *mut T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn byte_offset_from(self, origin: *const U) -> isize { // SAFETY: the caller must uphold the safety contract for `offset_from`. unsafe { self.cast::().offset_from(origin.cast::()) } } - /// Calculates the distance between two pointers, *where it's known that + /// Calculates the distance between two pointers within the same allocation, *where it's known that /// `self` is equal to or greater than `origin`*. The returned value is in /// units of T: the distance in bytes is divided by `mem::size_of::()`. /// @@ -841,7 +868,7 @@ impl *mut T { /// but it provides slightly more information to the optimizer, which can /// sometimes allow it to optimize slightly better with some backends. /// - /// This method can be though of as recovering the `count` that was passed + /// This method can be thought of as recovering the `count` that was passed /// to [`add`](#method.add) (or, with the parameters in the other order, /// to [`sub`](#method.sub)). The following are all equivalent, assuming /// that their safety preconditions are met: @@ -903,7 +930,30 @@ impl *mut T { unsafe { (self as *const T).sub_ptr(origin) } } - /// Adds an offset to a pointer (convenience for `.offset(count as isize)`). + /// Calculates the distance between two pointers within the same allocation, *where it's known that + /// `self` is equal to or greater than `origin`*. The returned value is in + /// units of **bytes**. + /// + /// This is purely a convenience for casting to a `u8` pointer and + /// using [`sub_ptr`][pointer::sub_ptr] on it. See that method for + /// documentation and safety requirements. + /// + /// For non-`Sized` pointees this operation considers only the data pointers, + /// ignoring the metadata. + #[unstable(feature = "ptr_sub_ptr", issue = "95892")] + #[rustc_const_unstable(feature = "const_ptr_sub_ptr", issue = "95892")] + #[inline] + #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces + pub const unsafe fn byte_sub_ptr(self, origin: *mut U) -> usize { + // SAFETY: the caller must uphold the safety contract for `byte_sub_ptr`. + unsafe { (self as *const T).byte_sub_ptr(origin) } + } + + /// Adds an unsigned offset to a pointer. + /// + /// This can only move the pointer forward (or not move it). If you need to move forward or + /// backward depending on the value, then you might want [`offset`](#method.offset) instead + /// which takes a signed offset. /// /// `count` is in units of T; e.g., a `count` of 3 represents a pointer /// offset of `3 * size_of::()` bytes. @@ -912,9 +962,10 @@ impl *mut T { /// /// If any of the following conditions are violated, the result is Undefined Behavior: /// - /// * The computed offset, `count * size_of::()` bytes, must not overflow `isize`. + /// * The offset in bytes, `count * size_of::()`, computed on mathematical integers (without + /// "wrapping around"), must fit in an `isize`. /// - /// * If the computed offset is non-zero, then `self` must be derived from a pointer to some + /// * If the computed offset is non-zero, then `self` must be [derived from][crate::ptr#provenance] a pointer to some /// [allocated object], and the entire memory range between `self` and the result must be in /// bounds of that allocated object. In particular, this range must not "wrap around" the edge /// of the address space. @@ -951,11 +1002,42 @@ impl *mut T { where T: Sized, { + #[cfg(debug_assertions)] + #[inline] + #[rustc_allow_const_fn_unstable(const_eval_select)] + const fn runtime_add_nowrap(this: *const (), count: usize, size: usize) -> bool { + #[inline] + fn runtime(this: *const (), count: usize, size: usize) -> bool { + let Some(byte_offset) = count.checked_mul(size) else { + return false; + }; + let (_, overflow) = this.addr().overflowing_add(byte_offset); + byte_offset <= (isize::MAX as usize) && !overflow + } + + const fn comptime(_: *const (), _: usize, _: usize) -> bool { + true + } + + intrinsics::const_eval_select((this, count, size), comptime, runtime) + } + + #[cfg(debug_assertions)] // Expensive, and doesn't catch much in the wild. + ub_checks::assert_unsafe_precondition!( + check_language_ub, + "ptr::add requires that the address calculation does not overflow", + ( + this: *const () = self as *const (), + count: usize = count, + size: usize = size_of::(), + ) => runtime_add_nowrap(this, count, size) + ); + // SAFETY: the caller must uphold the safety contract for `offset`. unsafe { intrinsics::offset(self, count) } } - /// Calculates the offset from a pointer in bytes (convenience for `.byte_offset(count as isize)`). + /// Adds an unsigned offset in bytes to a pointer. /// /// `count` is in units of bytes. /// @@ -969,15 +1051,17 @@ impl *mut T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn byte_add(self, count: usize) -> Self { // SAFETY: the caller must uphold the safety contract for `add`. unsafe { self.cast::().add(count).with_metadata_of(self) } } - /// Subtracts an offset from a pointer (convenience for - /// `.offset((count as isize).wrapping_neg())`). + /// Subtracts an unsigned offset from a pointer. + /// + /// This can only move the pointer backward (or not move it). If you need to move forward or + /// backward depending on the value, then you might want [`offset`](#method.offset) instead + /// which takes a signed offset. /// /// `count` is in units of T; e.g., a `count` of 3 represents a pointer /// offset of `3 * size_of::()` bytes. @@ -986,9 +1070,10 @@ impl *mut T { /// /// If any of the following conditions are violated, the result is Undefined Behavior: /// - /// * The computed offset, `count * size_of::()` bytes, must not overflow `isize`. + /// * The offset in bytes, `count * size_of::()`, computed on mathematical integers (without + /// "wrapping around"), must fit in an `isize`. /// - /// * If the computed offset is non-zero, then `self` must be derived from a pointer to some + /// * If the computed offset is non-zero, then `self` must be [derived from][crate::ptr#provenance] a pointer to some /// [allocated object], and the entire memory range between `self` and the result must be in /// bounds of that allocated object. In particular, this range must not "wrap around" the edge /// of the address space. @@ -1019,13 +1104,43 @@ impl *mut T { #[stable(feature = "pointer_methods", since = "1.26.0")] #[must_use = "returns a new pointer rather than modifying its argument"] #[rustc_const_stable(feature = "const_ptr_offset", since = "1.61.0")] - #[rustc_allow_const_fn_unstable(unchecked_neg)] + #[cfg_attr(bootstrap, rustc_allow_const_fn_unstable(unchecked_neg))] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn sub(self, count: usize) -> Self where T: Sized, { + #[cfg(debug_assertions)] + #[inline] + #[rustc_allow_const_fn_unstable(const_eval_select)] + const fn runtime_sub_nowrap(this: *const (), count: usize, size: usize) -> bool { + #[inline] + fn runtime(this: *const (), count: usize, size: usize) -> bool { + let Some(byte_offset) = count.checked_mul(size) else { + return false; + }; + byte_offset <= (isize::MAX as usize) && this.addr() >= byte_offset + } + + const fn comptime(_: *const (), _: usize, _: usize) -> bool { + true + } + + intrinsics::const_eval_select((this, count, size), comptime, runtime) + } + + #[cfg(debug_assertions)] // Expensive, and doesn't catch much in the wild. + ub_checks::assert_unsafe_precondition!( + check_language_ub, + "ptr::sub requires that the address calculation does not overflow", + ( + this: *const () = self as *const (), + count: usize = count, + size: usize = size_of::(), + ) => runtime_sub_nowrap(this, count, size) + ); + if T::IS_ZST { // Pointer arithmetic does nothing when the pointee is a ZST. self @@ -1033,12 +1148,11 @@ impl *mut T { // SAFETY: the caller must uphold the safety contract for `offset`. // Because the pointee is *not* a ZST, that means that `count` is // at most `isize::MAX`, and thus the negation cannot overflow. - unsafe { self.offset((count as isize).unchecked_neg()) } + unsafe { intrinsics::offset(self, intrinsics::unchecked_sub(0, count as isize)) } } } - /// Calculates the offset from a pointer in bytes (convenience for - /// `.byte_offset((count as isize).wrapping_neg())`). + /// Subtracts an unsigned offset in bytes from a pointer. /// /// `count` is in units of bytes. /// @@ -1052,15 +1166,13 @@ impl *mut T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn byte_sub(self, count: usize) -> Self { // SAFETY: the caller must uphold the safety contract for `sub`. unsafe { self.cast::().sub(count).with_metadata_of(self) } } - /// Calculates the offset from a pointer using wrapping arithmetic. - /// (convenience for `.wrapping_offset(count as isize)`) + /// Adds an unsigned offset to a pointer using wrapping arithmetic. /// /// `count` is in units of T; e.g., a `count` of 3 represents a pointer /// offset of `3 * size_of::()` bytes. @@ -1119,8 +1231,7 @@ impl *mut T { self.wrapping_offset(count as isize) } - /// Calculates the offset from a pointer in bytes using wrapping arithmetic. - /// (convenience for `.wrapping_byte_offset(count as isize)`) + /// Adds an unsigned offset in bytes to a pointer using wrapping arithmetic. /// /// `count` is in units of bytes. /// @@ -1133,13 +1244,11 @@ impl *mut T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] pub const fn wrapping_byte_add(self, count: usize) -> Self { self.cast::().wrapping_add(count).with_metadata_of(self) } - /// Calculates the offset from a pointer using wrapping arithmetic. - /// (convenience for `.wrapping_offset((count as isize).wrapping_neg())`) + /// Subtracts an unsigned offset from a pointer using wrapping arithmetic. /// /// `count` is in units of T; e.g., a `count` of 3 represents a pointer /// offset of `3 * size_of::()` bytes. @@ -1198,8 +1307,7 @@ impl *mut T { self.wrapping_offset((count as isize).wrapping_neg()) } - /// Calculates the offset from a pointer in bytes using wrapping arithmetic. - /// (convenience for `.wrapping_offset((count as isize).wrapping_neg())`) + /// Subtracts an unsigned offset in bytes from a pointer using wrapping arithmetic. /// /// `count` is in units of bytes. /// @@ -1212,7 +1320,6 @@ impl *mut T { #[inline(always)] #[stable(feature = "pointer_byte_offsets", since = "1.75.0")] #[rustc_const_stable(feature = "const_pointer_byte_offsets", since = "1.75.0")] - #[rustc_allow_const_fn_unstable(set_ptr_value)] pub const fn wrapping_byte_sub(self, count: usize) -> Self { self.cast::().wrapping_sub(count).with_metadata_of(self) } @@ -1284,7 +1391,7 @@ impl *mut T { /// See [`ptr::copy`] for safety concerns and examples. /// /// [`ptr::copy`]: crate::ptr::copy() - #[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] + #[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")] #[stable(feature = "pointer_methods", since = "1.26.0")] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces @@ -1304,7 +1411,7 @@ impl *mut T { /// See [`ptr::copy_nonoverlapping`] for safety concerns and examples. /// /// [`ptr::copy_nonoverlapping`]: crate::ptr::copy_nonoverlapping() - #[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] + #[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")] #[stable(feature = "pointer_methods", since = "1.26.0")] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces @@ -1324,7 +1431,7 @@ impl *mut T { /// See [`ptr::copy`] for safety concerns and examples. /// /// [`ptr::copy`]: crate::ptr::copy() - #[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] + #[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")] #[stable(feature = "pointer_methods", since = "1.26.0")] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces @@ -1344,7 +1451,7 @@ impl *mut T { /// See [`ptr::copy_nonoverlapping`] for safety concerns and examples. /// /// [`ptr::copy_nonoverlapping`]: crate::ptr::copy_nonoverlapping() - #[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] + #[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")] #[stable(feature = "pointer_methods", since = "1.26.0")] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces @@ -1375,7 +1482,7 @@ impl *mut T { /// /// [`ptr::write`]: crate::ptr::write() #[stable(feature = "pointer_methods", since = "1.26.0")] - #[rustc_const_unstable(feature = "const_ptr_write", issue = "86302")] + #[rustc_const_stable(feature = "const_ptr_write", since = "1.83.0")] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn write(self, val: T) @@ -1394,7 +1501,7 @@ impl *mut T { /// [`ptr::write_bytes`]: crate::ptr::write_bytes() #[doc(alias = "memset")] #[stable(feature = "pointer_methods", since = "1.26.0")] - #[rustc_const_unstable(feature = "const_ptr_write", issue = "86302")] + #[rustc_const_stable(feature = "const_ptr_write", since = "1.83.0")] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn write_bytes(self, val: u8, count: usize) @@ -1435,7 +1542,7 @@ impl *mut T { /// /// [`ptr::write_unaligned`]: crate::ptr::write_unaligned() #[stable(feature = "pointer_methods", since = "1.26.0")] - #[rustc_const_unstable(feature = "const_ptr_write", issue = "86302")] + #[rustc_const_stable(feature = "const_ptr_write", since = "1.83.0")] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces pub const unsafe fn write_unaligned(self, val: T) @@ -1584,7 +1691,6 @@ impl *mut T { /// /// ``` /// #![feature(const_pointer_is_aligned)] - /// #![feature(const_mut_refs)] /// /// // On some platforms, the alignment of primitives is less than their size. /// #[repr(align(4))] @@ -1710,7 +1816,6 @@ impl *mut T { /// ``` /// #![feature(pointer_is_aligned_to)] /// #![feature(const_pointer_is_aligned)] - /// #![feature(const_mut_refs)] /// /// // On some platforms, the alignment of i32 is less than 4. /// #[repr(align(4))] @@ -1788,6 +1893,7 @@ impl *mut T { } #[inline] + #[rustc_const_unstable(feature = "const_pointer_is_aligned", issue = "104203")] const fn const_impl(ptr: *mut (), align: usize) -> bool { // We can't use the address of `self` in a `const fn`, so we use `align_offset` instead. ptr.align_offset(align) == 0 @@ -1819,7 +1925,6 @@ impl *mut [T] { #[inline(always)] #[stable(feature = "slice_ptr_len", since = "1.79.0")] #[rustc_const_stable(feature = "const_slice_ptr_len", since = "1.79.0")] - #[rustc_allow_const_fn_unstable(ptr_metadata)] pub const fn len(self) -> usize { metadata(self) } @@ -2031,7 +2136,7 @@ impl *mut [T] { /// [allocated object]: crate::ptr#allocated-object #[inline] #[unstable(feature = "ptr_as_uninit", issue = "75402")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "ptr_as_uninit", issue = "75402")] pub const unsafe fn as_uninit_slice<'a>(self) -> Option<&'a [MaybeUninit]> { if self.is_null() { None @@ -2083,7 +2188,7 @@ impl *mut [T] { /// [allocated object]: crate::ptr#allocated-object #[inline] #[unstable(feature = "ptr_as_uninit", issue = "75402")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "ptr_as_uninit", issue = "75402")] pub const unsafe fn as_uninit_slice_mut<'a>(self) -> Option<&'a mut [MaybeUninit]> { if self.is_null() { None diff --git a/library/core/src/ptr/non_null.rs b/library/core/src/ptr/non_null.rs index 4d6e484915dbf..0aa306f803a70 100644 --- a/library/core/src/ptr/non_null.rs +++ b/library/core/src/ptr/non_null.rs @@ -112,9 +112,7 @@ impl NonNull { #[must_use] #[inline] pub const fn dangling() -> Self { - // SAFETY: mem::align_of() returns a non-zero usize which is then casted - // to a *mut T. Therefore, `ptr` is not null and the conditions for - // calling new_unchecked() are respected. + // SAFETY: ptr::dangling_mut() returns a non-null well-aligned pointer. unsafe { let ptr = crate::ptr::dangling_mut::(); NonNull::new_unchecked(ptr) @@ -138,7 +136,7 @@ impl NonNull { #[inline] #[must_use] #[unstable(feature = "ptr_as_uninit", issue = "75402")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "ptr_as_uninit", issue = "75402")] pub const unsafe fn as_uninit_ref<'a>(self) -> &'a MaybeUninit { // SAFETY: the caller must guarantee that `self` meets all the // requirements for a reference. @@ -162,7 +160,7 @@ impl NonNull { #[inline] #[must_use] #[unstable(feature = "ptr_as_uninit", issue = "75402")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "ptr_as_uninit", issue = "75402")] pub const unsafe fn as_uninit_mut<'a>(self) -> &'a mut MaybeUninit { // SAFETY: the caller must guarantee that `self` meets all the // requirements for a reference. @@ -239,6 +237,24 @@ impl NonNull { } } + /// Converts a reference to a `NonNull` pointer. + #[unstable(feature = "non_null_from_ref", issue = "130823")] + #[rustc_const_unstable(feature = "non_null_from_ref", issue = "130823")] + #[inline] + pub const fn from_ref(r: &T) -> Self { + // SAFETY: A reference cannot be null. + unsafe { NonNull { pointer: r as *const T } } + } + + /// Converts a mutable reference to a `NonNull` pointer. + #[unstable(feature = "non_null_from_ref", issue = "130823")] + #[rustc_const_unstable(feature = "non_null_from_ref", issue = "130823")] + #[inline] + pub const fn from_mut(r: &mut T) -> Self { + // SAFETY: A mutable reference cannot be null. + unsafe { NonNull { pointer: r as *mut T } } + } + /// Performs the same functionality as [`std::ptr::from_raw_parts`], except that a /// `NonNull` pointer is returned, as opposed to a raw `*const` pointer. /// @@ -274,40 +290,39 @@ impl NonNull { /// /// For more details see the equivalent method on a raw pointer, [`pointer::addr`]. /// - /// This API and its claimed semantics are part of the Strict Provenance experiment, - /// see the [`ptr` module documentation][crate::ptr]. + /// This is a [Strict Provenance][crate::ptr#strict-provenance] API. #[must_use] #[inline] - #[unstable(feature = "strict_provenance", issue = "95228")] + #[stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] pub fn addr(self) -> NonZero { // SAFETY: The pointer is guaranteed by the type to be non-null, // meaning that the address will be non-zero. unsafe { NonZero::new_unchecked(self.pointer.addr()) } } - /// Creates a new pointer with the given address. + /// Creates a new pointer with the given address and the [provenance][crate::ptr#provenance] of + /// `self`. /// /// For more details see the equivalent method on a raw pointer, [`pointer::with_addr`]. /// - /// This API and its claimed semantics are part of the Strict Provenance experiment, - /// see the [`ptr` module documentation][crate::ptr]. + /// This is a [Strict Provenance][crate::ptr#strict-provenance] API. #[must_use] #[inline] - #[unstable(feature = "strict_provenance", issue = "95228")] + #[stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] pub fn with_addr(self, addr: NonZero) -> Self { // SAFETY: The result of `ptr::from::with_addr` is non-null because `addr` is guaranteed to be non-zero. unsafe { NonNull::new_unchecked(self.pointer.with_addr(addr.get()) as *mut _) } } - /// Creates a new pointer by mapping `self`'s address to a new one. + /// Creates a new pointer by mapping `self`'s address to a new one, preserving the + /// [provenance][crate::ptr#provenance] of `self`. /// /// For more details see the equivalent method on a raw pointer, [`pointer::map_addr`]. /// - /// This API and its claimed semantics are part of the Strict Provenance experiment, - /// see the [`ptr` module documentation][crate::ptr]. + /// This is a [Strict Provenance][crate::ptr#strict-provenance] API. #[must_use] #[inline] - #[unstable(feature = "strict_provenance", issue = "95228")] + #[stable(feature = "strict_provenance", since = "CURRENT_RUSTC_VERSION")] pub fn map_addr(self, f: impl FnOnce(NonZero) -> NonZero) -> Self { self.with_addr(f(self.addr())) } @@ -403,7 +418,7 @@ impl NonNull { /// /// [the module documentation]: crate::ptr#safety #[stable(feature = "nonnull", since = "1.25.0")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_stable(feature = "const_ptr_as_ref", since = "1.83.0")] #[must_use] #[inline(always)] pub const unsafe fn as_mut<'a>(&mut self) -> &'a mut T { @@ -576,7 +591,6 @@ impl NonNull { #[must_use] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces - #[rustc_allow_const_fn_unstable(set_ptr_value)] #[stable(feature = "non_null_convenience", since = "1.80.0")] #[rustc_const_stable(feature = "non_null_convenience", since = "1.80.0")] pub const unsafe fn byte_add(self, count: usize) -> Self { @@ -630,7 +644,7 @@ impl NonNull { #[must_use = "returns a new pointer rather than modifying its argument"] #[stable(feature = "non_null_convenience", since = "1.80.0")] #[rustc_const_stable(feature = "non_null_convenience", since = "1.80.0")] - #[rustc_allow_const_fn_unstable(unchecked_neg)] + #[cfg_attr(bootstrap, rustc_allow_const_fn_unstable(unchecked_neg))] pub const unsafe fn sub(self, count: usize) -> Self where T: Sized, @@ -660,7 +674,6 @@ impl NonNull { #[must_use] #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces - #[rustc_allow_const_fn_unstable(set_ptr_value)] #[stable(feature = "non_null_convenience", since = "1.80.0")] #[rustc_const_stable(feature = "non_null_convenience", since = "1.80.0")] pub const unsafe fn byte_sub(self, count: usize) -> Self { @@ -672,7 +685,7 @@ impl NonNull { unsafe { NonNull { pointer: self.pointer.byte_sub(count) } } } - /// Calculates the distance between two pointers. The returned value is in + /// Calculates the distance between two pointers within the same allocation. The returned value is in /// units of T: the distance in bytes divided by `mem::size_of::()`. /// /// This is equivalent to `(self as isize - origin as isize) / (mem::size_of::() as isize)`, @@ -742,7 +755,6 @@ impl NonNull { /// *Incorrect* usage: /// /// ```rust,no_run - /// #![feature(strict_provenance)] /// use std::ptr::NonNull; /// /// let ptr1 = NonNull::new(Box::into_raw(Box::new(0u8))).unwrap(); @@ -770,7 +782,7 @@ impl NonNull { unsafe { self.pointer.offset_from(origin.pointer) } } - /// Calculates the distance between two pointers. The returned value is in + /// Calculates the distance between two pointers within the same allocation. The returned value is in /// units of **bytes**. /// /// This is purely a convenience for casting to a `u8` pointer and @@ -790,7 +802,7 @@ impl NonNull { // N.B. `wrapping_offset``, `wrapping_add`, etc are not implemented because they can wrap to null - /// Calculates the distance between two pointers, *where it's known that + /// Calculates the distance between two pointers within the same allocation, *where it's known that /// `self` is equal to or greater than `origin`*. The returned value is in /// units of T: the distance in bytes is divided by `mem::size_of::()`. /// @@ -863,6 +875,25 @@ impl NonNull { unsafe { self.pointer.sub_ptr(subtracted.pointer) } } + /// Calculates the distance between two pointers within the same allocation, *where it's known that + /// `self` is equal to or greater than `origin`*. The returned value is in + /// units of **bytes**. + /// + /// This is purely a convenience for casting to a `u8` pointer and + /// using [`sub_ptr`][NonNull::sub_ptr] on it. See that method for + /// documentation and safety requirements. + /// + /// For non-`Sized` pointees this operation considers only the data pointers, + /// ignoring the metadata. + #[inline(always)] + #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces + #[unstable(feature = "ptr_sub_ptr", issue = "95892")] + #[rustc_const_unstable(feature = "const_ptr_sub_ptr", issue = "95892")] + pub const unsafe fn byte_sub_ptr(self, origin: NonNull) -> usize { + // SAFETY: the caller must uphold the safety contract for `byte_sub_ptr`. + unsafe { self.pointer.byte_sub_ptr(origin.pointer) } + } + /// Reads the value from `self` without moving it. This leaves the /// memory in `self` unchanged. /// @@ -933,7 +964,7 @@ impl NonNull { #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[stable(feature = "non_null_convenience", since = "1.80.0")] - #[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] + #[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")] pub const unsafe fn copy_to(self, dest: NonNull, count: usize) where T: Sized, @@ -953,7 +984,7 @@ impl NonNull { #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[stable(feature = "non_null_convenience", since = "1.80.0")] - #[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] + #[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")] pub const unsafe fn copy_to_nonoverlapping(self, dest: NonNull, count: usize) where T: Sized, @@ -973,7 +1004,7 @@ impl NonNull { #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[stable(feature = "non_null_convenience", since = "1.80.0")] - #[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] + #[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")] pub const unsafe fn copy_from(self, src: NonNull, count: usize) where T: Sized, @@ -993,7 +1024,7 @@ impl NonNull { #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[stable(feature = "non_null_convenience", since = "1.80.0")] - #[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")] + #[rustc_const_stable(feature = "const_intrinsic_copy", since = "1.83.0")] pub const unsafe fn copy_from_nonoverlapping(self, src: NonNull, count: usize) where T: Sized, @@ -1023,7 +1054,7 @@ impl NonNull { #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[stable(feature = "non_null_convenience", since = "1.80.0")] - #[rustc_const_unstable(feature = "const_ptr_write", issue = "86302")] + #[rustc_const_stable(feature = "const_ptr_write", since = "1.83.0")] pub const unsafe fn write(self, val: T) where T: Sized, @@ -1042,7 +1073,7 @@ impl NonNull { #[doc(alias = "memset")] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[stable(feature = "non_null_convenience", since = "1.80.0")] - #[rustc_const_unstable(feature = "const_ptr_write", issue = "86302")] + #[rustc_const_stable(feature = "const_ptr_write", since = "1.83.0")] pub const unsafe fn write_bytes(self, val: u8, count: usize) where T: Sized, @@ -1083,7 +1114,7 @@ impl NonNull { #[inline(always)] #[cfg_attr(miri, track_caller)] // even without panics, this helps for Miri backtraces #[stable(feature = "non_null_convenience", since = "1.80.0")] - #[rustc_const_unstable(feature = "const_ptr_write", issue = "86302")] + #[rustc_const_stable(feature = "const_ptr_write", since = "1.83.0")] pub const unsafe fn write_unaligned(self, val: T) where T: Sized, @@ -1221,7 +1252,6 @@ impl NonNull { /// /// ``` /// #![feature(const_nonnull_new)] - /// #![feature(const_option)] /// #![feature(const_pointer_is_aligned)] /// use std::ptr::NonNull; /// @@ -1274,7 +1304,6 @@ impl NonNull { /// /// ``` /// #![feature(const_pointer_is_aligned)] - /// #![feature(const_option)] /// #![feature(const_nonnull_new)] /// use std::ptr::NonNull; /// @@ -1444,7 +1473,7 @@ impl NonNull<[T]> { /// (Note that this example artificially demonstrates a use of this method, /// but `let slice = NonNull::from(&x[..]);` would be a better way to write code like this.) #[stable(feature = "nonnull_slice_from_raw_parts", since = "1.70.0")] - #[rustc_const_unstable(feature = "const_slice_from_raw_parts_mut", issue = "67456")] + #[rustc_const_stable(feature = "const_slice_from_raw_parts_mut", since = "1.83.0")] #[must_use] #[inline] pub const fn slice_from_raw_parts(data: NonNull, len: usize) -> Self { @@ -1507,7 +1536,6 @@ impl NonNull<[T]> { #[inline] #[must_use] #[unstable(feature = "slice_ptr_get", issue = "74265")] - #[rustc_const_unstable(feature = "slice_ptr_get", issue = "74265")] pub const fn as_non_null_ptr(self) -> NonNull { self.cast() } @@ -1572,7 +1600,7 @@ impl NonNull<[T]> { #[inline] #[must_use] #[unstable(feature = "ptr_as_uninit", issue = "75402")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "ptr_as_uninit", issue = "75402")] pub const unsafe fn as_uninit_slice<'a>(self) -> &'a [MaybeUninit] { // SAFETY: the caller must uphold the safety contract for `as_uninit_slice`. unsafe { slice::from_raw_parts(self.cast().as_ptr(), self.len()) } @@ -1637,7 +1665,7 @@ impl NonNull<[T]> { #[inline] #[must_use] #[unstable(feature = "ptr_as_uninit", issue = "75402")] - #[rustc_const_unstable(feature = "const_ptr_as_ref", issue = "91822")] + #[rustc_const_unstable(feature = "ptr_as_uninit", issue = "75402")] pub const unsafe fn as_uninit_slice_mut<'a>(self) -> &'a mut [MaybeUninit] { // SAFETY: the caller must uphold the safety contract for `as_uninit_slice_mut`. unsafe { slice::from_raw_parts_mut(self.cast().as_ptr(), self.len()) } @@ -1762,9 +1790,8 @@ impl From<&mut T> for NonNull { /// /// This conversion is safe and infallible since references cannot be null. #[inline] - fn from(reference: &mut T) -> Self { - // SAFETY: A mutable reference cannot be null. - unsafe { NonNull { pointer: reference as *mut T } } + fn from(r: &mut T) -> Self { + NonNull::from_mut(r) } } @@ -1774,9 +1801,8 @@ impl From<&T> for NonNull { /// /// This conversion is safe and infallible since references cannot be null. #[inline] - fn from(reference: &T) -> Self { - // SAFETY: A reference cannot be null. - unsafe { NonNull { pointer: reference as *const T } } + fn from(r: &T) -> Self { + NonNull::from_ref(r) } } diff --git a/library/core/src/ptr/unique.rs b/library/core/src/ptr/unique.rs index 8a282c86a7868..fd0dd6c46681f 100644 --- a/library/core/src/ptr/unique.rs +++ b/library/core/src/ptr/unique.rs @@ -98,6 +98,7 @@ impl Unique { /// Creates a new `Unique` if `ptr` is non-null. #[inline] + #[rustc_const_unstable(feature = "const_align_offset", issue = "90962")] #[ensures(|result| result.is_none() == ptr.is_null())] #[ensures(|result| result.is_none() || result.unwrap().as_ptr() == ptr)] pub const fn new(ptr: *mut T) -> Option { diff --git a/library/core/src/random.rs b/library/core/src/random.rs new file mode 100644 index 0000000000000..051fe26086389 --- /dev/null +++ b/library/core/src/random.rs @@ -0,0 +1,62 @@ +//! Random value generation. +//! +//! The [`Random`] trait allows generating a random value for a type using a +//! given [`RandomSource`]. + +/// A source of randomness. +#[unstable(feature = "random", issue = "130703")] +pub trait RandomSource { + /// Fills `bytes` with random bytes. + fn fill_bytes(&mut self, bytes: &mut [u8]); +} + +/// A trait for getting a random value for a type. +/// +/// **Warning:** Be careful when manipulating random values! The +/// [`random`](Random::random) method on integers samples them with a uniform +/// distribution, so a value of 1 is just as likely as [`i32::MAX`]. By using +/// modulo operations, some of the resulting values can become more likely than +/// others. Use audited crates when in doubt. +#[unstable(feature = "random", issue = "130703")] +pub trait Random: Sized { + /// Generates a random value. + fn random(source: &mut (impl RandomSource + ?Sized)) -> Self; +} + +impl Random for bool { + fn random(source: &mut (impl RandomSource + ?Sized)) -> Self { + u8::random(source) & 1 == 1 + } +} + +macro_rules! impl_primitive { + ($t:ty) => { + impl Random for $t { + /// Generates a random value. + /// + /// **Warning:** Be careful when manipulating the resulting value! This + /// method samples according to a uniform distribution, so a value of 1 is + /// just as likely as [`MAX`](Self::MAX). By using modulo operations, some + /// values can become more likely than others. Use audited crates when in + /// doubt. + fn random(source: &mut (impl RandomSource + ?Sized)) -> Self { + let mut bytes = (0 as Self).to_ne_bytes(); + source.fill_bytes(&mut bytes); + Self::from_ne_bytes(bytes) + } + } + }; +} + +impl_primitive!(u8); +impl_primitive!(i8); +impl_primitive!(u16); +impl_primitive!(i16); +impl_primitive!(u32); +impl_primitive!(i32); +impl_primitive!(u64); +impl_primitive!(i64); +impl_primitive!(u128); +impl_primitive!(i128); +impl_primitive!(usize); +impl_primitive!(isize); diff --git a/library/core/src/range.rs b/library/core/src/range.rs index 408972c267f1a..427526fd14b91 100644 --- a/library/core/src/range.rs +++ b/library/core/src/range.rs @@ -24,9 +24,9 @@ mod iter; #[unstable(feature = "new_range_api", issue = "125687")] pub mod legacy; +use Bound::{Excluded, Included, Unbounded}; #[doc(inline)] pub use iter::{IterRange, IterRangeFrom, IterRangeInclusive}; -use Bound::{Excluded, Included, Unbounded}; #[doc(inline)] pub use crate::iter::Step; diff --git a/library/core/src/range/iter.rs b/library/core/src/range/iter.rs index 4935280df60bc..1e261d8c1d93a 100644 --- a/library/core/src/range/iter.rs +++ b/library/core/src/range/iter.rs @@ -2,7 +2,7 @@ use crate::iter::{ FusedIterator, Step, TrustedLen, TrustedRandomAccess, TrustedRandomAccessNoCoerce, TrustedStep, }; use crate::num::NonZero; -use crate::range::{legacy, Range, RangeFrom, RangeInclusive}; +use crate::range::{Range, RangeFrom, RangeInclusive, legacy}; /// By-value [`Range`] iterator. #[unstable(feature = "new_range_api", issue = "125687")] diff --git a/library/core/src/result.rs b/library/core/src/result.rs index 73b11f803d929..330d1eb14edb0 100644 --- a/library/core/src/result.rs +++ b/library/core/src/result.rs @@ -653,6 +653,7 @@ impl Result { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "result_ok_method")] pub fn ok(self) -> Option { match self { Ok(x) => Some(x), @@ -733,7 +734,7 @@ impl Result { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_result", issue = "82814")] + #[rustc_const_stable(feature = "const_result", since = "1.83.0")] pub const fn as_mut(&mut self) -> Result<&mut T, &mut E> { match *self { Ok(ref mut x) => Ok(x), @@ -1535,11 +1536,18 @@ impl Result<&T, E> { /// ``` #[inline] #[stable(feature = "result_copied", since = "1.59.0")] - pub fn copied(self) -> Result + #[rustc_const_stable(feature = "const_result", since = "1.83.0")] + #[rustc_allow_const_fn_unstable(const_precise_live_drops)] + pub const fn copied(self) -> Result where T: Copy, { - self.map(|&t| t) + // FIXME(const-hack): this implementation, which sidesteps using `Result::map` since it's not const + // ready yet, should be reverted when possible to avoid code repetition + match self { + Ok(&v) => Ok(v), + Err(e) => Err(e), + } } /// Maps a `Result<&T, E>` to a `Result` by cloning the contents of the @@ -1579,11 +1587,18 @@ impl Result<&mut T, E> { /// ``` #[inline] #[stable(feature = "result_copied", since = "1.59.0")] - pub fn copied(self) -> Result + #[rustc_const_stable(feature = "const_result", since = "1.83.0")] + #[rustc_allow_const_fn_unstable(const_precise_live_drops)] + pub const fn copied(self) -> Result where T: Copy, { - self.map(|&mut t| t) + // FIXME(const-hack): this implementation, which sidesteps using `Result::map` since it's not const + // ready yet, should be reverted when possible to avoid code repetition + match self { + Ok(&mut v) => Ok(v), + Err(e) => Err(e), + } } /// Maps a `Result<&mut T, E>` to a `Result` by cloning the contents of the @@ -1626,7 +1641,8 @@ impl Result, E> { /// ``` #[inline] #[stable(feature = "transpose_result", since = "1.33.0")] - #[rustc_const_unstable(feature = "const_result", issue = "82814")] + #[rustc_const_stable(feature = "const_result", since = "1.83.0")] + #[rustc_allow_const_fn_unstable(const_precise_live_drops)] pub const fn transpose(self) -> Option> { match self { Ok(Some(x)) => Some(Ok(x)), @@ -1663,8 +1679,13 @@ impl Result, E> { /// ``` #[inline] #[unstable(feature = "result_flattening", issue = "70142")] - pub fn flatten(self) -> Result { - self.and_then(convert::identity) + #[rustc_const_unstable(feature = "result_flattening", issue = "70142")] + pub const fn flatten(self) -> Result { + // FIXME(const-hack): could be written with `and_then` + match self { + Ok(inner) => inner, + Err(e) => Err(e), + } } } diff --git a/library/core/src/slice/ascii.rs b/library/core/src/slice/ascii.rs index d1ea52fab6b87..21e0460072fb3 100644 --- a/library/core/src/slice/ascii.rs +++ b/library/core/src/slice/ascii.rs @@ -67,10 +67,15 @@ impl [u8] { /// /// [`to_ascii_uppercase`]: #method.to_ascii_uppercase #[stable(feature = "ascii_methods_on_intrinsics", since = "1.23.0")] + #[rustc_const_stable(feature = "const_make_ascii", since = "CURRENT_RUSTC_VERSION")] #[inline] - pub fn make_ascii_uppercase(&mut self) { - for byte in self { + pub const fn make_ascii_uppercase(&mut self) { + // FIXME(const-hack): We would like to simply iterate using `for` loops but this isn't currently allowed in constant expressions. + let mut i = 0; + while i < self.len() { + let byte = &mut self[i]; byte.make_ascii_uppercase(); + i += 1; } } @@ -84,10 +89,15 @@ impl [u8] { /// /// [`to_ascii_lowercase`]: #method.to_ascii_lowercase #[stable(feature = "ascii_methods_on_intrinsics", since = "1.23.0")] + #[rustc_const_stable(feature = "const_make_ascii", since = "CURRENT_RUSTC_VERSION")] #[inline] - pub fn make_ascii_lowercase(&mut self) { - for byte in self { + pub const fn make_ascii_lowercase(&mut self) { + // FIXME(const-hack): We would like to simply iterate using `for` loops but this isn't currently allowed in constant expressions. + let mut i = 0; + while i < self.len() { + let byte = &mut self[i]; byte.make_ascii_lowercase(); + i += 1; } } @@ -336,6 +346,8 @@ pub const fn is_ascii_simple(mut bytes: &[u8]) -> bool { /// If any of these loads produces something for which `contains_nonascii` /// (above) returns true, then we know the answer is false. #[inline] +#[rustc_allow_const_fn_unstable(const_raw_ptr_comparison, const_pointer_is_aligned)] // only in a debug assertion +#[rustc_allow_const_fn_unstable(const_align_offset)] // behavior does not change when `align_offset` fails const fn is_ascii(s: &[u8]) -> bool { const USIZE_SIZE: usize = mem::size_of::(); diff --git a/library/core/src/slice/cmp.rs b/library/core/src/slice/cmp.rs index 1769612def0a5..9cb00644e6442 100644 --- a/library/core/src/slice/cmp.rs +++ b/library/core/src/slice/cmp.rs @@ -257,3 +257,29 @@ impl SliceContains for i8 { memchr::memchr(byte, bytes).is_some() } } + +macro_rules! impl_slice_contains { + ($($t:ty),*) => { + $( + impl SliceContains for $t { + #[inline] + fn slice_contains(&self, arr: &[$t]) -> bool { + // Make our LANE_COUNT 4x the normal lane count (aiming for 128 bit vectors). + // The compiler will nicely unroll it. + const LANE_COUNT: usize = 4 * (128 / (mem::size_of::<$t>() * 8)); + // SIMD + let mut chunks = arr.chunks_exact(LANE_COUNT); + for chunk in &mut chunks { + if chunk.iter().fold(false, |acc, x| acc | (*x == *self)) { + return true; + } + } + // Scalar remainder + return chunks.remainder().iter().any(|x| *x == *self); + } + } + )* + }; +} + +impl_slice_contains!(u16, u32, u64, i16, i32, i64, f32, f64, usize, isize); diff --git a/library/core/src/slice/index.rs b/library/core/src/slice/index.rs index de1492e82ce7d..231ab7396adfd 100644 --- a/library/core/src/slice/index.rs +++ b/library/core/src/slice/index.rs @@ -31,12 +31,13 @@ where #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never), cold)] #[cfg_attr(feature = "panic_immediate_abort", inline)] #[track_caller] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] +#[rustc_allow_const_fn_unstable(const_eval_select)] const fn slice_start_index_len_fail(index: usize, len: usize) -> ! { + // FIXME(const-hack): once integer formatting in panics is possible, we + // should use the same implementation at compiletime and runtime. const_eval_select((index, len), slice_start_index_len_fail_ct, slice_start_index_len_fail_rt) } -// FIXME const-hack #[inline] #[track_caller] fn slice_start_index_len_fail_rt(index: usize, len: usize) -> ! { @@ -52,12 +53,13 @@ const fn slice_start_index_len_fail_ct(_: usize, _: usize) -> ! { #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never), cold)] #[cfg_attr(feature = "panic_immediate_abort", inline)] #[track_caller] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] +#[rustc_allow_const_fn_unstable(const_eval_select)] const fn slice_end_index_len_fail(index: usize, len: usize) -> ! { + // FIXME(const-hack): once integer formatting in panics is possible, we + // should use the same implementation at compiletime and runtime. const_eval_select((index, len), slice_end_index_len_fail_ct, slice_end_index_len_fail_rt) } -// FIXME const-hack #[inline] #[track_caller] fn slice_end_index_len_fail_rt(index: usize, len: usize) -> ! { @@ -73,12 +75,13 @@ const fn slice_end_index_len_fail_ct(_: usize, _: usize) -> ! { #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never), cold)] #[cfg_attr(feature = "panic_immediate_abort", inline)] #[track_caller] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] +#[rustc_allow_const_fn_unstable(const_eval_select)] const fn slice_index_order_fail(index: usize, end: usize) -> ! { + // FIXME(const-hack): once integer formatting in panics is possible, we + // should use the same implementation at compiletime and runtime. const_eval_select((index, end), slice_index_order_fail_ct, slice_index_order_fail_rt) } -// FIXME const-hack #[inline] #[track_caller] fn slice_index_order_fail_rt(index: usize, end: usize) -> ! { @@ -246,7 +249,6 @@ pub unsafe trait SliceIndex: private_slice_index::Sealed { /// The methods `index` and `index_mut` panic if the index is out of bounds. #[stable(feature = "slice_get_slice_impls", since = "1.15.0")] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] unsafe impl SliceIndex<[T]> for usize { type Output = T; @@ -311,7 +313,6 @@ unsafe impl SliceIndex<[T]> for usize { /// Because `IndexRange` guarantees `start <= end`, fewer checks are needed here /// than there are for a general `Range` (which might be `100..3`). -#[rustc_const_unstable(feature = "const_index_range_slice_index", issue = "none")] unsafe impl SliceIndex<[T]> for ops::IndexRange { type Output = [T]; @@ -386,7 +387,6 @@ unsafe impl SliceIndex<[T]> for ops::IndexRange { /// - the start of the range is greater than the end of the range or /// - the end of the range is out of bounds. #[stable(feature = "slice_get_slice_impls", since = "1.15.0")] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] unsafe impl SliceIndex<[T]> for ops::Range { type Output = [T]; @@ -522,7 +522,6 @@ unsafe impl SliceIndex<[T]> for range::Range { /// The methods `index` and `index_mut` panic if the end of the range is out of bounds. #[stable(feature = "slice_get_slice_impls", since = "1.15.0")] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] unsafe impl SliceIndex<[T]> for ops::RangeTo { type Output = [T]; @@ -561,7 +560,6 @@ unsafe impl SliceIndex<[T]> for ops::RangeTo { /// The methods `index` and `index_mut` panic if the start of the range is out of bounds. #[stable(feature = "slice_get_slice_impls", since = "1.15.0")] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] unsafe impl SliceIndex<[T]> for ops::RangeFrom { type Output = [T]; @@ -644,7 +642,6 @@ unsafe impl SliceIndex<[T]> for range::RangeFrom { } #[stable(feature = "slice_get_slice_impls", since = "1.15.0")] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] unsafe impl SliceIndex<[T]> for ops::RangeFull { type Output = [T]; @@ -684,7 +681,6 @@ unsafe impl SliceIndex<[T]> for ops::RangeFull { /// - the start of the range is greater than the end of the range or /// - the end of the range is out of bounds. #[stable(feature = "inclusive_range", since = "1.26.0")] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] unsafe impl SliceIndex<[T]> for ops::RangeInclusive { type Output = [T]; @@ -766,7 +762,6 @@ unsafe impl SliceIndex<[T]> for range::RangeInclusive { /// The methods `index` and `index_mut` panic if the end of the range is out of bounds. #[stable(feature = "inclusive_range", since = "1.26.0")] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] unsafe impl SliceIndex<[T]> for ops::RangeToInclusive { type Output = [T]; diff --git a/library/core/src/slice/iter.rs b/library/core/src/slice/iter.rs index 62b170a87d445..c5746157d01b2 100644 --- a/library/core/src/slice/iter.rs +++ b/library/core/src/slice/iter.rs @@ -11,7 +11,7 @@ use crate::iter::{ use crate::marker::PhantomData; use crate::mem::{self, SizedTypeProperties}; use crate::num::NonZero; -use crate::ptr::{self, without_provenance, without_provenance_mut, NonNull}; +use crate::ptr::{NonNull, without_provenance, without_provenance_mut}; use crate::{cmp, fmt}; #[stable(feature = "boxed_slice_into_iter", since = "1.80.0")] diff --git a/library/core/src/slice/iter/macros.rs b/library/core/src/slice/iter/macros.rs index c2a3819464410..830debe02ea2b 100644 --- a/library/core/src/slice/iter/macros.rs +++ b/library/core/src/slice/iter/macros.rs @@ -14,11 +14,11 @@ macro_rules! if_zst { if T::IS_ZST { // SAFETY: for ZSTs, the pointer is storing a provenance-free length, // so consuming and updating it as a `usize` is fine. - let $len = unsafe { &mut *ptr::addr_of_mut!($this.end_or_len).cast::() }; + let $len = unsafe { &mut *(&raw mut $this.end_or_len).cast::() }; $zst_body } else { // SAFETY: for non-ZSTs, the type invariant ensures it cannot be null - let $end = unsafe { &mut *ptr::addr_of_mut!($this.end_or_len).cast::>() }; + let $end = unsafe { &mut *(&raw mut $this.end_or_len).cast::>() }; $other_body } }}; @@ -30,7 +30,7 @@ macro_rules! if_zst { $zst_body } else { // SAFETY: for non-ZSTs, the type invariant ensures it cannot be null - let $end = unsafe { *ptr::addr_of!($this.end_or_len).cast::>() }; + let $end = unsafe { *(&raw const $this.end_or_len).cast::>() }; $other_body } }}; diff --git a/library/core/src/slice/memchr.rs b/library/core/src/slice/memchr.rs index da7ceb2dd0a0d..5760462326261 100644 --- a/library/core/src/slice/memchr.rs +++ b/library/core/src/slice/memchr.rs @@ -15,7 +15,7 @@ const USIZE_BYTES: usize = mem::size_of::(); /// bytes where the borrow propagated all the way to the most significant /// bit." #[inline] -#[rustc_const_stable(feature = "const_memchr", since = "1.65.0")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "const_memchr", since = "1.65.0"))] const fn contains_zero_byte(x: usize) -> bool { x.wrapping_sub(LO_USIZE) & !x & HI_USIZE != 0 } @@ -23,7 +23,7 @@ const fn contains_zero_byte(x: usize) -> bool { /// Returns the first index matching the byte `x` in `text`. #[inline] #[must_use] -#[rustc_const_stable(feature = "const_memchr", since = "1.65.0")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "const_memchr", since = "1.65.0"))] pub const fn memchr(x: u8, text: &[u8]) -> Option { // Fast path for small slices. if text.len() < 2 * USIZE_BYTES { @@ -34,7 +34,7 @@ pub const fn memchr(x: u8, text: &[u8]) -> Option { } #[inline] -#[rustc_const_stable(feature = "const_memchr", since = "1.65.0")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "const_memchr", since = "1.65.0"))] const fn memchr_naive(x: u8, text: &[u8]) -> Option { let mut i = 0; @@ -51,9 +51,8 @@ const fn memchr_naive(x: u8, text: &[u8]) -> Option { } #[rustc_allow_const_fn_unstable(const_cmp)] -#[rustc_allow_const_fn_unstable(const_slice_index)] #[rustc_allow_const_fn_unstable(const_align_offset)] -#[rustc_const_stable(feature = "const_memchr", since = "1.65.0")] +#[cfg_attr(bootstrap, rustc_const_stable(feature = "const_memchr", since = "1.65.0"))] const fn memchr_aligned(x: u8, text: &[u8]) -> Option { // Scan for a single byte value by reading two `usize` words at a time. // diff --git a/library/core/src/slice/mod.rs b/library/core/src/slice/mod.rs index 166189f4b6cf3..52d2179b04de1 100644 --- a/library/core/src/slice/mod.rs +++ b/library/core/src/slice/mod.rs @@ -39,11 +39,11 @@ mod raw; mod rotate; mod specialize; +#[stable(feature = "inherent_ascii_escape", since = "1.60.0")] +pub use ascii::EscapeAscii; #[unstable(feature = "str_internals", issue = "none")] #[doc(hidden)] pub use ascii::is_ascii_simple; -#[stable(feature = "inherent_ascii_escape", since = "1.60.0")] -pub use ascii::EscapeAscii; #[stable(feature = "slice_get_slice", since = "1.28.0")] pub use index::SliceIndex; #[unstable(feature = "slice_range", issue = "76393")] @@ -111,7 +111,6 @@ impl [T] { #[lang = "slice_len_fn"] #[stable(feature = "rust1", since = "1.0.0")] #[rustc_const_stable(feature = "const_slice_len", since = "1.39.0")] - #[rustc_allow_const_fn_unstable(ptr_metadata)] #[inline] #[must_use] pub const fn len(&self) -> usize { @@ -156,7 +155,7 @@ impl [T] { if let [first, ..] = self { Some(first) } else { None } } - /// Returns a mutable pointer to the first element of the slice, or `None` if it is empty. + /// Returns a mutable reference to the first element of the slice, or `None` if it is empty. /// /// # Examples /// @@ -172,7 +171,7 @@ impl [T] { /// assert_eq!(None, y.first_mut()); /// ``` #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_slice_first_last", issue = "83570")] + #[rustc_const_stable(feature = "const_slice_first_last", since = "1.83.0")] #[inline] #[must_use] pub const fn first_mut(&mut self) -> Option<&mut T> { @@ -214,7 +213,7 @@ impl [T] { /// assert_eq!(x, &[3, 4, 5]); /// ``` #[stable(feature = "slice_splits", since = "1.5.0")] - #[rustc_const_unstable(feature = "const_slice_first_last", issue = "83570")] + #[rustc_const_stable(feature = "const_slice_first_last", since = "1.83.0")] #[inline] #[must_use] pub const fn split_first_mut(&mut self) -> Option<(&mut T, &mut [T])> { @@ -256,7 +255,7 @@ impl [T] { /// assert_eq!(x, &[4, 5, 3]); /// ``` #[stable(feature = "slice_splits", since = "1.5.0")] - #[rustc_const_unstable(feature = "const_slice_first_last", issue = "83570")] + #[rustc_const_stable(feature = "const_slice_first_last", since = "1.83.0")] #[inline] #[must_use] pub const fn split_last_mut(&mut self) -> Option<(&mut T, &mut [T])> { @@ -298,7 +297,7 @@ impl [T] { /// assert_eq!(None, y.last_mut()); /// ``` #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_const_unstable(feature = "const_slice_first_last", issue = "83570")] + #[rustc_const_stable(feature = "const_slice_first_last", since = "1.83.0")] #[inline] #[must_use] pub const fn last_mut(&mut self) -> Option<&mut T> { @@ -353,7 +352,7 @@ impl [T] { /// ``` #[inline] #[stable(feature = "slice_first_last_chunk", since = "1.77.0")] - #[rustc_const_unstable(feature = "const_slice_first_last_chunk", issue = "111774")] + #[rustc_const_stable(feature = "const_slice_first_last_chunk", since = "1.83.0")] pub const fn first_chunk_mut(&mut self) -> Option<&mut [T; N]> { if self.len() < N { None @@ -418,7 +417,7 @@ impl [T] { /// ``` #[inline] #[stable(feature = "slice_first_last_chunk", since = "1.77.0")] - #[rustc_const_unstable(feature = "const_slice_first_last_chunk", issue = "111774")] + #[rustc_const_stable(feature = "const_slice_first_last_chunk", since = "1.83.0")] pub const fn split_first_chunk_mut( &mut self, ) -> Option<(&mut [T; N], &mut [T])> { @@ -488,7 +487,7 @@ impl [T] { /// ``` #[inline] #[stable(feature = "slice_first_last_chunk", since = "1.77.0")] - #[rustc_const_unstable(feature = "const_slice_first_last_chunk", issue = "111774")] + #[rustc_const_stable(feature = "const_slice_first_last_chunk", since = "1.83.0")] pub const fn split_last_chunk_mut( &mut self, ) -> Option<(&mut [T], &mut [T; N])> { @@ -529,7 +528,7 @@ impl [T] { None } else { // SAFETY: We manually verified the bounds of the slice. - // FIXME: Without const traits, we need this instead of `get_unchecked`. + // FIXME(const-hack): Without const traits, we need this instead of `get_unchecked`. let last = unsafe { self.split_at_unchecked(self.len() - N).1 }; // SAFETY: We explicitly check for the correct number of elements, @@ -557,13 +556,13 @@ impl [T] { /// ``` #[inline] #[stable(feature = "slice_first_last_chunk", since = "1.77.0")] - #[rustc_const_unstable(feature = "const_slice_first_last_chunk", issue = "111774")] + #[rustc_const_stable(feature = "const_slice_first_last_chunk", since = "1.83.0")] pub const fn last_chunk_mut(&mut self) -> Option<&mut [T; N]> { if self.len() < N { None } else { // SAFETY: We manually verified the bounds of the slice. - // FIXME: Without const traits, we need this instead of `get_unchecked`. + // FIXME(const-hack): Without const traits, we need this instead of `get_unchecked`. let last = unsafe { self.split_at_mut_unchecked(self.len() - N).1 }; // SAFETY: We explicitly check for the correct number of elements, @@ -765,7 +764,6 @@ impl [T] { /// ``` #[stable(feature = "rust1", since = "1.0.0")] #[rustc_const_stable(feature = "const_ptr_offset", since = "1.61.0")] - #[rustc_allow_const_fn_unstable(const_mut_refs)] #[rustc_never_returns_null_ptr] #[inline(always)] #[must_use] @@ -846,7 +844,6 @@ impl [T] { /// [`as_mut_ptr`]: slice::as_mut_ptr #[stable(feature = "slice_ptr_range", since = "1.48.0")] #[rustc_const_stable(feature = "const_ptr_offset", since = "1.61.0")] - #[rustc_allow_const_fn_unstable(const_mut_refs)] #[inline] #[must_use] pub const fn as_mut_ptr_range(&mut self) -> Range<*mut T> { @@ -883,8 +880,8 @@ impl [T] { pub const fn swap(&mut self, a: usize, b: usize) { // FIXME: use swap_unchecked here (https://github.com/rust-lang/rust/pull/88540#issuecomment-944344343) // Can't take two mutable loans from one vector, so instead use raw pointers. - let pa = ptr::addr_of_mut!(self[a]); - let pb = ptr::addr_of_mut!(self[b]); + let pa = &raw mut self[a]; + let pb = &raw mut self[b]; // SAFETY: `pa` and `pb` have been created from safe mutable references and refer // to elements in the slice and therefore are guaranteed to be valid and aligned. // Note that accessing the elements behind `a` and `b` is checked and will @@ -1010,6 +1007,7 @@ impl [T] { /// ``` #[stable(feature = "rust1", since = "1.0.0")] #[inline] + #[cfg_attr(not(test), rustc_diagnostic_item = "slice_iter")] pub fn iter(&self) -> Iter<'_, T> { Iter::new(self) } @@ -1266,6 +1264,7 @@ impl [T] { /// // let chunks: &[[_; 0]] = slice.as_chunks_unchecked() // Zero-length chunks are never allowed /// ``` #[unstable(feature = "slice_as_chunks", issue = "74985")] + #[rustc_const_unstable(feature = "slice_as_chunks", issue = "74985")] #[inline] #[must_use] pub const unsafe fn as_chunks_unchecked(&self) -> &[[T; N]] { @@ -1311,6 +1310,7 @@ impl [T] { /// assert_eq!(chunks, &[['R', 'u'], ['s', 't']]); /// ``` #[unstable(feature = "slice_as_chunks", issue = "74985")] + #[rustc_const_unstable(feature = "slice_as_chunks", issue = "74985")] #[inline] #[track_caller] #[must_use] @@ -1345,6 +1345,7 @@ impl [T] { /// assert_eq!(chunks, &[['o', 'r'], ['e', 'm']]); /// ``` #[unstable(feature = "slice_as_chunks", issue = "74985")] + #[rustc_const_unstable(feature = "slice_as_chunks", issue = "74985")] #[inline] #[track_caller] #[must_use] @@ -1423,6 +1424,7 @@ impl [T] { /// // let chunks: &[[_; 0]] = slice.as_chunks_unchecked_mut() // Zero-length chunks are never allowed /// ``` #[unstable(feature = "slice_as_chunks", issue = "74985")] + #[rustc_const_unstable(feature = "slice_as_chunks", issue = "74985")] #[inline] #[must_use] pub const unsafe fn as_chunks_unchecked_mut(&mut self) -> &mut [[T; N]] { @@ -1463,6 +1465,7 @@ impl [T] { /// assert_eq!(v, &[1, 1, 2, 2, 9]); /// ``` #[unstable(feature = "slice_as_chunks", issue = "74985")] + #[rustc_const_unstable(feature = "slice_as_chunks", issue = "74985")] #[inline] #[track_caller] #[must_use] @@ -1503,6 +1506,7 @@ impl [T] { /// assert_eq!(v, &[9, 1, 1, 2, 2]); /// ``` #[unstable(feature = "slice_as_chunks", issue = "74985")] + #[rustc_const_unstable(feature = "slice_as_chunks", issue = "74985")] #[inline] #[track_caller] #[must_use] @@ -1862,7 +1866,6 @@ impl [T] { /// ``` #[stable(feature = "rust1", since = "1.0.0")] #[rustc_const_stable(feature = "const_slice_split_at_not_mut", since = "1.71.0")] - #[rustc_allow_const_fn_unstable(split_at_checked)] #[inline] #[track_caller] #[must_use] @@ -1899,7 +1902,7 @@ impl [T] { #[inline] #[track_caller] #[must_use] - #[rustc_const_unstable(feature = "const_slice_split_at_mut", issue = "101804")] + #[rustc_const_stable(feature = "const_slice_split_at_mut", since = "1.83.0")] pub const fn split_at_mut(&mut self, mid: usize) -> (&mut [T], &mut [T]) { match self.split_at_mut_checked(mid) { Some(pair) => pair, @@ -1952,7 +1955,7 @@ impl [T] { #[inline] #[must_use] pub const unsafe fn split_at_unchecked(&self, mid: usize) -> (&[T], &[T]) { - // HACK: the const function `from_raw_parts` is used to make this + // FIXME(const-hack): the const function `from_raw_parts` is used to make this // function const; previously the implementation used // `(self.get_unchecked(..mid), self.get_unchecked(mid..))` @@ -2001,7 +2004,7 @@ impl [T] { /// assert_eq!(v, [1, 2, 3, 4, 5, 6]); /// ``` #[stable(feature = "slice_split_at_unchecked", since = "1.79.0")] - #[rustc_const_unstable(feature = "const_slice_split_at_mut", issue = "101804")] + #[rustc_const_stable(feature = "const_slice_split_at_mut", since = "1.83.0")] #[inline] #[must_use] pub const unsafe fn split_at_mut_unchecked(&mut self, mid: usize) -> (&mut [T], &mut [T]) { @@ -2101,7 +2104,7 @@ impl [T] { /// assert_eq!(None, v.split_at_mut_checked(7)); /// ``` #[stable(feature = "split_at_checked", since = "1.80.0")] - #[rustc_const_unstable(feature = "const_slice_split_at_mut", issue = "101804")] + #[rustc_const_stable(feature = "const_slice_split_at_mut", since = "1.83.0")] #[inline] #[must_use] pub const fn split_at_mut_checked(&mut self, mid: usize) -> Option<(&mut [T], &mut [T])> { diff --git a/library/core/src/slice/raw.rs b/library/core/src/slice/raw.rs index 2cf3fecb47542..89840881c4d90 100644 --- a/library/core/src/slice/raw.rs +++ b/library/core/src/slice/raw.rs @@ -132,7 +132,7 @@ pub const unsafe fn from_raw_parts<'a, T>(data: *const T, len: usize) -> &'a [T] align: usize = align_of::(), len: usize = len, ) => - ub_checks::is_aligned_and_not_null(data, align) + ub_checks::is_aligned_and_not_null(data, align, false) && ub_checks::is_valid_allocation_size(size, len) ); &*ptr::slice_from_raw_parts(data, len) @@ -171,7 +171,7 @@ pub const unsafe fn from_raw_parts<'a, T>(data: *const T, len: usize) -> &'a [T] /// [`NonNull::dangling()`]: ptr::NonNull::dangling #[inline] #[stable(feature = "rust1", since = "1.0.0")] -#[rustc_const_unstable(feature = "const_slice_from_raw_parts_mut", issue = "67456")] +#[rustc_const_stable(feature = "const_slice_from_raw_parts_mut", since = "1.83.0")] #[must_use] #[rustc_diagnostic_item = "slice_from_raw_parts_mut"] pub const unsafe fn from_raw_parts_mut<'a, T>(data: *mut T, len: usize) -> &'a mut [T] { @@ -186,7 +186,7 @@ pub const unsafe fn from_raw_parts_mut<'a, T>(data: *mut T, len: usize) -> &'a m align: usize = align_of::(), len: usize = len, ) => - ub_checks::is_aligned_and_not_null(data, align) + ub_checks::is_aligned_and_not_null(data, align, false) && ub_checks::is_valid_allocation_size(size, len) ); &mut *ptr::slice_from_raw_parts_mut(data, len) @@ -203,7 +203,7 @@ pub const fn from_ref(s: &T) -> &[T] { /// Converts a reference to T into a slice of length 1 (without copying). #[stable(feature = "from_ref", since = "1.28.0")] -#[rustc_const_unstable(feature = "const_slice_from_ref", issue = "90206")] +#[rustc_const_stable(feature = "const_slice_from_ref", since = "1.83.0")] #[must_use] pub const fn from_mut(s: &mut T) -> &mut [T] { array::from_mut(s) diff --git a/library/core/src/slice/sort/select.rs b/library/core/src/slice/sort/select.rs index f6529f23bcb3f..3358c03d30a9b 100644 --- a/library/core/src/slice/sort/select.rs +++ b/library/core/src/slice/sort/select.rs @@ -7,6 +7,7 @@ //! better performance than one would get using heapsort as fallback. use crate::mem::{self, SizedTypeProperties}; +#[cfg(not(feature = "optimize_for_size"))] use crate::slice::sort::shared::pivot::choose_pivot; use crate::slice::sort::shared::smallsort::insertion_sort_shift_left; use crate::slice::sort::unstable::quicksort::partition; @@ -40,7 +41,13 @@ where let min_idx = min_index(v, &mut is_less).unwrap(); v.swap(min_idx, index); } else { - partition_at_index_loop(v, index, None, &mut is_less); + cfg_if! { + if #[cfg(feature = "optimize_for_size")] { + median_of_medians(v, &mut is_less, index); + } else { + partition_at_index_loop(v, index, None, &mut is_less); + } + } } let (left, right) = v.split_at_mut(index); @@ -53,6 +60,7 @@ where // most once, it doesn't make sense to use something more sophisticated than insertion-sort. const INSERTION_SORT_THRESHOLD: usize = 16; +#[cfg(not(feature = "optimize_for_size"))] fn partition_at_index_loop<'a, T, F>( mut v: &'a mut [T], mut index: usize, @@ -169,6 +177,7 @@ fn median_of_medians bool>(mut v: &mut [T], is_less: &mut if v.len() >= 2 { insertion_sort_shift_left(v, 1, is_less); } + return; } diff --git a/library/core/src/slice/sort/shared/mod.rs b/library/core/src/slice/sort/shared/mod.rs index ad1171bfc6a0a..e2cdcb3dd511d 100644 --- a/library/core/src/slice/sort/shared/mod.rs +++ b/library/core/src/slice/sort/shared/mod.rs @@ -1,3 +1,5 @@ +#![cfg_attr(any(feature = "optimize_for_size", target_pointer_width = "16"), allow(dead_code))] + use crate::marker::Freeze; pub(crate) mod pivot; diff --git a/library/core/src/slice/sort/shared/smallsort.rs b/library/core/src/slice/sort/shared/smallsort.rs index fae628a7c1474..6adf779a72f0c 100644 --- a/library/core/src/slice/sort/shared/smallsort.rs +++ b/library/core/src/slice/sort/shared/smallsort.rs @@ -378,7 +378,12 @@ where /// Swap two values in the slice pointed to by `v_base` at the position `a_pos` and `b_pos` if the /// value at position `b_pos` is less than the one at position `a_pos`. -pub unsafe fn swap_if_less(v_base: *mut T, a_pos: usize, b_pos: usize, is_less: &mut F) +/// +/// Purposefully not marked `#[inline]`, despite us wanting it to be inlined for integers like +/// types. `is_less` could be a huge function and we want to give the compiler an option to +/// not inline this function. For the same reasons that this function is very perf critical +/// it should be in the same module as the functions that use it. +unsafe fn swap_if_less(v_base: *mut T, a_pos: usize, b_pos: usize, is_less: &mut F) where F: FnMut(&T, &T) -> bool, { diff --git a/library/core/src/slice/sort/stable/mod.rs b/library/core/src/slice/sort/stable/mod.rs index a383b0f589ccf..7adcc83b818d1 100644 --- a/library/core/src/slice/sort/stable/mod.rs +++ b/library/core/src/slice/sort/stable/mod.rs @@ -1,15 +1,24 @@ //! This module contains the entry points for `slice::sort`. +#[cfg(not(any(feature = "optimize_for_size", target_pointer_width = "16")))] +use crate::cmp; +use crate::intrinsics; use crate::mem::{self, MaybeUninit, SizedTypeProperties}; +#[cfg(not(any(feature = "optimize_for_size", target_pointer_width = "16")))] use crate::slice::sort::shared::smallsort::{ - insertion_sort_shift_left, StableSmallSortTypeImpl, SMALL_SORT_GENERAL_SCRATCH_LEN, + SMALL_SORT_GENERAL_SCRATCH_LEN, StableSmallSortTypeImpl, insertion_sort_shift_left, }; -use crate::{cmp, intrinsics}; -pub(crate) mod drift; pub(crate) mod merge; + +#[cfg(not(any(feature = "optimize_for_size", target_pointer_width = "16")))] +pub(crate) mod drift; +#[cfg(not(any(feature = "optimize_for_size", target_pointer_width = "16")))] pub(crate) mod quicksort; +#[cfg(any(feature = "optimize_for_size", target_pointer_width = "16"))] +pub(crate) mod tiny; + /// Stable sort called driftsort by Orson Peters and Lukas Bergdoll. /// Design document: /// @@ -30,25 +39,53 @@ pub fn sort bool, BufT: BufGuard>(v: &mut [T], is_less return; } - // More advanced sorting methods than insertion sort are faster if called in - // a hot loop for small inputs, but for general-purpose code the small - // binary size of insertion sort is more important. The instruction cache in - // modern processors is very valuable, and for a single sort call in general - // purpose code any gains from an advanced method are cancelled by i-cache - // misses during the sort, and thrashing the i-cache for surrounding code. - const MAX_LEN_ALWAYS_INSERTION_SORT: usize = 20; - if intrinsics::likely(len <= MAX_LEN_ALWAYS_INSERTION_SORT) { - insertion_sort_shift_left(v, 1, is_less); - return; - } + cfg_if! { + if #[cfg(any(feature = "optimize_for_size", target_pointer_width = "16"))] { + let alloc_len = len / 2; + + cfg_if! { + if #[cfg(target_pointer_width = "16")] { + let mut heap_buf = BufT::with_capacity(alloc_len); + let scratch = heap_buf.as_uninit_slice_mut(); + } else { + // For small inputs 4KiB of stack storage suffices, which allows us to avoid + // calling the (de-)allocator. Benchmarks showed this was quite beneficial. + let mut stack_buf = AlignedStorage::::new(); + let stack_scratch = stack_buf.as_uninit_slice_mut(); + let mut heap_buf; + let scratch = if stack_scratch.len() >= alloc_len { + stack_scratch + } else { + heap_buf = BufT::with_capacity(alloc_len); + heap_buf.as_uninit_slice_mut() + }; + } + } - driftsort_main::(v, is_less); + tiny::mergesort(v, scratch, is_less); + } else { + // More advanced sorting methods than insertion sort are faster if called in + // a hot loop for small inputs, but for general-purpose code the small + // binary size of insertion sort is more important. The instruction cache in + // modern processors is very valuable, and for a single sort call in general + // purpose code any gains from an advanced method are cancelled by i-cache + // misses during the sort, and thrashing the i-cache for surrounding code. + const MAX_LEN_ALWAYS_INSERTION_SORT: usize = 20; + if intrinsics::likely(len <= MAX_LEN_ALWAYS_INSERTION_SORT) { + insertion_sort_shift_left(v, 1, is_less); + return; + } + + driftsort_main::(v, is_less); + } + } } /// See [`sort`] /// /// Deliberately don't inline the main sorting routine entrypoint to ensure the /// inlined insertion sort i-cache footprint remains minimal. +#[cfg(not(any(feature = "optimize_for_size", target_pointer_width = "16")))] #[inline(never)] fn driftsort_main bool, BufT: BufGuard>(v: &mut [T], is_less: &mut F) { // By allocating n elements of memory we can ensure the entire input can diff --git a/library/core/src/slice/sort/stable/quicksort.rs b/library/core/src/slice/sort/stable/quicksort.rs index 3319d67ab52fa..0c8308bfce00e 100644 --- a/library/core/src/slice/sort/stable/quicksort.rs +++ b/library/core/src/slice/sort/stable/quicksort.rs @@ -1,9 +1,9 @@ //! This module contains a stable quicksort and partition implementation. use crate::mem::{self, ManuallyDrop, MaybeUninit}; +use crate::slice::sort::shared::FreezeMarker; use crate::slice::sort::shared::pivot::choose_pivot; use crate::slice::sort::shared::smallsort::StableSmallSortTypeImpl; -use crate::slice::sort::shared::FreezeMarker; use crate::{intrinsics, ptr}; /// Sorts `v` recursively using quicksort. diff --git a/library/core/src/slice/sort/stable/tiny.rs b/library/core/src/slice/sort/stable/tiny.rs new file mode 100644 index 0000000000000..071ab8e107fe3 --- /dev/null +++ b/library/core/src/slice/sort/stable/tiny.rs @@ -0,0 +1,41 @@ +//! Binary-size optimized mergesort inspired by https://github.com/voultapher/tiny-sort-rs. + +use crate::mem::MaybeUninit; +use crate::ptr; +use crate::slice::sort::stable::merge; + +/// Tiny recursive top-down merge sort optimized for binary size. It has no adaptiveness whatsoever, +/// no run detection, etc. +#[inline(always)] +pub fn mergesort bool>( + v: &mut [T], + scratch: &mut [MaybeUninit], + is_less: &mut F, +) { + let len = v.len(); + + if len > 2 { + let mid = len / 2; + + // SAFETY: mid is in-bounds. + unsafe { + // Sort the left half recursively. + mergesort(v.get_unchecked_mut(..mid), scratch, is_less); + // Sort the right half recursively. + mergesort(v.get_unchecked_mut(mid..), scratch, is_less); + } + + merge::merge(v, scratch, mid, is_less); + } else if len == 2 { + // SAFETY: We checked the len, the pointers we create are valid and don't overlap. + unsafe { + let v_base = v.as_mut_ptr(); + let v_a = v_base; + let v_b = v_base.add(1); + + if is_less(&*v_b, &*v_a) { + ptr::swap_nonoverlapping(v_a, v_b, 1); + } + } + } +} diff --git a/library/core/src/slice/sort/unstable/heapsort.rs b/library/core/src/slice/sort/unstable/heapsort.rs index 27e2ad588ea09..85231779d031f 100644 --- a/library/core/src/slice/sort/unstable/heapsort.rs +++ b/library/core/src/slice/sort/unstable/heapsort.rs @@ -1,46 +1,46 @@ //! This module contains a branchless heapsort as fallback for unstable quicksort. -use crate::{intrinsics, ptr}; +use crate::{cmp, intrinsics, ptr}; /// Sorts `v` using heapsort, which guarantees *O*(*n* \* log(*n*)) worst-case. /// /// Never inline this, it sits the main hot-loop in `recurse` and is meant as unlikely algorithmic /// fallback. -/// -/// SAFETY: The caller has to guarantee that `v.len()` >= 2. #[inline(never)] -pub(crate) unsafe fn heapsort(v: &mut [T], is_less: &mut F) +pub(crate) fn heapsort(v: &mut [T], is_less: &mut F) where F: FnMut(&T, &T) -> bool, { - // SAFETY: See function safety. - unsafe { - intrinsics::assume(v.len() >= 2); - - // Build the heap in linear time. - for i in (0..v.len() / 2).rev() { - sift_down(v, i, is_less); - } + let len = v.len(); - // Pop maximal elements from the heap. - for i in (1..v.len()).rev() { + for i in (0..len + len / 2).rev() { + let sift_idx = if i >= len { + i - len + } else { v.swap(0, i); - sift_down(&mut v[..i], 0, is_less); + 0 + }; + + // SAFETY: The above calculation ensures that `sift_idx` is either 0 or + // `(len..(len + (len / 2))) - len`, which simplifies to `0..(len / 2)`. + // This guarantees the required `sift_idx <= len`. + unsafe { + sift_down(&mut v[..cmp::min(i, len)], sift_idx, is_less); } } } // This binary heap respects the invariant `parent >= child`. // -// SAFETY: The caller has to guarantee that node < `v.len()`. -#[inline(never)] +// SAFETY: The caller has to guarantee that `node <= v.len()`. +#[inline(always)] unsafe fn sift_down(v: &mut [T], mut node: usize, is_less: &mut F) where F: FnMut(&T, &T) -> bool, { // SAFETY: See function safety. unsafe { - intrinsics::assume(node < v.len()); + intrinsics::assume(node <= v.len()); } let len = v.len(); @@ -69,9 +69,7 @@ where break; } - // Swap `node` with the greater child, move one step down, and continue sifting. This - // could be ptr::swap_nonoverlapping but that adds a significant amount of binary-size. - ptr::swap(v_base.add(node), v_base.add(child)); + ptr::swap_nonoverlapping(v_base.add(node), v_base.add(child), 1); } node = child; diff --git a/library/core/src/slice/sort/unstable/mod.rs b/library/core/src/slice/sort/unstable/mod.rs index 932e01f4401e5..2eb653c4601a7 100644 --- a/library/core/src/slice/sort/unstable/mod.rs +++ b/library/core/src/slice/sort/unstable/mod.rs @@ -2,7 +2,9 @@ use crate::intrinsics; use crate::mem::SizedTypeProperties; +#[cfg(not(any(feature = "optimize_for_size", target_pointer_width = "16")))] use crate::slice::sort::shared::find_existing_run; +#[cfg(not(any(feature = "optimize_for_size", target_pointer_width = "16")))] use crate::slice::sort::shared::smallsort::insertion_sort_shift_left; pub(crate) mod heapsort; @@ -28,25 +30,32 @@ pub fn sort bool>(v: &mut [T], is_less: &mut F) { return; } - // More advanced sorting methods than insertion sort are faster if called in - // a hot loop for small inputs, but for general-purpose code the small - // binary size of insertion sort is more important. The instruction cache in - // modern processors is very valuable, and for a single sort call in general - // purpose code any gains from an advanced method are cancelled by i-cache - // misses during the sort, and thrashing the i-cache for surrounding code. - const MAX_LEN_ALWAYS_INSERTION_SORT: usize = 20; - if intrinsics::likely(len <= MAX_LEN_ALWAYS_INSERTION_SORT) { - insertion_sort_shift_left(v, 1, is_less); - return; - } + cfg_if! { + if #[cfg(any(feature = "optimize_for_size", target_pointer_width = "16"))] { + heapsort::heapsort(v, is_less); + } else { + // More advanced sorting methods than insertion sort are faster if called in + // a hot loop for small inputs, but for general-purpose code the small + // binary size of insertion sort is more important. The instruction cache in + // modern processors is very valuable, and for a single sort call in general + // purpose code any gains from an advanced method are cancelled by i-cache + // misses during the sort, and thrashing the i-cache for surrounding code. + const MAX_LEN_ALWAYS_INSERTION_SORT: usize = 20; + if intrinsics::likely(len <= MAX_LEN_ALWAYS_INSERTION_SORT) { + insertion_sort_shift_left(v, 1, is_less); + return; + } - ipnsort(v, is_less); + ipnsort(v, is_less); + } + } } /// See [`sort`] /// /// Deliberately don't inline the main sorting routine entrypoint to ensure the /// inlined insertion sort i-cache footprint remains minimal. +#[cfg(not(any(feature = "optimize_for_size", target_pointer_width = "16")))] #[inline(never)] fn ipnsort(v: &mut [T], is_less: &mut F) where diff --git a/library/core/src/slice/sort/unstable/quicksort.rs b/library/core/src/slice/sort/unstable/quicksort.rs index cd53656e9b4b8..4feef5deeb0fb 100644 --- a/library/core/src/slice/sort/unstable/quicksort.rs +++ b/library/core/src/slice/sort/unstable/quicksort.rs @@ -1,8 +1,12 @@ //! This module contains an unstable quicksort and two partition implementations. use crate::mem::{self, ManuallyDrop}; +#[cfg(not(feature = "optimize_for_size"))] use crate::slice::sort::shared::pivot::choose_pivot; +#[cfg(not(feature = "optimize_for_size"))] use crate::slice::sort::shared::smallsort::UnstableSmallSortTypeImpl; +#[cfg(not(feature = "optimize_for_size"))] +use crate::slice::sort::unstable::heapsort; use crate::{intrinsics, ptr}; /// Sorts `v` recursively. @@ -11,6 +15,7 @@ use crate::{intrinsics, ptr}; /// /// `limit` is the number of allowed imbalanced partitions before switching to `heapsort`. If zero, /// this function will immediately switch to heapsort. +#[cfg(not(feature = "optimize_for_size"))] pub(crate) fn quicksort<'a, T, F>( mut v: &'a mut [T], mut ancestor_pivot: Option<&'a T>, @@ -28,10 +33,7 @@ pub(crate) fn quicksort<'a, T, F>( // If too many bad pivot choices were made, simply fall back to heapsort in order to // guarantee `O(N x log(N))` worst-case. if limit == 0 { - // SAFETY: We assume the `small_sort` threshold is at least 1. - unsafe { - crate::slice::sort::unstable::heapsort::heapsort(v, is_less); - } + heapsort::heapsort(v, is_less); return; } @@ -98,13 +100,15 @@ where return 0; } - // Allows for panic-free code-gen by proving this property to the compiler. if pivot >= len { intrinsics::abort(); } - // Place the pivot at the beginning of slice. - v.swap(0, pivot); + // SAFETY: We checked that `pivot` is in-bounds. + unsafe { + // Place the pivot at the beginning of slice. + v.swap_unchecked(0, pivot); + } let (pivot, v_without_pivot) = v.split_at_mut(1); // Assuming that Rust generates noalias LLVM IR we can be sure that a partition function @@ -118,8 +122,15 @@ where // compile-time by only instantiating the code that is needed. Idea by Frank Steffahn. let num_lt = (const { inst_partition::() })(v_without_pivot, pivot, is_less); - // Place the pivot between the two partitions. - v.swap(0, num_lt); + if num_lt >= len { + intrinsics::abort(); + } + + // SAFETY: We checked that `num_lt` is in-bounds. + unsafe { + // Place the pivot between the two partitions. + v.swap_unchecked(0, num_lt); + } num_lt } @@ -129,7 +140,13 @@ const fn inst_partition bool>() -> fn(&mut [T], &T, &mut if mem::size_of::() <= MAX_BRANCHLESS_PARTITION_SIZE { // Specialize for types that are relatively cheap to copy, where branchless optimizations // have large leverage e.g. `u64` and `String`. - partition_lomuto_branchless_cyclic:: + cfg_if! { + if #[cfg(feature = "optimize_for_size")] { + partition_lomuto_branchless_simple:: + } else { + partition_lomuto_branchless_cyclic:: + } + } } else { partition_hoare_branchy_cyclic:: } @@ -215,6 +232,7 @@ where } } +#[cfg(not(feature = "optimize_for_size"))] struct PartitionState { // The current element that is being looked at, scans left to right through slice. right: *mut T, @@ -225,6 +243,7 @@ struct PartitionState { gap: GapGuardRaw, } +#[cfg(not(feature = "optimize_for_size"))] fn partition_lomuto_branchless_cyclic(v: &mut [T], pivot: &T, is_less: &mut F) -> usize where F: FnMut(&T, &T) -> bool, @@ -316,6 +335,27 @@ where } } +#[cfg(feature = "optimize_for_size")] +fn partition_lomuto_branchless_simple bool>( + v: &mut [T], + pivot: &T, + is_less: &mut F, +) -> usize { + let mut left = 0; + + for right in 0..v.len() { + // SAFETY: `left` can at max be incremented by 1 each loop iteration, which implies that + // left <= right and that both are in-bounds. + unsafe { + let right_is_lt = is_less(v.get_unchecked(right), pivot); + v.swap_unchecked(left, right); + left += right_is_lt as usize; + } + } + + left +} + struct GapGuard { pos: *mut T, value: ManuallyDrop, @@ -333,11 +373,13 @@ impl Drop for GapGuard { /// Ideally this wouldn't be needed and we could just use the regular GapGuard. /// See comment in [`partition_lomuto_branchless_cyclic`]. +#[cfg(not(feature = "optimize_for_size"))] struct GapGuardRaw { pos: *mut T, value: *mut T, } +#[cfg(not(feature = "optimize_for_size"))] impl Drop for GapGuardRaw { fn drop(&mut self) { // SAFETY: `self` MUST be constructed in a way that makes copying the gap value into diff --git a/library/core/src/str/converts.rs b/library/core/src/str/converts.rs index 1956a04829d1d..c997e5e443dac 100644 --- a/library/core/src/str/converts.rs +++ b/library/core/src/str/converts.rs @@ -1,7 +1,7 @@ //! Ways to create a `str` from bytes slice. -use super::validations::run_utf8_validation; use super::Utf8Error; +use super::validations::run_utf8_validation; use crate::{mem, ptr}; /// Converts a slice of bytes to a string slice. @@ -85,7 +85,7 @@ use crate::{mem, ptr}; #[rustc_allow_const_fn_unstable(str_internals)] #[rustc_diagnostic_item = "str_from_utf8"] pub const fn from_utf8(v: &[u8]) -> Result<&str, Utf8Error> { - // FIXME: This should use `?` again, once it's `const` + // FIXME(const-hack): This should use `?` again, once it's `const` match run_utf8_validation(v) { Ok(_) => { // SAFETY: validation succeeded. @@ -129,7 +129,7 @@ pub const fn from_utf8(v: &[u8]) -> Result<&str, Utf8Error> { #[rustc_const_unstable(feature = "const_str_from_utf8", issue = "91006")] #[rustc_diagnostic_item = "str_from_utf8_mut"] pub const fn from_utf8_mut(v: &mut [u8]) -> Result<&mut str, Utf8Error> { - // This should use `?` again, once it's `const` + // FIXME(const-hack): This should use `?` again, once it's `const` match run_utf8_validation(v) { Ok(_) => { // SAFETY: validation succeeded. @@ -195,7 +195,7 @@ pub const unsafe fn from_utf8_unchecked(v: &[u8]) -> &str { #[inline] #[must_use] #[stable(feature = "str_mut_extras", since = "1.20.0")] -#[rustc_const_unstable(feature = "const_str_from_utf8_unchecked_mut", issue = "91005")] +#[rustc_const_stable(feature = "const_str_from_utf8_unchecked_mut", since = "1.83.0")] #[rustc_diagnostic_item = "str_from_utf8_unchecked_mut"] pub const unsafe fn from_utf8_unchecked_mut(v: &mut [u8]) -> &mut str { // SAFETY: the caller must guarantee that the bytes `v` diff --git a/library/core/src/str/error.rs b/library/core/src/str/error.rs index a11b5add42ebf..4c8231a2286e1 100644 --- a/library/core/src/str/error.rs +++ b/library/core/src/str/error.rs @@ -100,7 +100,7 @@ impl Utf8Error { #[must_use] #[inline] pub const fn error_len(&self) -> Option { - // FIXME: This should become `map` again, once it's `const` + // FIXME(const-hack): This should become `map` again, once it's `const` match self.error_len { Some(len) => Some(len as usize), None => None, diff --git a/library/core/src/str/iter.rs b/library/core/src/str/iter.rs index d9301a8a66ea2..425c4eaee28ee 100644 --- a/library/core/src/str/iter.rs +++ b/library/core/src/str/iter.rs @@ -3,8 +3,8 @@ use super::pattern::{DoubleEndedSearcher, Pattern, ReverseSearcher, Searcher}; use super::validations::{next_code_point, next_code_point_reverse}; use super::{ - from_utf8_unchecked, BytesIsNotEmpty, CharEscapeDebugContinue, CharEscapeDefault, - CharEscapeUnicode, IsAsciiWhitespace, IsNotEmpty, IsWhitespace, LinesMap, UnsafeBytesToStr, + BytesIsNotEmpty, CharEscapeDebugContinue, CharEscapeDefault, CharEscapeUnicode, + IsAsciiWhitespace, IsNotEmpty, IsWhitespace, LinesMap, UnsafeBytesToStr, from_utf8_unchecked, }; use crate::fmt::{self, Write}; use crate::iter::{ diff --git a/library/core/src/str/lossy.rs b/library/core/src/str/lossy.rs index 3f31107acf050..e7677c8317a9f 100644 --- a/library/core/src/str/lossy.rs +++ b/library/core/src/str/lossy.rs @@ -8,6 +8,8 @@ impl [u8] { /// Creates an iterator over the contiguous valid UTF-8 ranges of this /// slice, and the non-UTF-8 fragments in between. /// + /// See the [`Utf8Chunk`] type for documenation of the items yielded by this iterator. + /// /// # Examples /// /// This function formats arbitrary but mostly-UTF-8 bytes into Rust source @@ -148,6 +150,8 @@ impl fmt::Debug for Debug<'_> { /// If you want a simple conversion from UTF-8 byte slices to string slices, /// [`from_utf8`] is easier to use. /// +/// See the [`Utf8Chunk`] type for documenation of the items yielded by this iterator. +/// /// [byteslice]: slice /// [`from_utf8`]: super::from_utf8 /// diff --git a/library/core/src/str/mod.rs b/library/core/src/str/mod.rs index cf9f1bfc0eb72..58d6e07de8da6 100644 --- a/library/core/src/str/mod.rs +++ b/library/core/src/str/mod.rs @@ -134,6 +134,7 @@ impl str { /// ``` #[stable(feature = "rust1", since = "1.0.0")] #[rustc_const_stable(feature = "const_str_len", since = "1.39.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "str_len")] #[must_use] #[inline] pub const fn len(&self) -> usize { @@ -184,8 +185,9 @@ impl str { /// ``` #[must_use] #[stable(feature = "is_char_boundary", since = "1.9.0")] + #[rustc_const_unstable(feature = "const_is_char_boundary", issue = "131516")] #[inline] - pub fn is_char_boundary(&self, index: usize) -> bool { + pub const fn is_char_boundary(&self, index: usize) -> bool { // 0 is always ok. // Test for 0 explicitly so that it can optimize out the check // easily and skip reading string data for that case. @@ -194,8 +196,8 @@ impl str { return true; } - match self.as_bytes().get(index) { - // For `None` we have two options: + if index >= self.len() { + // For `true` we have two options: // // - index == self.len() // Empty strings are valid, so return true @@ -204,13 +206,13 @@ impl str { // // The check is placed exactly here, because it improves generated // code on higher opt-levels. See PR #84751 for more details. - None => index == self.len(), - - Some(&b) => b.is_utf8_char_boundary(), + index == self.len() + } else { + self.as_bytes()[index].is_utf8_char_boundary() } } - /// Finds the closest `x` not exceeding `index` where `is_char_boundary(x)` is `true`. + /// Finds the closest `x` not exceeding `index` where [`is_char_boundary(x)`] is `true`. /// /// This method can help you truncate a string so that it's still valid UTF-8, but doesn't /// exceed a given number of bytes. Note that this is done purely at the character level @@ -218,6 +220,8 @@ impl str { /// split. For example, the emoji 🧑‍🔬 (scientist) could be split so that the string only /// includes 🧑 (person) instead. /// + /// [`is_char_boundary(x)`]: Self::is_char_boundary + /// /// # Examples /// /// ``` @@ -246,7 +250,7 @@ impl str { } } - /// Finds the closest `x` not below `index` where `is_char_boundary(x)` is `true`. + /// Finds the closest `x` not below `index` where [`is_char_boundary(x)`] is `true`. /// /// If `index` is greater than the length of the string, this returns the length of the string. /// @@ -254,7 +258,7 @@ impl str { /// for more details. /// /// [`floor_char_boundary`]: str::floor_char_boundary - /// + /// [`is_char_boundary(x)`]: Self::is_char_boundary /// /// # Examples /// @@ -338,9 +342,10 @@ impl str { /// assert_eq!("🍔∈🌏", s); /// ``` #[stable(feature = "str_mut_extras", since = "1.20.0")] + #[rustc_const_stable(feature = "const_str_as_mut", since = "1.83.0")] #[must_use] #[inline(always)] - pub unsafe fn as_bytes_mut(&mut self) -> &mut [u8] { + pub const unsafe fn as_bytes_mut(&mut self) -> &mut [u8] { // SAFETY: the cast from `&str` to `&[u8]` is safe since `str` // has the same layout as `&[u8]` (only std can make this guarantee). // The pointer dereference is safe since it comes from a mutable reference which @@ -383,10 +388,11 @@ impl str { /// It is your responsibility to make sure that the string slice only gets /// modified in a way that it remains valid UTF-8. #[stable(feature = "str_as_mut_ptr", since = "1.36.0")] + #[rustc_const_stable(feature = "const_str_as_mut", since = "1.83.0")] #[rustc_never_returns_null_ptr] #[must_use] #[inline(always)] - pub fn as_mut_ptr(&mut self) -> *mut u8 { + pub const fn as_mut_ptr(&mut self) -> *mut u8 { self as *mut str as *mut u8 } @@ -634,7 +640,8 @@ impl str { #[inline] #[must_use] #[stable(feature = "str_split_at", since = "1.4.0")] - pub fn split_at(&self, mid: usize) -> (&str, &str) { + #[rustc_const_unstable(feature = "const_str_split_at", issue = "131518")] + pub const fn split_at(&self, mid: usize) -> (&str, &str) { match self.split_at_checked(mid) { None => slice_error_fail(self, 0, mid), Some(pair) => pair, @@ -674,7 +681,8 @@ impl str { #[inline] #[must_use] #[stable(feature = "str_split_at", since = "1.4.0")] - pub fn split_at_mut(&mut self, mid: usize) -> (&mut str, &mut str) { + #[rustc_const_unstable(feature = "const_str_split_at", issue = "131518")] + pub const fn split_at_mut(&mut self, mid: usize) -> (&mut str, &mut str) { // is_char_boundary checks that the index is in [0, .len()] if self.is_char_boundary(mid) { // SAFETY: just checked that `mid` is on a char boundary. @@ -713,11 +721,12 @@ impl str { #[inline] #[must_use] #[stable(feature = "split_at_checked", since = "1.80.0")] - pub fn split_at_checked(&self, mid: usize) -> Option<(&str, &str)> { + #[rustc_const_unstable(feature = "const_str_split_at", issue = "131518")] + pub const fn split_at_checked(&self, mid: usize) -> Option<(&str, &str)> { // is_char_boundary checks that the index is in [0, .len()] if self.is_char_boundary(mid) { // SAFETY: just checked that `mid` is on a char boundary. - Some(unsafe { (self.get_unchecked(0..mid), self.get_unchecked(mid..self.len())) }) + Some(unsafe { self.split_at_unchecked(mid) }) } else { None } @@ -753,7 +762,8 @@ impl str { #[inline] #[must_use] #[stable(feature = "split_at_checked", since = "1.80.0")] - pub fn split_at_mut_checked(&mut self, mid: usize) -> Option<(&mut str, &mut str)> { + #[rustc_const_unstable(feature = "const_str_split_at", issue = "131518")] + pub const fn split_at_mut_checked(&mut self, mid: usize) -> Option<(&mut str, &mut str)> { // is_char_boundary checks that the index is in [0, .len()] if self.is_char_boundary(mid) { // SAFETY: just checked that `mid` is on a char boundary. @@ -769,7 +779,25 @@ impl str { /// /// The caller must ensure that `mid` is a valid byte offset from the start /// of the string and falls on the boundary of a UTF-8 code point. - unsafe fn split_at_mut_unchecked(&mut self, mid: usize) -> (&mut str, &mut str) { + const unsafe fn split_at_unchecked(&self, mid: usize) -> (&str, &str) { + let len = self.len(); + let ptr = self.as_ptr(); + // SAFETY: caller guarantees `mid` is on a char boundary. + unsafe { + ( + from_utf8_unchecked(slice::from_raw_parts(ptr, mid)), + from_utf8_unchecked(slice::from_raw_parts(ptr.add(mid), len - mid)), + ) + } + } + + /// Divides one string slice into two at an index. + /// + /// # Safety + /// + /// The caller must ensure that `mid` is a valid byte offset from the start + /// of the string and falls on the boundary of a UTF-8 code point. + const unsafe fn split_at_mut_unchecked(&mut self, mid: usize) -> (&mut str, &mut str) { let len = self.len(); let ptr = self.as_mut_ptr(); // SAFETY: caller guarantees `mid` is on a char boundary. @@ -830,6 +858,7 @@ impl str { /// ``` #[stable(feature = "rust1", since = "1.0.0")] #[inline] + #[cfg_attr(not(test), rustc_diagnostic_item = "str_chars")] pub fn chars(&self) -> Chars<'_> { Chars { iter: self.as_bytes().iter() } } @@ -1154,6 +1183,7 @@ impl str { /// assert!(bananas.starts_with(&['a', 'b', 'c', 'd'])); /// ``` #[stable(feature = "rust1", since = "1.0.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "str_starts_with")] pub fn starts_with(&self, pat: P) -> bool { pat.is_prefix_of(self) } @@ -1178,6 +1208,7 @@ impl str { /// assert!(!bananas.ends_with("nana")); /// ``` #[stable(feature = "rust1", since = "1.0.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "str_ends_with")] pub fn ends_with(&self, pat: P) -> bool where for<'a> P::Searcher<'a>: ReverseSearcher<'a>, @@ -2163,7 +2194,7 @@ impl str { /// Returns a string slice with the prefix removed. /// /// If the string starts with the pattern `prefix`, returns the substring after the prefix, - /// wrapped in `Some`. Unlike `trim_start_matches`, this method removes the prefix exactly once. + /// wrapped in `Some`. Unlike [`trim_start_matches`], this method removes the prefix exactly once. /// /// If the string does not start with `prefix`, returns `None`. /// @@ -2172,6 +2203,7 @@ impl str { /// /// [`char`]: prim@char /// [pattern]: self::pattern + /// [`trim_start_matches`]: Self::trim_start_matches /// /// # Examples /// @@ -2190,7 +2222,7 @@ impl str { /// Returns a string slice with the suffix removed. /// /// If the string ends with the pattern `suffix`, returns the substring before the suffix, - /// wrapped in `Some`. Unlike `trim_end_matches`, this method removes the suffix exactly once. + /// wrapped in `Some`. Unlike [`trim_end_matches`], this method removes the suffix exactly once. /// /// If the string does not end with `suffix`, returns `None`. /// @@ -2199,6 +2231,7 @@ impl str { /// /// [`char`]: prim@char /// [pattern]: self::pattern + /// [`trim_end_matches`]: Self::trim_end_matches /// /// # Examples /// @@ -2467,8 +2500,9 @@ impl str { /// assert_eq!("GRüßE, JüRGEN ❤", s); /// ``` #[stable(feature = "ascii_methods_on_intrinsics", since = "1.23.0")] + #[rustc_const_stable(feature = "const_make_ascii", since = "CURRENT_RUSTC_VERSION")] #[inline] - pub fn make_ascii_uppercase(&mut self) { + pub const fn make_ascii_uppercase(&mut self) { // SAFETY: changing ASCII letters only does not invalidate UTF-8. let me = unsafe { self.as_bytes_mut() }; me.make_ascii_uppercase() @@ -2494,8 +2528,9 @@ impl str { /// assert_eq!("grÜße, jÜrgen ❤", s); /// ``` #[stable(feature = "ascii_methods_on_intrinsics", since = "1.23.0")] + #[rustc_const_stable(feature = "const_make_ascii", since = "CURRENT_RUSTC_VERSION")] #[inline] - pub fn make_ascii_lowercase(&mut self) { + pub const fn make_ascii_lowercase(&mut self) { // SAFETY: changing ASCII letters only does not invalidate UTF-8. let me = unsafe { self.as_bytes_mut() }; me.make_ascii_lowercase() @@ -2734,6 +2769,17 @@ impl str { pub fn substr_range(&self, substr: &str) -> Option> { self.as_bytes().subslice_range(substr.as_bytes()) } + + /// Returns the same string as a string slice `&str`. + /// + /// This method is redundant when used directly on `&str`, but + /// it helps dereferencing other string-like types to string slices, + /// for example references to `Box` or `Arc`. + #[inline] + #[unstable(feature = "str_as_str", issue = "130366")] + pub fn as_str(&self) -> &str { + self + } } #[stable(feature = "rust1", since = "1.0.0")] diff --git a/library/core/src/str/pattern.rs b/library/core/src/str/pattern.rs index eda8ca17746dd..4c119992f8e9e 100644 --- a/library/core/src/str/pattern.rs +++ b/library/core/src/str/pattern.rs @@ -64,9 +64,9 @@ use crate::kani::mem::same_allocation; /// [`Searcher`] type, which does the actual work of finding /// occurrences of the pattern in a string. /// -/// Depending on the type of the pattern, the behaviour of methods like +/// Depending on the type of the pattern, the behavior of methods like /// [`str::find`] and [`str::contains`] can change. The table below describes -/// some of those behaviours. +/// some of those behaviors. /// /// | Pattern type | Match condition | /// |--------------------------|-------------------------------------------| @@ -167,6 +167,21 @@ pub trait Pattern: Sized { None } } + + /// Returns the pattern as utf-8 bytes if possible. + fn as_utf8_pattern(&self) -> Option> { + None + } +} +/// Result of calling [`Pattern::as_utf8_pattern()`]. +/// Can be used for inspecting the contents of a [`Pattern`] in cases +/// where the underlying representation can be represented as UTF-8. +#[derive(Copy, Clone, Eq, PartialEq, Debug)] +pub enum Utf8Pattern<'a> { + /// Type returned by String and str types. + StringPattern(&'a [u8]), + /// Type returned by char types. + CharPattern(char), } // Searcher @@ -606,6 +621,11 @@ impl Pattern for char { { self.encode_utf8(&mut [0u8; 4]).strip_suffix_of(haystack) } + + #[inline] + fn as_utf8_pattern(&self) -> Option> { + Some(Utf8Pattern::CharPattern(*self)) + } } ///////////////////////////////////////////////////////////////////////////// @@ -1029,6 +1049,11 @@ impl<'b> Pattern for &'b str { None } } + + #[inline] + fn as_utf8_pattern(&self) -> Option> { + Some(Utf8Pattern::StringPattern(self.as_bytes())) + } } ///////////////////////////////////////////////////////////////////////////// @@ -1821,7 +1846,7 @@ fn simd_contains(needle: &str, haystack: &str) -> Option { } mask &= !(1 << trailing); } - return false; + false }; let test_chunk = |idx| -> u16 { @@ -1837,7 +1862,7 @@ fn simd_contains(needle: &str, haystack: &str) -> Option { let both = eq_first.bitand(eq_last); let mask = both.to_bitmask() as u16; - return mask; + mask }; let mut i = 0; diff --git a/library/core/src/str/traits.rs b/library/core/src/str/traits.rs index b69c476ae5e53..77c70b978fd15 100644 --- a/library/core/src/str/traits.rs +++ b/library/core/src/str/traits.rs @@ -92,7 +92,6 @@ const fn str_index_overflow_fail() -> ! { /// /// Equivalent to `&self[0 .. len]` or `&mut self[0 .. len]`. #[stable(feature = "str_checked_slicing", since = "1.20.0")] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] unsafe impl SliceIndex for ops::RangeFull { type Output = str; #[inline] @@ -157,7 +156,6 @@ unsafe impl SliceIndex for ops::RangeFull { /// // &s[3 .. 100]; /// ``` #[stable(feature = "str_checked_slicing", since = "1.20.0")] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] unsafe impl SliceIndex for ops::Range { type Output = str; #[inline] @@ -429,7 +427,6 @@ unsafe impl SliceIndex for (ops::Bound, ops::Bound) { /// Panics if `end` does not point to the starting byte offset of a /// character (as defined by `is_char_boundary`), or if `end > len`. #[stable(feature = "str_checked_slicing", since = "1.20.0")] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] unsafe impl SliceIndex for ops::RangeTo { type Output = str; #[inline] @@ -498,7 +495,6 @@ unsafe impl SliceIndex for ops::RangeTo { /// Panics if `begin` does not point to the starting byte offset of /// a character (as defined by `is_char_boundary`), or if `begin > len`. #[stable(feature = "str_checked_slicing", since = "1.20.0")] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] unsafe impl SliceIndex for ops::RangeFrom { type Output = str; #[inline] @@ -625,7 +621,6 @@ unsafe impl SliceIndex for range::RangeFrom { /// to the ending byte offset of a character (`end + 1` is either a starting /// byte offset or equal to `len`), if `begin > end`, or if `end >= len`. #[stable(feature = "inclusive_range", since = "1.26.0")] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] unsafe impl SliceIndex for ops::RangeInclusive { type Output = str; #[inline] @@ -714,7 +709,6 @@ unsafe impl SliceIndex for range::RangeInclusive { /// (`end + 1` is either a starting byte offset as defined by /// `is_char_boundary`, or equal to `len`), or if `end >= len`. #[stable(feature = "inclusive_range", since = "1.26.0")] -#[rustc_const_unstable(feature = "const_slice_index", issue = "none")] unsafe impl SliceIndex for ops::RangeToInclusive { type Output = str; #[inline] diff --git a/library/core/src/sync/atomic.rs b/library/core/src/sync/atomic.rs index b06a3bd4487d3..93b4ad5c1c941 100644 --- a/library/core/src/sync/atomic.rs +++ b/library/core/src/sync/atomic.rs @@ -24,25 +24,42 @@ //! //! ## Memory model for atomic accesses //! -//! Rust atomics currently follow the same rules as [C++20 atomics][cpp], specifically `atomic_ref`. -//! Basically, creating a *shared reference* to one of the Rust atomic types corresponds to creating -//! an `atomic_ref` in C++; the `atomic_ref` is destroyed when the lifetime of the shared reference -//! ends. A Rust atomic type that is exclusively owned or behind a mutable reference does *not* -//! correspond to an “atomic object” in C++, since the underlying primitive can be mutably accessed, -//! for example with `get_mut`, to perform non-atomic operations. +//! Rust atomics currently follow the same rules as [C++20 atomics][cpp], specifically the rules +//! from the [`intro.races`][cpp-intro.races] section, without the "consume" memory ordering. Since +//! C++ uses an object-based memory model whereas Rust is access-based, a bit of translation work +//! has to be done to apply the C++ rules to Rust: whenever C++ talks about "the value of an +//! object", we understand that to mean the resulting bytes obtained when doing a read. When the C++ +//! standard talks about "the value of an atomic object", this refers to the result of doing an +//! atomic load (via the operations provided in this module). A "modification of an atomic object" +//! refers to an atomic store. //! -//! [cpp]: https://en.cppreference.com/w/cpp/atomic +//! The end result is *almost* equivalent to saying that creating a *shared reference* to one of the +//! Rust atomic types corresponds to creating an `atomic_ref` in C++, with the `atomic_ref` being +//! destroyed when the lifetime of the shared reference ends. The main difference is that Rust +//! permits concurrent atomic and non-atomic reads to the same memory as those cause no issue in the +//! C++ memory model, they are just forbidden in C++ because memory is partitioned into "atomic +//! objects" and "non-atomic objects" (with `atomic_ref` temporarily converting a non-atomic object +//! into an atomic object). +//! +//! The most important aspect of this model is that *data races* are undefined behavior. A data race +//! is defined as conflicting non-synchronized accesses where at least one of the accesses is +//! non-atomic. Here, accesses are *conflicting* if they affect overlapping regions of memory and at +//! least one of them is a write. They are *non-synchronized* if neither of them *happens-before* +//! the other, according to the happens-before order of the memory model. //! -//! Each method takes an [`Ordering`] which represents the strength of -//! the memory barrier for that operation. These orderings are the -//! same as the [C++20 atomic orderings][1]. For more information see the [nomicon][2]. +//! The other possible cause of undefined behavior in the memory model are mixed-size accesses: Rust +//! inherits the C++ limitation that non-synchronized conflicting atomic accesses may not partially +//! overlap. In other words, every pair of non-synchronized atomic accesses must be either disjoint, +//! access the exact same memory (including using the same access size), or both be reads. //! -//! [1]: https://en.cppreference.com/w/cpp/atomic/memory_order -//! [2]: ../../../nomicon/atomics.html +//! Each atomic access takes an [`Ordering`] which defines how the operation interacts with the +//! happens-before order. These orderings behave the same as the corresponding [C++20 atomic +//! orderings][cpp_memory_order]. For more information, see the [nomicon]. //! -//! Since C++ does not support mixing atomic and non-atomic accesses, or non-synchronized -//! different-sized accesses to the same data, Rust does not support those operations either. -//! Note that both of those restrictions only apply if the accesses are non-synchronized. +//! [cpp]: https://en.cppreference.com/w/cpp/atomic +//! [cpp-intro.races]: https://timsong-cpp.github.io/cppwp/n4868/intro.multithread#intro.races +//! [cpp_memory_order]: https://en.cppreference.com/w/cpp/atomic/memory_order +//! [nomicon]: ../../../nomicon/atomics.html //! //! ```rust,no_run undefined_behavior //! use std::sync::atomic::{AtomicU16, AtomicU8, Ordering}; @@ -52,27 +69,29 @@ //! let atomic = AtomicU16::new(0); //! //! thread::scope(|s| { -//! // This is UB: mixing atomic and non-atomic accesses -//! s.spawn(|| atomic.store(1, Ordering::Relaxed)); -//! s.spawn(|| unsafe { atomic.as_ptr().write(2) }); +//! // This is UB: conflicting non-synchronized accesses, at least one of which is non-atomic. +//! s.spawn(|| atomic.store(1, Ordering::Relaxed)); // atomic store +//! s.spawn(|| unsafe { atomic.as_ptr().write(2) }); // non-atomic write //! }); //! //! thread::scope(|s| { -//! // This is UB: even reads are not allowed to be mixed -//! s.spawn(|| atomic.load(Ordering::Relaxed)); -//! s.spawn(|| unsafe { atomic.as_ptr().read() }); +//! // This is fine: the accesses do not conflict (as none of them performs any modification). +//! // In C++ this would be disallowed since creating an `atomic_ref` precludes +//! // further non-atomic accesses, but Rust does not have that limitation. +//! s.spawn(|| atomic.load(Ordering::Relaxed)); // atomic load +//! s.spawn(|| unsafe { atomic.as_ptr().read() }); // non-atomic read //! }); //! //! thread::scope(|s| { -//! // This is fine, `join` synchronizes the code in a way such that atomic -//! // and non-atomic accesses can't happen "at the same time" -//! let handle = s.spawn(|| atomic.store(1, Ordering::Relaxed)); -//! handle.join().unwrap(); -//! s.spawn(|| unsafe { atomic.as_ptr().write(2) }); +//! // This is fine: `join` synchronizes the code in a way such that the atomic +//! // store happens-before the non-atomic write. +//! let handle = s.spawn(|| atomic.store(1, Ordering::Relaxed)); // atomic store +//! handle.join().unwrap(); // synchronize +//! s.spawn(|| unsafe { atomic.as_ptr().write(2) }); // non-atomic write //! }); //! //! thread::scope(|s| { -//! // This is UB: using different-sized atomic accesses to the same data +//! // This is UB: non-synchronized conflicting differently-sized atomic accesses. //! s.spawn(|| atomic.store(1, Ordering::Relaxed)); //! s.spawn(|| unsafe { //! let differently_sized = transmute::<&AtomicU16, &AtomicU8>(&atomic); @@ -81,8 +100,8 @@ //! }); //! //! thread::scope(|s| { -//! // This is fine, `join` synchronizes the code in a way such that -//! // differently-sized accesses can't happen "at the same time" +//! // This is fine: `join` synchronizes the code in a way such that +//! // the 1-byte store happens-before the 2-byte store. //! let handle = s.spawn(|| atomic.store(1, Ordering::Relaxed)); //! handle.join().unwrap(); //! s.spawn(|| unsafe { @@ -137,7 +156,7 @@ //! //! # Atomic accesses to read-only memory //! -//! In general, *all* atomic accesses on read-only memory are Undefined Behavior. For instance, attempting +//! In general, *all* atomic accesses on read-only memory are undefined behavior. For instance, attempting //! to do a `compare_exchange` that will definitely fail (making it conceptually a read-only //! operation) can still cause a segmentation fault if the underlying memory page is mapped read-only. Since //! atomic `load`s might be implemented using compare-exchange operations, even a `load` can fault @@ -153,7 +172,7 @@ //! //! As an exception from the general rule stated above, "sufficiently small" atomic loads with //! `Ordering::Relaxed` are implemented in a way that works on read-only memory, and are hence not -//! Undefined Behavior. The exact size limit for what makes a load "sufficiently small" varies +//! undefined behavior. The exact size limit for what makes a load "sufficiently small" varies //! depending on the target: //! //! | `target_arch` | Size limit | @@ -577,7 +596,7 @@ impl AtomicBool { #[stable(feature = "atomic_access", since = "1.15.0")] #[rustc_const_stable(feature = "const_atomic_into_inner", since = "1.79.0")] pub const fn into_inner(self) -> bool { - self.v.primitive_into_inner() != 0 + self.v.into_inner() != 0 } /// Loads a value from the bool. @@ -1394,7 +1413,7 @@ impl AtomicPtr { #[stable(feature = "atomic_access", since = "1.15.0")] #[rustc_const_stable(feature = "const_atomic_into_inner", since = "1.79.0")] pub const fn into_inner(self) -> *mut T { - self.p.primitive_into_inner() + self.p.into_inner() } /// Loads a value from the pointer. @@ -1739,7 +1758,7 @@ impl AtomicPtr { /// # Examples /// /// ``` - /// #![feature(strict_provenance_atomic_ptr, strict_provenance)] + /// #![feature(strict_provenance_atomic_ptr)] /// use core::sync::atomic::{AtomicPtr, Ordering}; /// /// let atom = AtomicPtr::::new(core::ptr::null_mut()); @@ -1819,7 +1838,7 @@ impl AtomicPtr { /// # Examples /// /// ``` - /// #![feature(strict_provenance_atomic_ptr, strict_provenance)] + /// #![feature(strict_provenance_atomic_ptr)] /// use core::sync::atomic::{AtomicPtr, Ordering}; /// /// let atom = AtomicPtr::::new(core::ptr::null_mut()); @@ -1855,7 +1874,7 @@ impl AtomicPtr { /// # Examples /// /// ``` - /// #![feature(strict_provenance_atomic_ptr, strict_provenance)] + /// #![feature(strict_provenance_atomic_ptr)] /// use core::sync::atomic::{AtomicPtr, Ordering}; /// /// let atom = AtomicPtr::::new(core::ptr::without_provenance_mut(1)); @@ -1900,7 +1919,7 @@ impl AtomicPtr { /// # Examples /// /// ``` - /// #![feature(strict_provenance_atomic_ptr, strict_provenance)] + /// #![feature(strict_provenance_atomic_ptr)] /// use core::sync::atomic::{AtomicPtr, Ordering}; /// /// let pointer = &mut 3i64 as *mut i64; @@ -1951,7 +1970,7 @@ impl AtomicPtr { /// # Examples /// /// ``` - /// #![feature(strict_provenance_atomic_ptr, strict_provenance)] + /// #![feature(strict_provenance_atomic_ptr)] /// use core::sync::atomic::{AtomicPtr, Ordering}; /// /// let pointer = &mut 3i64 as *mut i64; @@ -2001,7 +2020,7 @@ impl AtomicPtr { /// # Examples /// /// ``` - /// #![feature(strict_provenance_atomic_ptr, strict_provenance)] + /// #![feature(strict_provenance_atomic_ptr)] /// use core::sync::atomic::{AtomicPtr, Ordering}; /// /// let pointer = &mut 3i64 as *mut i64; @@ -2103,7 +2122,8 @@ macro_rules! atomic_int { $stable_access:meta, $stable_from:meta, $stable_nand:meta, - $const_stable:meta, + $const_stable_new:meta, + $const_stable_into_inner:meta, $diagnostic_item:meta, $s_int_type:literal, $extra_feature:expr, @@ -2185,7 +2205,7 @@ macro_rules! atomic_int { /// ``` #[inline] #[$stable] - #[$const_stable] + #[$const_stable_new] #[must_use] pub const fn new(v: $int_type) -> Self { Self {v: UnsafeCell::new(v)} @@ -2387,9 +2407,9 @@ macro_rules! atomic_int { /// ``` #[inline] #[$stable_access] - #[rustc_const_stable(feature = "const_atomic_into_inner", since = "1.79.0")] + #[$const_stable_into_inner] pub const fn into_inner(self) -> $int_type { - self.v.primitive_into_inner() + self.v.into_inner() } /// Loads a value from the atomic integer. @@ -3035,6 +3055,7 @@ atomic_int! { stable(feature = "integer_atomics_stable", since = "1.34.0"), stable(feature = "integer_atomics_stable", since = "1.34.0"), rustc_const_stable(feature = "const_integer_atomics", since = "1.34.0"), + rustc_const_stable(feature = "const_atomic_into_inner", since = "1.79.0"), cfg_attr(not(test), rustc_diagnostic_item = "AtomicI8"), "i8", "", @@ -3053,6 +3074,7 @@ atomic_int! { stable(feature = "integer_atomics_stable", since = "1.34.0"), stable(feature = "integer_atomics_stable", since = "1.34.0"), rustc_const_stable(feature = "const_integer_atomics", since = "1.34.0"), + rustc_const_stable(feature = "const_atomic_into_inner", since = "1.79.0"), cfg_attr(not(test), rustc_diagnostic_item = "AtomicU8"), "u8", "", @@ -3071,6 +3093,7 @@ atomic_int! { stable(feature = "integer_atomics_stable", since = "1.34.0"), stable(feature = "integer_atomics_stable", since = "1.34.0"), rustc_const_stable(feature = "const_integer_atomics", since = "1.34.0"), + rustc_const_stable(feature = "const_atomic_into_inner", since = "1.79.0"), cfg_attr(not(test), rustc_diagnostic_item = "AtomicI16"), "i16", "", @@ -3089,6 +3112,7 @@ atomic_int! { stable(feature = "integer_atomics_stable", since = "1.34.0"), stable(feature = "integer_atomics_stable", since = "1.34.0"), rustc_const_stable(feature = "const_integer_atomics", since = "1.34.0"), + rustc_const_stable(feature = "const_atomic_into_inner", since = "1.79.0"), cfg_attr(not(test), rustc_diagnostic_item = "AtomicU16"), "u16", "", @@ -3107,6 +3131,7 @@ atomic_int! { stable(feature = "integer_atomics_stable", since = "1.34.0"), stable(feature = "integer_atomics_stable", since = "1.34.0"), rustc_const_stable(feature = "const_integer_atomics", since = "1.34.0"), + rustc_const_stable(feature = "const_atomic_into_inner", since = "1.79.0"), cfg_attr(not(test), rustc_diagnostic_item = "AtomicI32"), "i32", "", @@ -3125,6 +3150,7 @@ atomic_int! { stable(feature = "integer_atomics_stable", since = "1.34.0"), stable(feature = "integer_atomics_stable", since = "1.34.0"), rustc_const_stable(feature = "const_integer_atomics", since = "1.34.0"), + rustc_const_stable(feature = "const_atomic_into_inner", since = "1.79.0"), cfg_attr(not(test), rustc_diagnostic_item = "AtomicU32"), "u32", "", @@ -3143,6 +3169,7 @@ atomic_int! { stable(feature = "integer_atomics_stable", since = "1.34.0"), stable(feature = "integer_atomics_stable", since = "1.34.0"), rustc_const_stable(feature = "const_integer_atomics", since = "1.34.0"), + rustc_const_stable(feature = "const_atomic_into_inner", since = "1.79.0"), cfg_attr(not(test), rustc_diagnostic_item = "AtomicI64"), "i64", "", @@ -3161,6 +3188,7 @@ atomic_int! { stable(feature = "integer_atomics_stable", since = "1.34.0"), stable(feature = "integer_atomics_stable", since = "1.34.0"), rustc_const_stable(feature = "const_integer_atomics", since = "1.34.0"), + rustc_const_stable(feature = "const_atomic_into_inner", since = "1.79.0"), cfg_attr(not(test), rustc_diagnostic_item = "AtomicU64"), "u64", "", @@ -3178,7 +3206,8 @@ atomic_int! { unstable(feature = "integer_atomics", issue = "99069"), unstable(feature = "integer_atomics", issue = "99069"), unstable(feature = "integer_atomics", issue = "99069"), - rustc_const_stable(feature = "const_integer_atomics", since = "1.34.0"), + rustc_const_unstable(feature = "integer_atomics", issue = "99069"), + rustc_const_unstable(feature = "integer_atomics", issue = "99069"), cfg_attr(not(test), rustc_diagnostic_item = "AtomicI128"), "i128", "#![feature(integer_atomics)]\n\n", @@ -3196,7 +3225,8 @@ atomic_int! { unstable(feature = "integer_atomics", issue = "99069"), unstable(feature = "integer_atomics", issue = "99069"), unstable(feature = "integer_atomics", issue = "99069"), - rustc_const_stable(feature = "const_integer_atomics", since = "1.34.0"), + rustc_const_unstable(feature = "integer_atomics", issue = "99069"), + rustc_const_unstable(feature = "integer_atomics", issue = "99069"), cfg_attr(not(test), rustc_diagnostic_item = "AtomicU128"), "u128", "#![feature(integer_atomics)]\n\n", @@ -3219,6 +3249,7 @@ macro_rules! atomic_int_ptr_sized { stable(feature = "atomic_from", since = "1.23.0"), stable(feature = "atomic_nand", since = "1.27.0"), rustc_const_stable(feature = "const_ptr_sized_atomics", since = "1.24.0"), + rustc_const_stable(feature = "const_atomic_into_inner", since = "1.79.0"), cfg_attr(not(test), rustc_diagnostic_item = "AtomicIsize"), "isize", "", @@ -3237,6 +3268,7 @@ macro_rules! atomic_int_ptr_sized { stable(feature = "atomic_from", since = "1.23.0"), stable(feature = "atomic_nand", since = "1.27.0"), rustc_const_stable(feature = "const_ptr_sized_atomics", since = "1.24.0"), + rustc_const_stable(feature = "const_atomic_into_inner", since = "1.79.0"), cfg_attr(not(test), rustc_diagnostic_item = "AtomicUsize"), "usize", "", diff --git a/library/core/src/sync/exclusive.rs b/library/core/src/sync/exclusive.rs index fbf8dafad1869..af25f13973918 100644 --- a/library/core/src/sync/exclusive.rs +++ b/library/core/src/sync/exclusive.rs @@ -106,6 +106,7 @@ impl Exclusive { /// Unwrap the value contained in the `Exclusive` #[unstable(feature = "exclusive_wrapper", issue = "98407")] + #[rustc_const_unstable(feature = "exclusive_wrapper", issue = "98407")] #[must_use] #[inline] pub const fn into_inner(self) -> T { @@ -129,6 +130,7 @@ impl Exclusive { /// access to the underlying value, but _pinned_ `Exclusive`s only /// produce _pinned_ access to the underlying value. #[unstable(feature = "exclusive_wrapper", issue = "98407")] + #[rustc_const_unstable(feature = "exclusive_wrapper", issue = "98407")] #[must_use] #[inline] pub const fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut T> { @@ -152,6 +154,7 @@ impl Exclusive { /// a _pinned mutable_ reference to a `T`. This allows you to skip /// building an `Exclusive` with [`Exclusive::new`]. #[unstable(feature = "exclusive_wrapper", issue = "98407")] + #[rustc_const_unstable(feature = "exclusive_wrapper", issue = "98407")] #[must_use] #[inline] pub const fn from_pin_mut(r: Pin<&'_ mut T>) -> Pin<&'_ mut Exclusive> { diff --git a/library/core/src/task/wake.rs b/library/core/src/task/wake.rs index 5e559ad8d2ca7..fb7af8234ddb1 100644 --- a/library/core/src/task/wake.rs +++ b/library/core/src/task/wake.rs @@ -2,7 +2,7 @@ use crate::any::Any; use crate::marker::PhantomData; -use crate::mem::{transmute, ManuallyDrop}; +use crate::mem::{ManuallyDrop, transmute}; use crate::panic::AssertUnwindSafe; use crate::{fmt, ptr}; @@ -321,7 +321,7 @@ impl<'a> ContextBuilder<'a> { /// Creates a ContextBuilder from a Waker. #[inline] #[unstable(feature = "local_waker", issue = "118959")] - #[rustc_const_stable(feature = "const_waker", since = "1.82.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_waker", since = "1.82.0"))] pub const fn from_waker(waker: &'a Waker) -> Self { // SAFETY: LocalWaker is just Waker without thread safety let local_waker = unsafe { transmute(waker) }; @@ -379,7 +379,7 @@ impl<'a> ContextBuilder<'a> { /// Builds the `Context`. #[inline] #[unstable(feature = "local_waker", issue = "118959")] - #[rustc_const_stable(feature = "const_waker", since = "1.82.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_waker", since = "1.82.0"))] pub const fn build(self) -> Context<'a> { let ContextBuilder { waker, local_waker, ext, _marker, _marker2 } = self; Context { waker, local_waker, ext: AssertUnwindSafe(ext), _marker, _marker2 } @@ -414,6 +414,7 @@ impl<'a> ContextBuilder<'a> { /// [`Wake`]: ../../alloc/task/trait.Wake.html #[repr(transparent)] #[stable(feature = "futures_api", since = "1.36.0")] +#[cfg_attr(not(test), rustc_diagnostic_item = "Waker")] pub struct Waker { waker: RawWaker, } @@ -518,8 +519,8 @@ impl Waker { /// [`Wake`]: ../../alloc/task/trait.Wake.html #[inline] #[must_use] - #[stable(feature = "waker_getters", since = "CURRENT_RUSTC_VERSION")] - #[rustc_const_stable(feature = "waker_getters", since = "CURRENT_RUSTC_VERSION")] + #[stable(feature = "waker_getters", since = "1.83.0")] + #[rustc_const_stable(feature = "waker_getters", since = "1.83.0")] pub const unsafe fn new(data: *const (), vtable: &'static RawWakerVTable) -> Self { Waker { waker: RawWaker { data, vtable } } } @@ -583,7 +584,7 @@ impl Waker { /// Gets the `data` pointer used to create this `Waker`. #[inline] #[must_use] - #[stable(feature = "waker_getters", since = "CURRENT_RUSTC_VERSION")] + #[stable(feature = "waker_getters", since = "1.83.0")] pub fn data(&self) -> *const () { self.waker.data } @@ -591,7 +592,7 @@ impl Waker { /// Gets the `vtable` pointer used to create this `Waker`. #[inline] #[must_use] - #[stable(feature = "waker_getters", since = "CURRENT_RUSTC_VERSION")] + #[stable(feature = "waker_getters", since = "1.83.0")] pub fn vtable(&self) -> &'static RawWakerVTable { self.waker.vtable } diff --git a/library/core/src/time.rs b/library/core/src/time.rs index c19eeedb35426..5081e777af464 100644 --- a/library/core/src/time.rs +++ b/library/core/src/time.rs @@ -213,10 +213,9 @@ impl Duration { // SAFETY: nanos < NANOS_PER_SEC, therefore nanos is within the valid range Duration { secs, nanos: unsafe { Nanoseconds(nanos) } } } else { - let secs = match secs.checked_add((nanos / NANOS_PER_SEC) as u64) { - Some(secs) => secs, - None => panic!("overflow in Duration::new"), - }; + let secs = secs + .checked_add((nanos / NANOS_PER_SEC) as u64) + .expect("overflow in Duration::new"); let nanos = nanos % NANOS_PER_SEC; // SAFETY: nanos % NANOS_PER_SEC < NANOS_PER_SEC, therefore nanos is within the valid range Duration { secs, nanos: unsafe { Nanoseconds(nanos) } } @@ -625,7 +624,6 @@ impl Duration { /// ``` #[stable(feature = "duration_abs_diff", since = "1.81.0")] #[rustc_const_stable(feature = "duration_abs_diff", since = "1.81.0")] - #[rustc_allow_const_fn_unstable(const_option)] #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline] @@ -768,6 +766,7 @@ impl Duration { let total_nanos = self.nanos.0 as u64 * rhs as u64; let extra_secs = total_nanos / (NANOS_PER_SEC as u64); let nanos = (total_nanos % (NANOS_PER_SEC as u64)) as u32; + // FIXME(const-hack): use `and_then` once that is possible. if let Some(s) = self.secs.checked_mul(rhs as u64) { if let Some(secs) = s.checked_add(extra_secs) { debug_assert!(nanos < NANOS_PER_SEC); @@ -845,7 +844,7 @@ impl Duration { #[stable(feature = "duration_float", since = "1.38.0")] #[must_use] #[inline] - #[rustc_const_unstable(feature = "duration_consts_float", issue = "72440")] + #[rustc_const_stable(feature = "duration_consts_float", since = "1.83.0")] pub const fn as_secs_f64(&self) -> f64 { (self.secs as f64) + (self.nanos.0 as f64) / (NANOS_PER_SEC as f64) } @@ -864,7 +863,7 @@ impl Duration { #[stable(feature = "duration_float", since = "1.38.0")] #[must_use] #[inline] - #[rustc_const_unstable(feature = "duration_consts_float", issue = "72440")] + #[rustc_const_stable(feature = "duration_consts_float", since = "1.83.0")] pub const fn as_secs_f32(&self) -> f32 { (self.secs as f32) + (self.nanos.0 as f32) / (NANOS_PER_SEC as f32) } @@ -884,7 +883,7 @@ impl Duration { #[unstable(feature = "duration_millis_float", issue = "122451")] #[must_use] #[inline] - #[rustc_const_unstable(feature = "duration_consts_float", issue = "72440")] + #[rustc_const_unstable(feature = "duration_millis_float", issue = "122451")] pub const fn as_millis_f64(&self) -> f64 { (self.secs as f64) * (MILLIS_PER_SEC as f64) + (self.nanos.0 as f64) / (NANOS_PER_MILLI as f64) @@ -905,7 +904,7 @@ impl Duration { #[unstable(feature = "duration_millis_float", issue = "122451")] #[must_use] #[inline] - #[rustc_const_unstable(feature = "duration_consts_float", issue = "72440")] + #[rustc_const_unstable(feature = "duration_millis_float", issue = "122451")] pub const fn as_millis_f32(&self) -> f32 { (self.secs as f32) * (MILLIS_PER_SEC as f32) + (self.nanos.0 as f32) / (NANOS_PER_MILLI as f32) @@ -1085,7 +1084,7 @@ impl Duration { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline] - #[rustc_const_unstable(feature = "duration_consts_float", issue = "72440")] + #[rustc_const_stable(feature = "duration_consts_float", since = "1.83.0")] pub const fn div_duration_f64(self, rhs: Duration) -> f64 { let self_nanos = (self.secs as f64) * (NANOS_PER_SEC as f64) + (self.nanos.0 as f64); let rhs_nanos = (rhs.secs as f64) * (NANOS_PER_SEC as f64) + (rhs.nanos.0 as f64); @@ -1106,7 +1105,7 @@ impl Duration { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline] - #[rustc_const_unstable(feature = "duration_consts_float", issue = "72440")] + #[rustc_const_stable(feature = "duration_consts_float", since = "1.83.0")] pub const fn div_duration_f32(self, rhs: Duration) -> f32 { let self_nanos = (self.secs as f32) * (NANOS_PER_SEC as f32) + (self.nanos.0 as f32); let rhs_nanos = (rhs.secs as f32) * (NANOS_PER_SEC as f32) + (rhs.nanos.0 as f32); diff --git a/library/core/src/ub_checks.rs b/library/core/src/ub_checks.rs index f5713fafb7c72..64b7c11b38cd5 100644 --- a/library/core/src/ub_checks.rs +++ b/library/core/src/ub_checks.rs @@ -47,7 +47,7 @@ use crate::intrinsics::{self, const_eval_select}; /// order to call it. Since the precompiled standard library is built with full debuginfo and these /// variables cannot be optimized out in MIR, an innocent-looking `let` can produce enough /// debuginfo to have a measurable compile-time impact on debug builds. -#[allow_internal_unstable(const_ub_checks)] // permit this to be called in stably-const fn +#[cfg_attr(bootstrap, allow_internal_unstable(const_ub_checks))] // permit this to be called in stably-const fn #[macro_export] #[unstable(feature = "ub_checks", issue = "none")] macro_rules! assert_unsafe_precondition { @@ -64,7 +64,8 @@ macro_rules! assert_unsafe_precondition { #[rustc_no_mir_inline] #[inline] #[rustc_nounwind] - #[rustc_const_unstable(feature = "const_ub_checks", issue = "none")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "const_ub_checks", issue = "none"))] + #[rustc_allow_const_fn_unstable(const_ptr_is_null, const_ub_checks)] // only for UB checks const fn precondition_check($($name:$ty),*) { if !$e { ::core::panicking::panic_nounwind( @@ -90,8 +91,9 @@ pub use intrinsics::ub_checks as check_library_ub; /// /// The intention is to not do that when running in the interpreter, as that one has its own /// language UB checks which generally produce better errors. -#[rustc_const_unstable(feature = "const_ub_checks", issue = "none")] +#[cfg_attr(bootstrap, rustc_const_unstable(feature = "const_ub_checks", issue = "none"))] #[inline] +#[rustc_allow_const_fn_unstable(const_eval_select)] pub(crate) const fn check_language_ub() -> bool { #[inline] fn runtime() -> bool { @@ -109,15 +111,16 @@ pub(crate) const fn check_language_ub() -> bool { intrinsics::ub_checks() && const_eval_select((), comptime, runtime) } -/// Checks whether `ptr` is properly aligned with respect to -/// `align_of::()`. +/// Checks whether `ptr` is properly aligned with respect to the given alignment, and +/// if `is_zst == false`, that `ptr` is not null. /// /// In `const` this is approximate and can fail spuriously. It is primarily intended /// for `assert_unsafe_precondition!` with `check_language_ub`, in which case the /// check is anyway not executed in `const`. #[inline] -pub(crate) const fn is_aligned_and_not_null(ptr: *const (), align: usize) -> bool { - !ptr.is_null() && ptr.is_aligned_to(align) +#[rustc_const_unstable(feature = "const_ub_checks", issue = "none")] +pub(crate) const fn is_aligned_and_not_null(ptr: *const (), align: usize, is_zst: bool) -> bool { + ptr.is_aligned_to(align) && (is_zst || !ptr.is_null()) } #[inline] @@ -132,6 +135,7 @@ pub(crate) const fn is_valid_allocation_size(size: usize, len: usize) -> bool { /// Note that in const-eval this function just returns `true` and therefore must /// only be used with `assert_unsafe_precondition!`, similar to `is_aligned_and_not_null`. #[inline] +#[rustc_const_unstable(feature = "const_ub_checks", issue = "none")] pub(crate) const fn is_nonoverlapping( src: *const (), dst: *const (), @@ -209,3 +213,58 @@ mod predicates { mod predicates { pub use crate::kani::mem::{can_dereference, can_write, can_read_unaligned, can_write_unaligned}; } + +/// This trait should be used to specify and check type safety invariants for a +/// type. For type invariants, we refer to the definitions in the Rust's Unsafe +/// Code Guidelines Reference: +/// +/// +/// In summary, the reference distinguishes two kinds of type invariants: +/// - *Validity invariant*: An invariant that all data must uphold any time +/// it's accessed or copied in a typed manner. This invariant is exploited by +/// the compiler to perform optimizations. +/// - *Safety invariant*: An invariant that safe code may assume all data to +/// uphold. This invariant can be temporarily violated by unsafe code, but +/// must always be upheld when interfacing with unknown safe code. +/// +/// Therefore, validity invariants must be upheld at all times, while safety +/// invariants only need to be upheld at the boundaries to safe code. +pub trait Invariant { + /// Specify the type's safety invariants + fn is_safe(&self) -> bool; +} + +/// Any value is considered safe for the type +macro_rules! trivial_invariant { + ( $type: ty ) => { + impl Invariant for $type { + #[inline(always)] + fn is_safe(&self) -> bool { + true + } + } + }; +} + +trivial_invariant!(u8); +trivial_invariant!(u16); +trivial_invariant!(u32); +trivial_invariant!(u64); +trivial_invariant!(u128); +trivial_invariant!(usize); + +trivial_invariant!(i8); +trivial_invariant!(i16); +trivial_invariant!(i32); +trivial_invariant!(i64); +trivial_invariant!(i128); +trivial_invariant!(isize); + +trivial_invariant!(()); +trivial_invariant!(bool); +trivial_invariant!(char); + +trivial_invariant!(f16); +trivial_invariant!(f32); +trivial_invariant!(f64); +trivial_invariant!(f128); diff --git a/library/core/src/unicode/mod.rs b/library/core/src/unicode/mod.rs index 6066aa9921607..5ee17570bbc42 100644 --- a/library/core/src/unicode/mod.rs +++ b/library/core/src/unicode/mod.rs @@ -31,3 +31,21 @@ mod unicode_data; /// [Unicode 11.0 or later, Section 3.1 Versions of the Unicode Standard](https://www.unicode.org/versions/Unicode11.0.0/ch03.pdf#page=4). #[stable(feature = "unicode_version", since = "1.45.0")] pub const UNICODE_VERSION: (u8, u8, u8) = unicode_data::UNICODE_VERSION; + +#[cfg(kani)] +mod verify { + use super::conversions::{to_upper, to_lower}; + use crate::kani; + + /// Checks that `to_upper` does not trigger UB or panics for all valid characters. + #[kani::proof] + fn check_to_upper_safety() { + let _ = to_upper(kani::any()); + } + + /// Checks that `to_lower` does not trigger UB or panics for all valid characters. + #[kani::proof] + fn check_to_lower_safety() { + let _ = to_lower(kani::any()); + } +} diff --git a/library/core/src/unicode/printable.rs b/library/core/src/unicode/printable.rs index 33c3ef8083de5..d8fb50e4ed296 100644 --- a/library/core/src/unicode/printable.rs +++ b/library/core/src/unicode/printable.rs @@ -118,7 +118,7 @@ const SINGLETONS0U: &[(u8, u8)] = &[ (0x30, 4), (0x31, 2), (0x32, 1), - (0xa7, 2), + (0xa7, 4), (0xa9, 2), (0xaa, 4), (0xab, 8), @@ -155,17 +155,18 @@ const SINGLETONS0L: &[u8] = &[ 0x11, 0x16, 0x17, 0x5b, 0x5c, 0xf6, 0xf7, 0xfe, 0xff, 0x80, 0x6d, 0x71, 0xde, 0xdf, 0x0e, 0x1f, 0x6e, 0x6f, 0x1c, 0x1d, 0x5f, 0x7d, 0x7e, 0xae, - 0xaf, 0x7f, 0xbb, 0xbc, 0x16, 0x17, 0x1e, 0x1f, + 0xaf, 0x4d, 0xbb, 0xbc, 0x16, 0x17, 0x1e, 0x1f, 0x46, 0x47, 0x4e, 0x4f, 0x58, 0x5a, 0x5c, 0x5e, 0x7e, 0x7f, 0xb5, 0xc5, 0xd4, 0xd5, 0xdc, 0xf0, 0xf1, 0xf5, 0x72, 0x73, 0x8f, 0x74, 0x75, 0x96, 0x26, 0x2e, 0x2f, 0xa7, 0xaf, 0xb7, 0xbf, 0xc7, 0xcf, 0xd7, 0xdf, 0x9a, 0x00, 0x40, 0x97, 0x98, - 0x30, 0x8f, 0x1f, 0xd2, 0xd4, 0xce, 0xff, 0x4e, - 0x4f, 0x5a, 0x5b, 0x07, 0x08, 0x0f, 0x10, 0x27, - 0x2f, 0xee, 0xef, 0x6e, 0x6f, 0x37, 0x3d, 0x3f, - 0x42, 0x45, 0x90, 0x91, 0x53, 0x67, 0x75, 0xc8, - 0xc9, 0xd0, 0xd1, 0xd8, 0xd9, 0xe7, 0xfe, 0xff, + 0x30, 0x8f, 0x1f, 0xce, 0xcf, 0xd2, 0xd4, 0xce, + 0xff, 0x4e, 0x4f, 0x5a, 0x5b, 0x07, 0x08, 0x0f, + 0x10, 0x27, 0x2f, 0xee, 0xef, 0x6e, 0x6f, 0x37, + 0x3d, 0x3f, 0x42, 0x45, 0x90, 0x91, 0x53, 0x67, + 0x75, 0xc8, 0xc9, 0xd0, 0xd1, 0xd8, 0xd9, 0xe7, + 0xfe, 0xff, ]; #[rustfmt::skip] const SINGLETONS1U: &[(u8, u8)] = &[ @@ -183,7 +184,7 @@ const SINGLETONS1U: &[(u8, u8)] = &[ (0x10, 1), (0x11, 2), (0x12, 5), - (0x13, 17), + (0x13, 28), (0x14, 1), (0x15, 2), (0x17, 2), @@ -211,7 +212,7 @@ const SINGLETONS1U: &[(u8, u8)] = &[ (0xee, 32), (0xf0, 4), (0xf8, 2), - (0xfa, 3), + (0xfa, 4), (0xfb, 1), ]; #[rustfmt::skip] @@ -224,23 +225,24 @@ const SINGLETONS1L: &[u8] = &[ 0xbd, 0x35, 0xe0, 0x12, 0x87, 0x89, 0x8e, 0x9e, 0x04, 0x0d, 0x0e, 0x11, 0x12, 0x29, 0x31, 0x34, 0x3a, 0x45, 0x46, 0x49, 0x4a, 0x4e, 0x4f, 0x64, - 0x65, 0x5c, 0xb6, 0xb7, 0x1b, 0x1c, 0x07, 0x08, - 0x0a, 0x0b, 0x14, 0x17, 0x36, 0x39, 0x3a, 0xa8, - 0xa9, 0xd8, 0xd9, 0x09, 0x37, 0x90, 0x91, 0xa8, - 0x07, 0x0a, 0x3b, 0x3e, 0x66, 0x69, 0x8f, 0x92, - 0x11, 0x6f, 0x5f, 0xbf, 0xee, 0xef, 0x5a, 0x62, - 0xf4, 0xfc, 0xff, 0x53, 0x54, 0x9a, 0x9b, 0x2e, - 0x2f, 0x27, 0x28, 0x55, 0x9d, 0xa0, 0xa1, 0xa3, - 0xa4, 0xa7, 0xa8, 0xad, 0xba, 0xbc, 0xc4, 0x06, - 0x0b, 0x0c, 0x15, 0x1d, 0x3a, 0x3f, 0x45, 0x51, - 0xa6, 0xa7, 0xcc, 0xcd, 0xa0, 0x07, 0x19, 0x1a, - 0x22, 0x25, 0x3e, 0x3f, 0xe7, 0xec, 0xef, 0xff, - 0xc5, 0xc6, 0x04, 0x20, 0x23, 0x25, 0x26, 0x28, - 0x33, 0x38, 0x3a, 0x48, 0x4a, 0x4c, 0x50, 0x53, - 0x55, 0x56, 0x58, 0x5a, 0x5c, 0x5e, 0x60, 0x63, - 0x65, 0x66, 0x6b, 0x73, 0x78, 0x7d, 0x7f, 0x8a, - 0xa4, 0xaa, 0xaf, 0xb0, 0xc0, 0xd0, 0xae, 0xaf, - 0x6e, 0x6f, 0xbe, 0x93, + 0x65, 0x8a, 0x8c, 0x8d, 0x8f, 0xb6, 0xc1, 0xc3, + 0xc4, 0xc6, 0xcb, 0xd6, 0x5c, 0xb6, 0xb7, 0x1b, + 0x1c, 0x07, 0x08, 0x0a, 0x0b, 0x14, 0x17, 0x36, + 0x39, 0x3a, 0xa8, 0xa9, 0xd8, 0xd9, 0x09, 0x37, + 0x90, 0x91, 0xa8, 0x07, 0x0a, 0x3b, 0x3e, 0x66, + 0x69, 0x8f, 0x92, 0x11, 0x6f, 0x5f, 0xbf, 0xee, + 0xef, 0x5a, 0x62, 0xf4, 0xfc, 0xff, 0x53, 0x54, + 0x9a, 0x9b, 0x2e, 0x2f, 0x27, 0x28, 0x55, 0x9d, + 0xa0, 0xa1, 0xa3, 0xa4, 0xa7, 0xa8, 0xad, 0xba, + 0xbc, 0xc4, 0x06, 0x0b, 0x0c, 0x15, 0x1d, 0x3a, + 0x3f, 0x45, 0x51, 0xa6, 0xa7, 0xcc, 0xcd, 0xa0, + 0x07, 0x19, 0x1a, 0x22, 0x25, 0x3e, 0x3f, 0xe7, + 0xec, 0xef, 0xff, 0xc5, 0xc6, 0x04, 0x20, 0x23, + 0x25, 0x26, 0x28, 0x33, 0x38, 0x3a, 0x48, 0x4a, + 0x4c, 0x50, 0x53, 0x55, 0x56, 0x58, 0x5a, 0x5c, + 0x5e, 0x60, 0x63, 0x65, 0x66, 0x6b, 0x73, 0x78, + 0x7d, 0x7f, 0x8a, 0xa4, 0xaa, 0xaf, 0xb0, 0xc0, + 0xd0, 0xae, 0xaf, 0x6e, 0x6f, 0xdd, 0xde, 0x93, ]; #[rustfmt::skip] const NORMAL0: &[u8] = &[ @@ -252,8 +254,8 @@ const NORMAL0: &[u8] = &[ 0x06, 0x11, 0x81, 0xac, 0x0e, 0x80, 0xab, 0x05, - 0x1f, 0x09, - 0x81, 0x1b, 0x03, + 0x1f, 0x08, + 0x81, 0x1c, 0x03, 0x19, 0x08, 0x01, 0x04, 0x2f, 0x04, @@ -318,11 +320,10 @@ const NORMAL0: &[u8] = &[ 0x80, 0xac, 0x06, 0x0a, 0x06, 0x2f, 0x31, - 0x4d, 0x03, - 0x80, 0xa4, 0x08, + 0x80, 0xf4, 0x08, 0x3c, 0x03, 0x0f, 0x03, - 0x3c, 0x07, + 0x3e, 0x05, 0x38, 0x08, 0x2b, 0x05, 0x82, 0xff, 0x11, @@ -332,7 +333,7 @@ const NORMAL0: &[u8] = &[ 0x21, 0x0f, 0x21, 0x0f, 0x80, 0x8c, 0x04, - 0x82, 0x97, 0x19, + 0x82, 0x9a, 0x16, 0x0b, 0x15, 0x88, 0x94, 0x05, 0x2f, 0x05, @@ -343,13 +344,12 @@ const NORMAL0: &[u8] = &[ 0x74, 0x0c, 0x80, 0xd6, 0x1a, 0x81, 0x10, 0x05, - 0x80, 0xdf, 0x0b, + 0x80, 0xe1, 0x09, 0xf2, 0x9e, 0x03, 0x37, 0x09, 0x81, 0x5c, 0x14, 0x80, 0xb8, 0x08, - 0x80, 0xcb, 0x05, - 0x0a, 0x18, + 0x80, 0xdd, 0x15, 0x3b, 0x03, 0x0a, 0x06, 0x38, 0x08, @@ -402,7 +402,8 @@ const NORMAL1: &[u8] = &[ 0x24, 0x04, 0x28, 0x08, 0x34, 0x0b, - 0x4e, 0x43, + 0x4e, 0x03, + 0x34, 0x0c, 0x81, 0x37, 0x09, 0x16, 0x0a, 0x08, 0x18, @@ -431,9 +432,13 @@ const NORMAL1: &[u8] = &[ 0x33, 0x0d, 0x33, 0x07, 0x2e, 0x08, - 0x0a, 0x81, 0x26, - 0x52, 0x4b, - 0x2b, 0x08, + 0x0a, 0x06, + 0x26, 0x03, + 0x1d, 0x08, + 0x02, 0x80, 0xd0, + 0x52, 0x10, + 0x03, 0x37, + 0x2c, 0x08, 0x2a, 0x16, 0x1a, 0x26, 0x1c, 0x14, @@ -453,7 +458,9 @@ const NORMAL1: &[u8] = &[ 0x51, 0x06, 0x01, 0x05, 0x10, 0x03, - 0x05, 0x80, 0x8b, + 0x05, 0x0b, + 0x59, 0x08, + 0x02, 0x1d, 0x62, 0x1e, 0x48, 0x08, 0x0a, 0x80, 0xa6, @@ -462,7 +469,8 @@ const NORMAL1: &[u8] = &[ 0x0a, 0x06, 0x0d, 0x13, 0x3a, 0x06, - 0x0a, 0x36, + 0x0a, 0x06, + 0x14, 0x1c, 0x2c, 0x04, 0x17, 0x80, 0xb9, 0x3c, 0x64, @@ -473,7 +481,9 @@ const NORMAL1: &[u8] = &[ 0x48, 0x08, 0x53, 0x0d, 0x49, 0x07, - 0x0a, 0x80, 0xf6, + 0x0a, 0x80, 0xb6, + 0x22, 0x0e, + 0x0a, 0x06, 0x46, 0x0a, 0x1d, 0x03, 0x47, 0x49, @@ -484,7 +494,7 @@ const NORMAL1: &[u8] = &[ 0x0a, 0x81, 0x36, 0x19, 0x07, 0x3b, 0x03, - 0x1c, 0x56, + 0x1d, 0x55, 0x01, 0x0f, 0x32, 0x0d, 0x83, 0x9b, 0x66, @@ -492,15 +502,18 @@ const NORMAL1: &[u8] = &[ 0x80, 0xc4, 0x8a, 0x4c, 0x63, 0x0d, 0x84, 0x30, 0x10, - 0x16, 0x8f, 0xaa, - 0x82, 0x47, 0xa1, 0xb9, + 0x16, 0x0a, + 0x8f, 0x9b, 0x05, + 0x82, 0x47, 0x9a, 0xb9, + 0x3a, 0x86, 0xc6, 0x82, 0x39, 0x07, 0x2a, 0x04, 0x5c, 0x06, 0x26, 0x0a, 0x46, 0x0a, 0x28, 0x05, - 0x13, 0x82, 0xb0, + 0x13, 0x81, 0xb0, + 0x3a, 0x80, 0xc6, 0x5b, 0x65, 0x4b, 0x04, 0x39, 0x07, @@ -508,8 +521,8 @@ const NORMAL1: &[u8] = &[ 0x05, 0x0b, 0x02, 0x0e, 0x97, 0xf8, 0x08, - 0x84, 0xd6, 0x2a, - 0x09, 0xa2, 0xe7, + 0x84, 0xd6, 0x29, + 0x0a, 0xa2, 0xe7, 0x81, 0x33, 0x0f, 0x01, 0x1d, 0x06, 0x0e, @@ -518,7 +531,9 @@ const NORMAL1: &[u8] = &[ 0x6b, 0x05, 0x0d, 0x03, 0x09, 0x07, - 0x10, 0x92, 0x60, + 0x10, 0x8f, 0x60, + 0x80, 0xfa, 0x06, + 0x81, 0xb4, 0x4c, 0x47, 0x09, 0x74, 0x3c, 0x80, 0xf6, 0x0a, @@ -543,7 +558,9 @@ const NORMAL1: &[u8] = &[ 0x1f, 0x11, 0x3a, 0x05, 0x01, 0x81, 0xd0, - 0x2a, 0x82, 0xe6, + 0x2a, 0x80, 0xd6, + 0x2b, 0x04, + 0x01, 0x81, 0xe0, 0x80, 0xf7, 0x29, 0x4c, 0x04, 0x0a, 0x04, @@ -575,14 +592,13 @@ const NORMAL1: &[u8] = &[ 0x38, 0x08, 0x0a, 0x06, 0x28, 0x08, - 0x22, 0x4e, + 0x2c, 0x04, + 0x02, 0x3e, 0x81, 0x54, 0x0c, 0x1d, 0x03, + 0x0a, 0x05, + 0x38, 0x07, + 0x1c, 0x06, 0x09, 0x07, - 0x36, 0x08, - 0x0e, 0x04, - 0x09, 0x07, - 0x09, 0x07, - 0x80, 0xcb, 0x25, - 0x0a, 0x84, 0x06, + 0x80, 0xfa, 0x84, 0x06, ]; diff --git a/library/core/src/unicode/unicode_data.rs b/library/core/src/unicode/unicode_data.rs index 1b3d6729663b5..cba53bf5054e6 100644 --- a/library/core/src/unicode/unicode_data.rs +++ b/library/core/src/unicode/unicode_data.rs @@ -18,16 +18,14 @@ const fn bitset_search< let bucket_idx = (needle / 64) as usize; let chunk_map_idx = bucket_idx / CHUNK_SIZE; let chunk_piece = bucket_idx % CHUNK_SIZE; - // FIXME: const-hack: Revert to `slice::get` after `const_slice_index` - // feature stabilizes. + // FIXME(const-hack): Revert to `slice::get` when slice indexing becomes possible in const. let chunk_idx = if chunk_map_idx < chunk_idx_map.len() { chunk_idx_map[chunk_map_idx] } else { return false; }; let idx = bitset_chunk_idx[chunk_idx as usize][chunk_piece] as usize; - // FIXME: const-hack: Revert to `slice::get` after `const_slice_index` - // feature stabilizes. + // FIXME(const-hack): Revert to `slice::get` when slice indexing becomes possible in const. let word = if idx < bitset_canonical.len() { bitset_canonical[idx] } else { @@ -99,75 +97,77 @@ fn skip_search( offset_idx % 2 == 1 } -pub const UNICODE_VERSION: (u8, u8, u8) = (15, 1, 0); +pub const UNICODE_VERSION: (u8, u8, u8) = (16, 0, 0); #[rustfmt::skip] pub mod alphabetic { - static SHORT_OFFSET_RUNS: [u32; 54] = [ - 706, 33559113, 872420973, 952114966, 1161831606, 1310731264, 1314926597, 1394619392, - 1444957632, 1447077005, 1451271693, 1459672996, 1648425216, 1658911342, 1661009214, - 1707147904, 1793132343, 1887506048, 2040601600, 2392923872, 2481005466, 2504077200, - 2514564144, 2520859648, 2527151687, 2529257472, 2531355193, 2533453376, 2564917240, - 2596375766, 2600579056, 2606870819, 2621551356, 2642525184, 2644628480, 2665600678, - 2743197440, 2791432848, 2841765072, 2850154464, 2854350336, 2887905584, 3026321408, - 3038947040, 3041048378, 3045248674, 3053644769, 3057839710, 3062036480, 3064134174, - 3066232832, 3068334923, 3070436272, 3075744688, + static SHORT_OFFSET_RUNS: [u32; 53] = [ + 706, 33559113, 876615277, 956309270, 1166025910, 1314925568, 1319120901, 1398813696, + 1449151936, 1451271309, 1455465997, 1463867300, 1652619520, 1663105646, 1665203518, + 1711342208, 1797326647, 1895898848, 2560697242, 2583768976, 2594255920, 2600551419, + 2608940615, 2613141760, 2615240704, 2619435577, 2621533504, 2652997624, 2688650454, + 2692853744, 2699145507, 2713826044, 2734799872, 2736903168, 2757875366, 2835472128, + 2883707536, 2934039760, 2942429152, 2955013632, 2988568880, 3126984704, 3139610336, + 3141711674, 3145911970, 3154308065, 3158503006, 3162699776, 3164797470, 3166896128, + 3168998219, 3171099568, 3176407984, ]; - static OFFSETS: [u8; 1467] = [ - 65, 26, 6, 26, 47, 1, 10, 1, 4, 1, 5, 23, 1, 31, 1, 0, 4, 12, 14, 5, 7, 1, 1, 1, 86, 1, 42, - 5, 1, 2, 2, 4, 1, 1, 6, 1, 1, 3, 1, 1, 1, 20, 1, 83, 1, 139, 8, 166, 1, 38, 2, 1, 6, 41, 39, - 14, 1, 1, 1, 2, 1, 2, 1, 1, 8, 27, 4, 4, 29, 11, 5, 56, 1, 7, 14, 102, 1, 8, 4, 8, 4, 3, 10, - 3, 2, 1, 16, 48, 13, 101, 24, 33, 9, 2, 4, 1, 5, 24, 2, 19, 19, 25, 7, 11, 5, 24, 1, 6, 17, - 42, 10, 12, 3, 7, 6, 76, 1, 16, 1, 3, 4, 15, 13, 19, 1, 8, 2, 2, 2, 22, 1, 7, 1, 1, 3, 4, 3, - 8, 2, 2, 2, 2, 1, 1, 8, 1, 4, 2, 1, 5, 12, 2, 10, 1, 4, 3, 1, 6, 4, 2, 2, 22, 1, 7, 1, 2, 1, - 2, 1, 2, 4, 5, 4, 2, 2, 2, 4, 1, 7, 4, 1, 1, 17, 6, 11, 3, 1, 9, 1, 3, 1, 22, 1, 7, 1, 2, 1, - 5, 3, 9, 1, 3, 1, 2, 3, 1, 15, 4, 21, 4, 4, 3, 1, 8, 2, 2, 2, 22, 1, 7, 1, 2, 1, 5, 3, 8, 2, - 2, 2, 2, 9, 2, 4, 2, 1, 5, 13, 1, 16, 2, 1, 6, 3, 3, 1, 4, 3, 2, 1, 1, 1, 2, 3, 2, 3, 3, 3, - 12, 4, 5, 3, 3, 1, 3, 3, 1, 6, 1, 40, 13, 1, 3, 1, 23, 1, 16, 3, 8, 1, 3, 1, 3, 8, 2, 1, 3, - 2, 1, 2, 4, 28, 4, 1, 8, 1, 3, 1, 23, 1, 10, 1, 5, 3, 8, 1, 3, 1, 3, 8, 2, 6, 2, 1, 4, 13, - 3, 12, 13, 1, 3, 1, 41, 2, 8, 1, 3, 1, 3, 1, 1, 5, 4, 7, 5, 22, 6, 1, 3, 1, 18, 3, 24, 1, 9, - 1, 1, 2, 7, 8, 6, 1, 1, 1, 8, 18, 2, 13, 58, 5, 7, 6, 1, 51, 2, 1, 1, 1, 5, 1, 24, 1, 1, 1, - 19, 1, 3, 2, 5, 1, 1, 6, 1, 14, 4, 32, 1, 63, 8, 1, 36, 4, 19, 4, 16, 1, 36, 67, 55, 1, 1, - 2, 5, 16, 64, 10, 4, 2, 38, 1, 1, 5, 1, 2, 43, 1, 0, 1, 4, 2, 7, 1, 1, 1, 4, 2, 41, 1, 4, 2, - 33, 1, 4, 2, 7, 1, 1, 1, 4, 2, 15, 1, 57, 1, 4, 2, 67, 37, 16, 16, 86, 2, 6, 3, 0, 2, 17, 1, - 26, 5, 75, 3, 11, 7, 20, 11, 21, 12, 20, 12, 13, 1, 3, 1, 2, 12, 52, 2, 19, 14, 1, 4, 1, 67, - 89, 7, 43, 5, 70, 10, 31, 1, 12, 4, 9, 23, 30, 2, 5, 11, 44, 4, 26, 54, 28, 4, 63, 2, 20, - 50, 1, 23, 2, 11, 3, 49, 52, 1, 15, 1, 8, 51, 42, 2, 4, 10, 44, 1, 11, 14, 55, 22, 3, 10, - 36, 2, 9, 7, 43, 2, 3, 41, 4, 1, 6, 1, 2, 3, 1, 5, 192, 39, 14, 11, 0, 2, 6, 2, 38, 2, 6, 2, - 8, 1, 1, 1, 1, 1, 1, 1, 31, 2, 53, 1, 7, 1, 1, 3, 3, 1, 7, 3, 4, 2, 6, 4, 13, 5, 3, 1, 7, - 116, 1, 13, 1, 16, 13, 101, 1, 4, 1, 2, 10, 1, 1, 3, 5, 6, 1, 1, 1, 1, 1, 1, 4, 1, 11, 2, 4, - 5, 5, 4, 1, 17, 41, 0, 52, 0, 229, 6, 4, 3, 2, 12, 38, 1, 1, 5, 1, 2, 56, 7, 1, 16, 23, 9, - 7, 1, 7, 1, 7, 1, 7, 1, 7, 1, 7, 1, 7, 1, 7, 1, 32, 47, 1, 0, 3, 25, 9, 7, 5, 2, 5, 4, 86, - 6, 3, 1, 90, 1, 4, 5, 43, 1, 94, 17, 32, 48, 16, 0, 0, 64, 0, 67, 46, 2, 0, 3, 16, 10, 2, - 20, 47, 5, 8, 3, 113, 39, 9, 2, 103, 2, 64, 5, 2, 1, 1, 1, 5, 24, 20, 1, 33, 24, 52, 12, 68, - 1, 1, 44, 6, 3, 1, 1, 3, 10, 33, 5, 35, 13, 29, 3, 51, 1, 12, 15, 1, 16, 16, 10, 5, 1, 55, - 9, 14, 18, 23, 3, 69, 1, 1, 1, 1, 24, 3, 2, 16, 2, 4, 11, 6, 2, 6, 2, 6, 9, 7, 1, 7, 1, 43, - 1, 14, 6, 123, 21, 0, 12, 23, 4, 49, 0, 0, 2, 106, 38, 7, 12, 5, 5, 12, 1, 13, 1, 5, 1, 1, - 1, 2, 1, 2, 1, 108, 33, 0, 18, 64, 2, 54, 40, 12, 116, 5, 1, 135, 36, 26, 6, 26, 11, 89, 3, - 6, 2, 6, 2, 6, 2, 3, 35, 12, 1, 26, 1, 19, 1, 2, 1, 15, 2, 14, 34, 123, 69, 53, 0, 29, 3, - 49, 47, 32, 13, 30, 5, 43, 5, 30, 2, 36, 4, 8, 1, 5, 42, 158, 18, 36, 4, 36, 4, 40, 8, 52, - 12, 11, 1, 15, 1, 7, 1, 2, 1, 11, 1, 15, 1, 7, 1, 2, 67, 0, 9, 22, 10, 8, 24, 6, 1, 42, 1, - 9, 69, 6, 2, 1, 1, 44, 1, 2, 3, 1, 2, 23, 10, 23, 9, 31, 65, 19, 1, 2, 10, 22, 10, 26, 70, - 56, 6, 2, 64, 4, 1, 2, 5, 8, 1, 3, 1, 29, 42, 29, 3, 29, 35, 8, 1, 28, 27, 54, 10, 22, 10, - 19, 13, 18, 110, 73, 55, 51, 13, 51, 13, 40, 0, 42, 1, 2, 3, 2, 78, 29, 10, 1, 8, 22, 42, - 18, 46, 21, 27, 23, 9, 70, 43, 5, 10, 57, 9, 1, 13, 25, 23, 51, 17, 4, 8, 35, 3, 1, 9, 64, - 1, 4, 9, 2, 10, 1, 1, 1, 35, 18, 1, 34, 2, 1, 6, 4, 62, 7, 1, 1, 1, 4, 1, 15, 1, 10, 7, 57, - 23, 4, 1, 8, 2, 2, 2, 22, 1, 7, 1, 2, 1, 5, 3, 8, 2, 2, 2, 2, 3, 1, 6, 1, 5, 7, 156, 66, 1, - 3, 1, 4, 20, 3, 30, 66, 2, 2, 1, 1, 184, 54, 2, 7, 25, 6, 34, 63, 1, 1, 3, 1, 59, 54, 2, 1, - 71, 27, 2, 14, 21, 7, 185, 57, 103, 64, 31, 8, 2, 1, 2, 8, 1, 2, 1, 30, 1, 2, 2, 2, 2, 4, - 93, 8, 2, 46, 2, 6, 1, 1, 1, 2, 27, 51, 2, 10, 17, 72, 5, 1, 18, 73, 0, 9, 1, 45, 1, 7, 1, - 1, 49, 30, 2, 22, 1, 14, 73, 7, 1, 2, 1, 44, 3, 1, 1, 2, 1, 3, 1, 1, 2, 2, 24, 6, 1, 2, 1, - 37, 1, 2, 1, 4, 1, 1, 0, 23, 9, 17, 1, 41, 3, 3, 111, 1, 79, 0, 102, 111, 17, 196, 0, 97, - 15, 0, 17, 6, 0, 0, 0, 0, 7, 31, 17, 79, 17, 30, 18, 48, 16, 4, 31, 21, 5, 19, 0, 64, 128, - 75, 4, 57, 7, 17, 64, 2, 1, 1, 12, 2, 14, 0, 8, 0, 42, 9, 0, 4, 1, 7, 1, 2, 1, 0, 15, 1, 29, - 3, 2, 1, 14, 4, 8, 0, 0, 107, 5, 13, 3, 9, 7, 10, 4, 1, 0, 85, 1, 71, 1, 2, 2, 1, 2, 2, 2, - 4, 1, 12, 1, 1, 1, 7, 1, 65, 1, 4, 2, 8, 1, 7, 1, 28, 1, 4, 1, 5, 1, 1, 3, 7, 1, 0, 2, 25, - 1, 25, 1, 31, 1, 25, 1, 31, 1, 25, 1, 31, 1, 25, 1, 31, 1, 25, 1, 8, 0, 31, 6, 6, 213, 7, 1, - 17, 2, 7, 1, 2, 1, 5, 5, 62, 33, 1, 112, 45, 10, 7, 16, 1, 0, 30, 18, 44, 0, 28, 0, 7, 1, 4, - 1, 2, 1, 15, 1, 197, 59, 68, 3, 1, 3, 1, 0, 4, 1, 27, 1, 2, 1, 1, 2, 1, 1, 10, 1, 4, 1, 1, - 1, 1, 6, 1, 4, 1, 1, 1, 1, 1, 1, 3, 1, 2, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 2, - 4, 1, 7, 1, 4, 1, 4, 1, 1, 1, 10, 1, 17, 5, 3, 1, 5, 1, 17, 0, 26, 6, 26, 6, 26, 0, 0, 32, - 0, 6, 222, 2, 0, 14, 0, 15, 0, 0, 0, 0, 0, 5, 0, 0, + static OFFSETS: [u8; 1515] = [ + 65, 26, 6, 26, 47, 1, 10, 1, 4, 1, 5, 23, 1, 31, 1, 0, 4, 12, 14, 5, 7, 1, 1, 1, 86, 1, 29, + 18, 1, 2, 2, 4, 1, 1, 6, 1, 1, 3, 1, 1, 1, 20, 1, 83, 1, 139, 8, 166, 1, 38, 2, 1, 6, 41, + 39, 14, 1, 1, 1, 2, 1, 2, 1, 1, 8, 27, 4, 4, 29, 11, 5, 56, 1, 7, 14, 102, 1, 8, 4, 8, 4, 3, + 10, 3, 2, 1, 16, 48, 13, 101, 24, 33, 9, 2, 4, 1, 5, 24, 2, 19, 19, 25, 7, 11, 5, 24, 1, 6, + 8, 1, 8, 42, 10, 12, 3, 7, 6, 76, 1, 16, 1, 3, 4, 15, 13, 19, 1, 8, 2, 2, 2, 22, 1, 7, 1, 1, + 3, 4, 3, 8, 2, 2, 2, 2, 1, 1, 8, 1, 4, 2, 1, 5, 12, 2, 10, 1, 4, 3, 1, 6, 4, 2, 2, 22, 1, 7, + 1, 2, 1, 2, 1, 2, 4, 5, 4, 2, 2, 2, 4, 1, 7, 4, 1, 1, 17, 6, 11, 3, 1, 9, 1, 3, 1, 22, 1, 7, + 1, 2, 1, 5, 3, 9, 1, 3, 1, 2, 3, 1, 15, 4, 21, 4, 4, 3, 1, 8, 2, 2, 2, 22, 1, 7, 1, 2, 1, 5, + 3, 8, 2, 2, 2, 2, 9, 2, 4, 2, 1, 5, 13, 1, 16, 2, 1, 6, 3, 3, 1, 4, 3, 2, 1, 1, 1, 2, 3, 2, + 3, 3, 3, 12, 4, 5, 3, 3, 1, 3, 3, 1, 6, 1, 40, 13, 1, 3, 1, 23, 1, 16, 3, 8, 1, 3, 1, 3, 8, + 2, 1, 3, 2, 1, 2, 4, 28, 4, 1, 8, 1, 3, 1, 23, 1, 10, 1, 5, 3, 8, 1, 3, 1, 3, 8, 2, 6, 2, 1, + 4, 13, 3, 12, 13, 1, 3, 1, 41, 2, 8, 1, 3, 1, 3, 1, 1, 5, 4, 7, 5, 22, 6, 1, 3, 1, 18, 3, + 24, 1, 9, 1, 1, 2, 7, 8, 6, 1, 1, 1, 8, 18, 2, 13, 58, 5, 7, 6, 1, 51, 2, 1, 1, 1, 5, 1, 24, + 1, 1, 1, 19, 1, 3, 2, 5, 1, 1, 6, 1, 14, 4, 32, 1, 63, 8, 1, 36, 4, 19, 4, 16, 1, 36, 67, + 55, 1, 1, 2, 5, 16, 64, 10, 4, 2, 38, 1, 1, 5, 1, 2, 43, 1, 0, 1, 4, 2, 7, 1, 1, 1, 4, 2, + 41, 1, 4, 2, 33, 1, 4, 2, 7, 1, 1, 1, 4, 2, 15, 1, 57, 1, 4, 2, 67, 37, 16, 16, 86, 2, 6, 3, + 0, 2, 17, 1, 26, 5, 75, 3, 11, 7, 20, 11, 21, 12, 20, 12, 13, 1, 3, 1, 2, 12, 52, 2, 19, 14, + 1, 4, 1, 67, 89, 7, 43, 5, 70, 10, 31, 1, 12, 4, 9, 23, 30, 2, 5, 11, 44, 4, 26, 54, 28, 4, + 63, 2, 20, 50, 1, 23, 2, 11, 3, 49, 52, 1, 15, 1, 8, 51, 42, 2, 4, 10, 44, 1, 11, 14, 55, + 22, 3, 10, 36, 2, 11, 5, 43, 2, 3, 41, 4, 1, 6, 1, 2, 3, 1, 5, 192, 19, 34, 11, 0, 2, 6, 2, + 38, 2, 6, 2, 8, 1, 1, 1, 1, 1, 1, 1, 31, 2, 53, 1, 7, 1, 1, 3, 3, 1, 7, 3, 4, 2, 6, 4, 13, + 5, 3, 1, 7, 116, 1, 13, 1, 16, 13, 101, 1, 4, 1, 2, 10, 1, 1, 3, 5, 6, 1, 1, 1, 1, 1, 1, 4, + 1, 11, 2, 4, 5, 5, 4, 1, 17, 41, 0, 52, 0, 229, 6, 4, 3, 2, 12, 38, 1, 1, 5, 1, 2, 56, 7, 1, + 16, 23, 9, 7, 1, 7, 1, 7, 1, 7, 1, 7, 1, 7, 1, 7, 1, 7, 1, 32, 47, 1, 0, 3, 25, 9, 7, 5, 2, + 5, 4, 86, 6, 3, 1, 90, 1, 4, 5, 43, 1, 94, 17, 32, 48, 16, 0, 0, 64, 0, 67, 46, 2, 0, 3, 16, + 10, 2, 20, 47, 5, 8, 3, 113, 39, 9, 2, 103, 2, 67, 2, 2, 1, 1, 1, 8, 21, 20, 1, 33, 24, 52, + 12, 68, 1, 1, 44, 6, 3, 1, 1, 3, 10, 33, 5, 35, 13, 29, 3, 51, 1, 12, 15, 1, 16, 16, 10, 5, + 1, 55, 9, 14, 18, 23, 3, 69, 1, 1, 1, 1, 24, 3, 2, 16, 2, 4, 11, 6, 2, 6, 2, 6, 9, 7, 1, 7, + 1, 43, 1, 14, 6, 123, 21, 0, 12, 23, 4, 49, 0, 0, 2, 106, 38, 7, 12, 5, 5, 12, 1, 13, 1, 5, + 1, 1, 1, 2, 1, 2, 1, 108, 33, 0, 18, 64, 2, 54, 40, 12, 116, 5, 1, 135, 36, 26, 6, 26, 11, + 89, 3, 6, 2, 6, 2, 6, 2, 3, 35, 12, 1, 26, 1, 19, 1, 2, 1, 15, 2, 14, 34, 123, 69, 53, 0, + 29, 3, 49, 47, 32, 13, 30, 5, 43, 5, 30, 2, 36, 4, 8, 1, 5, 42, 158, 18, 36, 4, 36, 4, 40, + 8, 52, 12, 11, 1, 15, 1, 7, 1, 2, 1, 11, 1, 15, 1, 7, 1, 2, 3, 52, 12, 0, 9, 22, 10, 8, 24, + 6, 1, 42, 1, 9, 69, 6, 2, 1, 1, 44, 1, 2, 3, 1, 2, 23, 10, 23, 9, 31, 65, 19, 1, 2, 10, 22, + 10, 26, 70, 56, 6, 2, 64, 4, 1, 2, 5, 8, 1, 3, 1, 29, 42, 29, 3, 29, 35, 8, 1, 28, 27, 54, + 10, 22, 10, 19, 13, 18, 110, 73, 55, 51, 13, 51, 13, 40, 34, 28, 3, 1, 5, 23, 250, 42, 1, 2, + 3, 2, 16, 3, 55, 1, 3, 29, 10, 1, 8, 22, 42, 18, 46, 21, 27, 23, 9, 70, 43, 5, 10, 57, 9, 1, + 13, 25, 23, 51, 17, 4, 8, 35, 3, 1, 9, 64, 1, 4, 9, 2, 10, 1, 1, 1, 35, 18, 1, 34, 2, 1, 6, + 4, 62, 7, 1, 1, 1, 4, 1, 15, 1, 10, 7, 57, 23, 4, 1, 8, 2, 2, 2, 22, 1, 7, 1, 2, 1, 5, 3, 8, + 2, 2, 2, 2, 3, 1, 6, 1, 5, 7, 28, 10, 1, 1, 2, 1, 1, 38, 1, 10, 1, 1, 2, 1, 1, 4, 1, 2, 3, + 1, 1, 1, 44, 66, 1, 3, 1, 4, 20, 3, 30, 66, 2, 2, 1, 1, 184, 54, 2, 7, 25, 6, 34, 63, 1, 1, + 3, 1, 59, 54, 2, 1, 71, 27, 2, 14, 21, 7, 185, 57, 103, 64, 31, 8, 2, 1, 2, 8, 1, 2, 1, 30, + 1, 2, 2, 2, 2, 4, 93, 8, 2, 46, 2, 6, 1, 1, 1, 2, 27, 51, 2, 10, 17, 72, 5, 1, 18, 73, 199, + 33, 31, 9, 1, 45, 1, 7, 1, 1, 49, 30, 2, 22, 1, 14, 73, 7, 1, 2, 1, 44, 3, 1, 1, 2, 1, 3, 1, + 1, 2, 2, 24, 6, 1, 2, 1, 37, 1, 2, 1, 4, 1, 1, 0, 23, 9, 17, 1, 41, 3, 3, 111, 1, 79, 0, + 102, 111, 17, 196, 0, 97, 15, 0, 17, 6, 25, 0, 5, 0, 0, 47, 0, 0, 7, 31, 17, 79, 17, 30, 18, + 48, 16, 4, 31, 21, 5, 19, 0, 45, 211, 64, 128, 75, 4, 57, 7, 17, 64, 2, 1, 1, 12, 2, 14, 0, + 8, 0, 41, 10, 0, 4, 1, 7, 1, 2, 1, 0, 15, 1, 29, 3, 2, 1, 14, 4, 8, 0, 0, 107, 5, 13, 3, 9, + 7, 10, 4, 1, 0, 85, 1, 71, 1, 2, 2, 1, 2, 2, 2, 4, 1, 12, 1, 1, 1, 7, 1, 65, 1, 4, 2, 8, 1, + 7, 1, 28, 1, 4, 1, 5, 1, 1, 3, 7, 1, 0, 2, 25, 1, 25, 1, 31, 1, 25, 1, 31, 1, 25, 1, 31, 1, + 25, 1, 31, 1, 25, 1, 8, 0, 31, 6, 6, 213, 7, 1, 17, 2, 7, 1, 2, 1, 5, 5, 62, 33, 1, 112, 45, + 10, 7, 16, 1, 0, 30, 18, 44, 0, 28, 228, 30, 2, 1, 0, 7, 1, 4, 1, 2, 1, 15, 1, 197, 59, 68, + 3, 1, 3, 1, 0, 4, 1, 27, 1, 2, 1, 1, 2, 1, 1, 10, 1, 4, 1, 1, 1, 1, 6, 1, 4, 1, 1, 1, 1, 1, + 1, 3, 1, 2, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 2, 4, 1, 7, 1, 4, 1, 4, 1, 1, 1, + 10, 1, 17, 5, 3, 1, 5, 1, 17, 0, 26, 6, 26, 6, 26, 0, 0, 32, 0, 6, 222, 2, 0, 14, 0, 15, 0, + 0, 0, 0, 0, 5, 0, 0, ]; pub fn lookup(c: char) -> bool { super::skip_search( @@ -180,18 +180,18 @@ pub mod alphabetic { #[rustfmt::skip] pub mod case_ignorable { - static SHORT_OFFSET_RUNS: [u32; 35] = [ + static SHORT_OFFSET_RUNS: [u32; 37] = [ 688, 44045149, 572528402, 576724925, 807414908, 878718981, 903913493, 929080568, 933275148, 937491230, 1138818560, 1147208189, 1210124160, 1222707713, 1235291428, 1260457643, - 1264654383, 1499535675, 1507925040, 1566646003, 1629566000, 1650551536, 1658941263, - 1671540720, 1688321181, 1700908800, 1709298023, 1717688832, 1738661888, 1763828398, - 1797383403, 1805773008, 1809970171, 1819148289, 1824457200, + 1277237295, 1537284411, 1545673776, 1604394739, 1667314736, 1692492062, 1700883184, + 1709272384, 1721855823, 1730260976, 1747041437, 1759629056, 1768018279, 1776409088, + 1797382144, 1822548654, 1856103659, 1864493264, 1872884731, 1882062849, 1887371760, ]; - static OFFSETS: [u8; 875] = [ + static OFFSETS: [u8; 905] = [ 39, 1, 6, 1, 11, 1, 35, 1, 1, 1, 71, 1, 4, 1, 1, 1, 4, 1, 2, 2, 0, 192, 4, 2, 4, 1, 9, 2, 1, 1, 251, 7, 207, 1, 5, 1, 49, 45, 1, 1, 1, 2, 1, 2, 1, 1, 44, 1, 11, 6, 10, 11, 1, 1, 35, 1, 10, 21, 16, 1, 101, 8, 1, 10, 1, 4, 33, 1, 1, 1, 30, 27, 91, 11, 58, 11, 4, 1, 2, 1, 24, - 24, 43, 3, 44, 1, 7, 2, 6, 8, 41, 58, 55, 1, 1, 1, 4, 8, 4, 1, 3, 7, 10, 2, 13, 1, 15, 1, + 24, 43, 3, 44, 1, 7, 2, 5, 9, 41, 58, 55, 1, 1, 1, 4, 8, 4, 1, 3, 7, 10, 2, 13, 1, 15, 1, 58, 1, 4, 4, 8, 1, 20, 2, 26, 1, 2, 2, 57, 1, 4, 2, 4, 2, 2, 3, 3, 1, 30, 2, 3, 1, 11, 2, 57, 1, 4, 5, 1, 2, 4, 1, 20, 2, 22, 6, 1, 1, 58, 1, 2, 1, 1, 4, 8, 1, 7, 2, 11, 2, 30, 1, 61, 1, 12, 1, 50, 1, 3, 1, 55, 1, 1, 3, 5, 3, 1, 4, 7, 2, 11, 2, 29, 1, 58, 1, 2, 1, 6, 1, @@ -209,17 +209,18 @@ pub mod case_ignorable { 2, 2, 17, 1, 21, 2, 66, 6, 2, 2, 2, 2, 12, 1, 8, 1, 35, 1, 11, 1, 51, 1, 1, 3, 2, 2, 5, 2, 1, 1, 27, 1, 14, 2, 5, 2, 1, 1, 100, 5, 9, 3, 121, 1, 2, 1, 4, 1, 0, 1, 147, 17, 0, 16, 3, 1, 12, 16, 34, 1, 2, 1, 169, 1, 7, 1, 6, 1, 11, 1, 35, 1, 1, 1, 47, 1, 45, 2, 67, 1, 21, 3, - 0, 1, 226, 1, 149, 5, 0, 6, 1, 42, 1, 9, 0, 3, 1, 2, 5, 4, 40, 3, 4, 1, 165, 2, 0, 4, 0, 2, - 80, 3, 70, 11, 49, 4, 123, 1, 54, 15, 41, 1, 2, 2, 10, 3, 49, 4, 2, 2, 2, 1, 4, 1, 10, 1, - 50, 3, 36, 5, 1, 8, 62, 1, 12, 2, 52, 9, 10, 4, 2, 1, 95, 3, 2, 1, 1, 2, 6, 1, 2, 1, 157, 1, - 3, 8, 21, 2, 57, 2, 3, 1, 37, 7, 3, 5, 195, 8, 2, 3, 1, 1, 23, 1, 84, 6, 1, 1, 4, 2, 1, 2, - 238, 4, 6, 2, 1, 2, 27, 2, 85, 8, 2, 1, 1, 2, 106, 1, 1, 1, 2, 6, 1, 1, 101, 3, 2, 4, 1, 5, - 0, 9, 1, 2, 0, 2, 1, 1, 4, 1, 144, 4, 2, 2, 4, 1, 32, 10, 40, 6, 2, 4, 8, 1, 9, 6, 2, 3, 46, - 13, 1, 2, 0, 7, 1, 6, 1, 1, 82, 22, 2, 7, 1, 2, 1, 2, 122, 6, 3, 1, 1, 2, 1, 7, 1, 1, 72, 2, - 3, 1, 1, 1, 0, 2, 11, 2, 52, 5, 5, 1, 1, 1, 0, 17, 6, 15, 0, 5, 59, 7, 9, 4, 0, 1, 63, 17, - 64, 2, 1, 2, 0, 4, 1, 7, 1, 2, 0, 2, 1, 4, 0, 46, 2, 23, 0, 3, 9, 16, 2, 7, 30, 4, 148, 3, - 0, 55, 4, 50, 8, 1, 14, 1, 22, 5, 1, 15, 0, 7, 1, 17, 2, 7, 1, 2, 1, 5, 5, 62, 33, 1, 160, - 14, 0, 1, 61, 4, 0, 5, 0, 7, 109, 8, 0, 5, 0, 1, 30, 96, 128, 240, 0, + 0, 1, 226, 1, 149, 5, 0, 6, 1, 42, 1, 9, 0, 3, 1, 2, 5, 4, 40, 3, 4, 1, 165, 2, 0, 4, 38, 1, + 26, 5, 1, 1, 0, 2, 79, 4, 70, 11, 49, 4, 123, 1, 54, 15, 41, 1, 2, 2, 10, 3, 49, 4, 2, 2, 2, + 1, 4, 1, 10, 1, 50, 3, 36, 5, 1, 8, 62, 1, 12, 2, 52, 9, 10, 4, 2, 1, 95, 3, 2, 1, 1, 2, 6, + 1, 2, 1, 157, 1, 3, 8, 21, 2, 57, 2, 3, 1, 37, 7, 3, 5, 70, 6, 13, 1, 1, 1, 1, 1, 14, 2, 85, + 8, 2, 3, 1, 1, 23, 1, 84, 6, 1, 1, 4, 2, 1, 2, 238, 4, 6, 2, 1, 2, 27, 2, 85, 8, 2, 1, 1, 2, + 106, 1, 1, 1, 2, 6, 1, 1, 101, 1, 1, 1, 2, 4, 1, 5, 0, 9, 1, 2, 0, 2, 1, 1, 4, 1, 144, 4, 2, + 2, 4, 1, 32, 10, 40, 6, 2, 4, 8, 1, 9, 6, 2, 3, 46, 13, 1, 2, 0, 7, 1, 6, 1, 1, 82, 22, 2, + 7, 1, 2, 1, 2, 122, 6, 3, 1, 1, 2, 1, 7, 1, 1, 72, 2, 3, 1, 1, 1, 0, 2, 11, 2, 52, 5, 5, 1, + 1, 1, 23, 1, 0, 17, 6, 15, 0, 12, 3, 3, 0, 5, 59, 7, 9, 4, 0, 3, 40, 2, 0, 1, 63, 17, 64, 2, + 1, 2, 0, 4, 1, 7, 1, 2, 0, 2, 1, 4, 0, 46, 2, 23, 0, 3, 9, 16, 2, 7, 30, 4, 148, 3, 0, 55, + 4, 50, 8, 1, 14, 1, 22, 5, 1, 15, 0, 7, 1, 17, 2, 7, 1, 2, 1, 5, 5, 62, 33, 1, 160, 14, 0, + 1, 61, 4, 0, 5, 254, 2, 0, 7, 109, 8, 0, 5, 0, 1, 30, 96, 128, 240, 0, ]; pub fn lookup(c: char) -> bool { super::skip_search( @@ -234,22 +235,22 @@ pub mod case_ignorable { pub mod cased { static SHORT_OFFSET_RUNS: [u32; 22] = [ 4256, 115348384, 136322176, 144711446, 163587254, 320875520, 325101120, 350268208, - 392231680, 404815649, 413205504, 421595008, 467733632, 484513952, 492924480, 497144832, - 501339814, 578936576, 627171376, 639756544, 643952944, 649261450, + 392231680, 404815649, 413205504, 421595008, 467733632, 484513952, 501313088, 505533440, + 509728422, 587325184, 635559984, 648145152, 652341552, 657650058, ]; - static OFFSETS: [u8; 315] = [ + static OFFSETS: [u8; 319] = [ 65, 26, 6, 26, 47, 1, 10, 1, 4, 1, 5, 23, 1, 31, 1, 195, 1, 4, 4, 208, 1, 36, 7, 2, 30, 5, 96, 1, 42, 4, 2, 2, 2, 4, 1, 1, 6, 1, 1, 3, 1, 1, 1, 20, 1, 83, 1, 139, 8, 166, 1, 38, 9, - 41, 0, 38, 1, 1, 5, 1, 2, 43, 1, 4, 0, 86, 2, 6, 0, 9, 7, 43, 2, 3, 64, 192, 64, 0, 2, 6, 2, - 38, 2, 6, 2, 8, 1, 1, 1, 1, 1, 1, 1, 31, 2, 53, 1, 7, 1, 1, 3, 3, 1, 7, 3, 4, 2, 6, 4, 13, - 5, 3, 1, 7, 116, 1, 13, 1, 16, 13, 101, 1, 4, 1, 2, 10, 1, 1, 3, 5, 6, 1, 1, 1, 1, 1, 1, 4, - 1, 6, 4, 1, 2, 4, 5, 5, 4, 1, 17, 32, 3, 2, 0, 52, 0, 229, 6, 4, 3, 2, 12, 38, 1, 1, 5, 1, - 0, 46, 18, 30, 132, 102, 3, 4, 1, 59, 5, 2, 1, 1, 1, 5, 24, 5, 1, 3, 0, 43, 1, 14, 6, 80, 0, - 7, 12, 5, 0, 26, 6, 26, 0, 80, 96, 36, 4, 36, 116, 11, 1, 15, 1, 7, 1, 2, 1, 11, 1, 15, 1, - 7, 1, 2, 0, 1, 2, 3, 1, 42, 1, 9, 0, 51, 13, 51, 0, 64, 0, 64, 0, 85, 1, 71, 1, 2, 2, 1, 2, - 2, 2, 4, 1, 12, 1, 1, 1, 7, 1, 65, 1, 4, 2, 8, 1, 7, 1, 28, 1, 4, 1, 5, 1, 1, 3, 7, 1, 0, 2, - 25, 1, 25, 1, 31, 1, 25, 1, 31, 1, 25, 1, 31, 1, 25, 1, 31, 1, 25, 1, 8, 0, 10, 1, 20, 6, 6, - 0, 62, 0, 68, 0, 26, 6, 26, 6, 26, 0, + 41, 0, 38, 1, 1, 5, 1, 2, 43, 1, 4, 0, 86, 2, 6, 0, 11, 5, 43, 2, 3, 64, 192, 64, 0, 2, 6, + 2, 38, 2, 6, 2, 8, 1, 1, 1, 1, 1, 1, 1, 31, 2, 53, 1, 7, 1, 1, 3, 3, 1, 7, 3, 4, 2, 6, 4, + 13, 5, 3, 1, 7, 116, 1, 13, 1, 16, 13, 101, 1, 4, 1, 2, 10, 1, 1, 3, 5, 6, 1, 1, 1, 1, 1, 1, + 4, 1, 6, 4, 1, 2, 4, 5, 5, 4, 1, 17, 32, 3, 2, 0, 52, 0, 229, 6, 4, 3, 2, 12, 38, 1, 1, 5, + 1, 0, 46, 18, 30, 132, 102, 3, 4, 1, 62, 2, 2, 1, 1, 1, 8, 21, 5, 1, 3, 0, 43, 1, 14, 6, 80, + 0, 7, 12, 5, 0, 26, 6, 26, 0, 80, 96, 36, 4, 36, 116, 11, 1, 15, 1, 7, 1, 2, 1, 11, 1, 15, + 1, 7, 1, 2, 0, 1, 2, 3, 1, 42, 1, 9, 0, 51, 13, 51, 93, 22, 10, 22, 0, 64, 0, 64, 0, 85, 1, + 71, 1, 2, 2, 1, 2, 2, 2, 4, 1, 12, 1, 1, 1, 7, 1, 65, 1, 4, 2, 8, 1, 7, 1, 28, 1, 4, 1, 5, + 1, 1, 3, 7, 1, 0, 2, 25, 1, 25, 1, 31, 1, 25, 1, 31, 1, 25, 1, 31, 1, 25, 1, 31, 1, 25, 1, + 8, 0, 10, 1, 20, 6, 6, 0, 62, 0, 68, 0, 26, 6, 26, 6, 26, 0, ]; pub fn lookup(c: char) -> bool { super::skip_search( @@ -279,41 +280,41 @@ pub mod cc { #[rustfmt::skip] pub mod grapheme_extend { - static SHORT_OFFSET_RUNS: [u32; 33] = [ - 768, 2098307, 6292881, 10490717, 522196754, 526393356, 731917551, 740306986, 752920175, - 761309186, 778107678, 908131840, 912326558, 920715773, 924912129, 937495844, 962662059, - 966858799, 1214323760, 1285627635, 1348547648, 1369533168, 1377922895, 1386331293, - 1398918912, 1403113829, 1411504640, 1440866304, 1466032814, 1495393516, 1503783120, - 1508769824, 1518273008, + static SHORT_OFFSET_RUNS: [u32; 34] = [ + 768, 2098307, 6292881, 10490717, 522196754, 526393356, 723528943, 731918378, 744531567, + 752920578, 769719070, 908131840, 912326558, 920715773, 924912129, 937495844, 962662059, + 971053103, 1256266800, 1323376371, 1386296384, 1407279390, 1415670512, 1424060239, + 1432468637, 1449250560, 1453445477, 1461836288, 1487003648, 1512170158, 1541530860, + 1549920464, 1559101472, 1568604656, ]; - static OFFSETS: [u8; 727] = [ + static OFFSETS: [u8; 751] = [ 0, 112, 0, 7, 0, 45, 1, 1, 1, 2, 1, 2, 1, 1, 72, 11, 48, 21, 16, 1, 101, 7, 2, 6, 2, 2, 1, - 4, 35, 1, 30, 27, 91, 11, 58, 9, 9, 1, 24, 4, 1, 9, 1, 3, 1, 5, 43, 3, 60, 8, 42, 24, 1, 32, + 4, 35, 1, 30, 27, 91, 11, 58, 9, 9, 1, 24, 4, 1, 9, 1, 3, 1, 5, 43, 3, 59, 9, 42, 24, 1, 32, 55, 1, 1, 1, 4, 8, 4, 1, 3, 7, 10, 2, 29, 1, 58, 1, 1, 1, 2, 4, 8, 1, 9, 1, 10, 2, 26, 1, 2, 2, 57, 1, 4, 2, 4, 2, 2, 3, 3, 1, 30, 2, 3, 1, 11, 2, 57, 1, 4, 5, 1, 2, 4, 1, 20, 2, 22, 6, 1, 1, 58, 1, 1, 2, 1, 4, 8, 1, 7, 3, 10, 2, 30, 1, 59, 1, 1, 1, 12, 1, 9, 1, 40, 1, 3, 1, - 55, 1, 1, 3, 5, 3, 1, 4, 7, 2, 11, 2, 29, 1, 58, 1, 2, 1, 2, 1, 3, 1, 5, 2, 7, 2, 11, 2, 28, + 55, 1, 1, 3, 5, 3, 1, 4, 7, 2, 11, 2, 29, 1, 58, 1, 2, 2, 1, 1, 3, 3, 1, 4, 7, 2, 11, 2, 28, 2, 57, 2, 1, 1, 2, 4, 8, 1, 9, 1, 10, 2, 29, 1, 72, 1, 4, 1, 2, 3, 1, 1, 8, 1, 81, 1, 2, 7, 12, 8, 98, 1, 2, 9, 11, 7, 73, 2, 27, 1, 1, 1, 1, 1, 55, 14, 1, 5, 1, 2, 5, 11, 1, 36, 9, 1, - 102, 4, 1, 6, 1, 2, 2, 2, 25, 2, 4, 3, 16, 4, 13, 1, 2, 2, 6, 1, 15, 1, 0, 3, 0, 3, 29, 2, - 30, 2, 30, 2, 64, 2, 1, 7, 8, 1, 2, 11, 9, 1, 45, 3, 1, 1, 117, 2, 34, 1, 118, 3, 4, 2, 9, - 1, 6, 3, 219, 2, 2, 1, 58, 1, 1, 7, 1, 1, 1, 1, 2, 8, 6, 10, 2, 1, 48, 31, 49, 4, 48, 7, 1, - 1, 5, 1, 40, 9, 12, 2, 32, 4, 2, 2, 1, 3, 56, 1, 1, 2, 3, 1, 1, 3, 58, 8, 2, 2, 152, 3, 1, - 13, 1, 7, 4, 1, 6, 1, 3, 2, 198, 64, 0, 1, 195, 33, 0, 3, 141, 1, 96, 32, 0, 6, 105, 2, 0, - 4, 1, 10, 32, 2, 80, 2, 0, 1, 3, 1, 4, 1, 25, 2, 5, 1, 151, 2, 26, 18, 13, 1, 38, 8, 25, 11, - 46, 3, 48, 1, 2, 4, 2, 2, 39, 1, 67, 6, 2, 2, 2, 2, 12, 1, 8, 1, 47, 1, 51, 1, 1, 3, 2, 2, - 5, 2, 1, 1, 42, 2, 8, 1, 238, 1, 2, 1, 4, 1, 0, 1, 0, 16, 16, 16, 0, 2, 0, 1, 226, 1, 149, - 5, 0, 3, 1, 2, 5, 4, 40, 3, 4, 1, 165, 2, 0, 4, 0, 2, 80, 3, 70, 11, 49, 4, 123, 1, 54, 15, - 41, 1, 2, 2, 10, 3, 49, 4, 2, 2, 7, 1, 61, 3, 36, 5, 1, 8, 62, 1, 12, 2, 52, 9, 10, 4, 2, 1, - 95, 3, 2, 1, 1, 2, 6, 1, 2, 1, 157, 1, 3, 8, 21, 2, 57, 2, 1, 1, 1, 1, 22, 1, 14, 7, 3, 5, - 195, 8, 2, 3, 1, 1, 23, 1, 81, 1, 2, 6, 1, 1, 2, 1, 1, 2, 1, 2, 235, 1, 2, 4, 6, 2, 1, 2, - 27, 2, 85, 8, 2, 1, 1, 2, 106, 1, 1, 1, 2, 6, 1, 1, 101, 3, 2, 4, 1, 5, 0, 9, 1, 2, 245, 1, - 10, 2, 1, 1, 4, 1, 144, 4, 2, 2, 4, 1, 32, 10, 40, 6, 2, 4, 8, 1, 9, 6, 2, 3, 46, 13, 1, 2, - 0, 7, 1, 6, 1, 1, 82, 22, 2, 7, 1, 2, 1, 2, 122, 6, 3, 1, 1, 2, 1, 7, 1, 1, 72, 2, 3, 1, 1, - 1, 0, 2, 11, 2, 52, 5, 5, 1, 1, 1, 0, 1, 6, 15, 0, 5, 59, 7, 0, 1, 63, 4, 81, 1, 0, 2, 0, - 46, 2, 23, 0, 1, 1, 3, 4, 5, 8, 8, 2, 7, 30, 4, 148, 3, 0, 55, 4, 50, 8, 1, 14, 1, 22, 5, 1, - 15, 0, 7, 1, 17, 2, 7, 1, 2, 1, 5, 100, 1, 160, 7, 0, 1, 61, 4, 0, 4, 0, 7, 109, 7, 0, 96, - 128, 240, 0, + 102, 4, 1, 6, 1, 2, 2, 2, 25, 2, 4, 3, 16, 4, 13, 1, 2, 2, 6, 1, 15, 1, 0, 3, 0, 4, 28, 3, + 29, 2, 30, 2, 64, 2, 1, 7, 8, 1, 2, 11, 9, 1, 45, 3, 1, 1, 117, 2, 34, 1, 118, 3, 4, 2, 9, + 1, 6, 3, 219, 2, 2, 1, 58, 1, 1, 7, 1, 1, 1, 1, 2, 8, 6, 10, 2, 1, 48, 31, 49, 4, 48, 10, 4, + 3, 38, 9, 12, 2, 32, 4, 2, 6, 56, 1, 1, 2, 3, 1, 1, 5, 56, 8, 2, 2, 152, 3, 1, 13, 1, 7, 4, + 1, 6, 1, 3, 2, 198, 64, 0, 1, 195, 33, 0, 3, 141, 1, 96, 32, 0, 6, 105, 2, 0, 4, 1, 10, 32, + 2, 80, 2, 0, 1, 3, 1, 4, 1, 25, 2, 5, 1, 151, 2, 26, 18, 13, 1, 38, 8, 25, 11, 1, 1, 44, 3, + 48, 1, 2, 4, 2, 2, 2, 1, 36, 1, 67, 6, 2, 2, 2, 2, 12, 1, 8, 1, 47, 1, 51, 1, 1, 3, 2, 2, 5, + 2, 1, 1, 42, 2, 8, 1, 238, 1, 2, 1, 4, 1, 0, 1, 0, 16, 16, 16, 0, 2, 0, 1, 226, 1, 149, 5, + 0, 3, 1, 2, 5, 4, 40, 3, 4, 1, 165, 2, 0, 4, 65, 5, 0, 2, 79, 4, 70, 11, 49, 4, 123, 1, 54, + 15, 41, 1, 2, 2, 10, 3, 49, 4, 2, 2, 7, 1, 61, 3, 36, 5, 1, 8, 62, 1, 12, 2, 52, 9, 1, 1, 8, + 4, 2, 1, 95, 3, 2, 4, 6, 1, 2, 1, 157, 1, 3, 8, 21, 2, 57, 2, 1, 1, 1, 1, 12, 1, 9, 1, 14, + 7, 3, 5, 67, 1, 2, 6, 1, 1, 2, 1, 1, 3, 4, 3, 1, 1, 14, 2, 85, 8, 2, 3, 1, 1, 23, 1, 81, 1, + 2, 6, 1, 1, 2, 1, 1, 2, 1, 2, 235, 1, 2, 4, 6, 2, 1, 2, 27, 2, 85, 8, 2, 1, 1, 2, 106, 1, 1, + 1, 2, 8, 101, 1, 1, 1, 2, 4, 1, 5, 0, 9, 1, 2, 245, 1, 10, 4, 4, 1, 144, 4, 2, 2, 4, 1, 32, + 10, 40, 6, 2, 4, 8, 1, 9, 6, 2, 3, 46, 13, 1, 2, 0, 7, 1, 6, 1, 1, 82, 22, 2, 7, 1, 2, 1, 2, + 122, 6, 3, 1, 1, 2, 1, 7, 1, 1, 72, 2, 3, 1, 1, 1, 0, 2, 11, 2, 52, 5, 5, 3, 23, 1, 0, 1, 6, + 15, 0, 12, 3, 3, 0, 5, 59, 7, 0, 1, 63, 4, 81, 1, 11, 2, 0, 2, 0, 46, 2, 23, 0, 5, 3, 6, 8, + 8, 2, 7, 30, 4, 148, 3, 0, 55, 4, 50, 8, 1, 14, 1, 22, 5, 1, 15, 0, 7, 1, 17, 2, 7, 1, 2, 1, + 5, 100, 1, 160, 7, 0, 1, 61, 4, 0, 4, 254, 2, 0, 7, 109, 7, 0, 96, 128, 240, 0, ]; #[inline] pub fn lookup(c: char) -> bool { @@ -330,36 +331,36 @@ pub mod grapheme_extend { #[rustfmt::skip] pub mod lowercase { - const BITSET_CHUNKS_MAP: &'static [u8; 123] = &[ + static BITSET_CHUNKS_MAP: [u8; 123] = [ 14, 17, 0, 0, 9, 0, 0, 12, 13, 10, 0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 1, 0, 15, 0, 8, 0, 0, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 19, 0, 3, 18, 0, 7, ]; - const BITSET_INDEX_CHUNKS: &'static [[u8; 16]; 20] = &[ + static BITSET_INDEX_CHUNKS: [[u8; 16]; 20] = [ [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], - [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 59, 0, 0], - [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 14, 55, 0], + [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 61, 0, 0], + [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 14, 56, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 40, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 44, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 9, 0, 0, 0, 0, 0, 0], - [0, 0, 0, 0, 0, 0, 0, 0, 0, 65, 43, 0, 51, 47, 49, 33], - [0, 0, 0, 0, 10, 56, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], - [0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + [0, 0, 0, 0, 0, 0, 0, 0, 0, 67, 43, 0, 52, 48, 50, 33], + [0, 0, 0, 0, 10, 57, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + [0, 0, 0, 3, 0, 16, 58, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 27], - [0, 0, 0, 60, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], - [0, 0, 0, 69, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], - [0, 0, 57, 0, 55, 55, 55, 0, 22, 22, 67, 22, 36, 25, 24, 37], - [0, 5, 68, 0, 29, 15, 73, 0, 0, 0, 0, 0, 0, 0, 0, 0], - [0, 64, 34, 17, 23, 52, 53, 48, 46, 8, 35, 42, 0, 28, 13, 31], - [11, 58, 0, 6, 0, 0, 30, 0, 0, 0, 0, 0, 0, 0, 32, 0], + [0, 0, 0, 62, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + [0, 0, 0, 71, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + [0, 0, 46, 0, 56, 56, 56, 0, 22, 22, 69, 22, 36, 25, 24, 37], + [0, 5, 70, 0, 29, 15, 75, 0, 0, 0, 0, 0, 0, 0, 0, 0], + [0, 66, 34, 17, 23, 53, 54, 49, 47, 8, 35, 42, 0, 28, 13, 31], + [11, 60, 0, 6, 0, 0, 30, 0, 0, 0, 0, 0, 0, 0, 32, 0], [16, 26, 22, 38, 39, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], - [16, 50, 2, 21, 66, 9, 57, 0, 0, 0, 0, 0, 0, 0, 0, 0], - [16, 70, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], - [63, 41, 54, 12, 75, 61, 18, 1, 7, 62, 74, 20, 71, 72, 4, 45], + [16, 51, 2, 21, 68, 9, 59, 0, 0, 0, 0, 0, 0, 0, 0, 0], + [16, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + [65, 41, 55, 12, 77, 63, 18, 1, 7, 64, 76, 20, 73, 74, 4, 45], ]; - const BITSET_CANONICAL: &'static [u64; 55] = &[ + static BITSET_CANONICAL: [u64; 56] = [ 0b0000000000000000000000000000000000000000000000000000000000000000, 0b1111111111111111110000000000000000000000000011111111111111111111, 0b1010101010101010101010101010101010101010101010101010100000000010, @@ -393,7 +394,7 @@ pub mod lowercase { 0b0001101111111011111111111111101111111111100000000000000000000000, 0b0001100100101111101010101010101010101010111000110111111111111111, 0b0000011111111101111111111111111111111111111111111111111110111001, - 0b0000011101011100000000000000000000000010101010100000010100001010, + 0b0000011101011100000000000000000000001010101010100010010100001010, 0b0000010000100000000001000000000000000000000000000000000000000000, 0b0000000111111111111111111111111111111111111011111111111111111111, 0b0000000011111111000000001111111100000000001111110000000011111111, @@ -406,6 +407,7 @@ pub mod lowercase { 0b0000000000000000000000000000000000111010101010101010101010101010, 0b0000000000000000000000000000000000000000111110000000000001111111, 0b0000000000000000000000000000000000000000000000000000101111110111, + 0b0000000000000000000000000000000000000000000000000000010111111111, 0b1001001111111010101010101010101010101010101010101010101010101010, 0b1001010111111111101010101010101010101010101010101010101010101010, 0b1010101000101001101010101010101010110101010101010101001001000000, @@ -416,9 +418,9 @@ pub mod lowercase { 0b1110011001010001001011010010101001001110001001000011000100101001, 0b1110101111000000000000000000000000001111111111111111111111111100, ]; - const BITSET_MAPPING: &'static [(u8, u8); 21] = &[ - (0, 64), (1, 188), (1, 183), (1, 176), (1, 109), (1, 124), (1, 126), (1, 66), (1, 70), - (1, 77), (2, 146), (2, 144), (2, 83), (3, 93), (3, 147), (3, 133), (4, 12), (4, 6), + static BITSET_MAPPING: [(u8, u8); 22] = [ + (0, 64), (1, 188), (1, 186), (1, 183), (1, 176), (1, 109), (1, 124), (1, 126), (1, 66), + (1, 70), (1, 77), (2, 146), (2, 144), (2, 83), (3, 93), (3, 147), (3, 133), (4, 12), (4, 6), (5, 187), (6, 78), (7, 132), ]; @@ -436,14 +438,15 @@ pub mod lowercase { #[rustfmt::skip] pub mod n { - static SHORT_OFFSET_RUNS: [u32; 39] = [ + static SHORT_OFFSET_RUNS: [u32; 42] = [ 1632, 18876774, 31461440, 102765417, 111154926, 115349830, 132128880, 165684320, 186656630, 195046653, 199241735, 203436434, 216049184, 241215536, 249605104, 274792208, 278987015, - 283181793, 295766104, 320933114, 383848032, 392238160, 434181712, 442570976, 455154768, - 463544144, 476128256, 484534880, 488730240, 505533120, 509728718, 522314048, 526508784, - 530703600, 534898887, 539094129, 547483904, 568458224, 573766650, + 283181793, 295766104, 320933114, 383848032, 396432464, 438376016, 446765280, 463543280, + 471932752, 488711168, 497115440, 501312096, 505507184, 522284672, 526503152, 530698944, + 534894542, 547479872, 551674608, 555869424, 560064711, 568454257, 576844032, 597818352, + 603126778, ]; - static OFFSETS: [u8; 275] = [ + static OFFSETS: [u8; 289] = [ 48, 10, 120, 2, 5, 1, 2, 3, 0, 10, 134, 10, 198, 10, 0, 10, 118, 10, 4, 6, 108, 10, 118, 10, 118, 10, 2, 6, 110, 13, 115, 10, 8, 7, 103, 10, 104, 7, 7, 19, 109, 10, 96, 10, 118, 10, 70, 20, 0, 10, 70, 10, 0, 20, 0, 3, 239, 10, 6, 10, 22, 10, 0, 10, 128, 11, 165, 10, 6, 10, @@ -451,11 +454,11 @@ pub mod n { 1, 0, 1, 25, 9, 14, 3, 0, 4, 138, 10, 30, 8, 1, 15, 32, 10, 39, 15, 0, 10, 188, 10, 0, 6, 154, 10, 38, 10, 198, 10, 22, 10, 86, 10, 0, 10, 0, 10, 0, 45, 12, 57, 17, 2, 0, 27, 36, 4, 29, 1, 8, 1, 134, 5, 202, 10, 0, 8, 25, 7, 39, 9, 75, 5, 22, 6, 160, 2, 2, 16, 2, 46, 64, 9, - 52, 2, 30, 3, 75, 5, 104, 8, 24, 8, 41, 7, 0, 6, 48, 10, 0, 31, 158, 10, 42, 4, 112, 7, 134, - 30, 128, 10, 60, 10, 144, 10, 7, 20, 251, 10, 0, 10, 118, 10, 0, 10, 102, 10, 102, 12, 0, - 19, 93, 10, 0, 29, 227, 10, 70, 10, 0, 10, 102, 21, 0, 111, 0, 10, 86, 10, 134, 10, 1, 7, 0, - 23, 0, 20, 12, 20, 108, 25, 0, 50, 0, 10, 0, 10, 0, 10, 0, 9, 128, 10, 0, 59, 1, 3, 1, 4, - 76, 45, 1, 15, 0, 13, 0, 10, 0, + 52, 2, 30, 3, 75, 5, 104, 8, 24, 8, 41, 7, 0, 6, 48, 10, 6, 10, 0, 31, 158, 10, 42, 4, 112, + 7, 134, 30, 128, 10, 60, 10, 144, 10, 7, 20, 251, 10, 0, 10, 118, 10, 0, 10, 102, 10, 6, 20, + 76, 12, 0, 19, 93, 10, 0, 10, 86, 29, 227, 10, 70, 10, 0, 10, 102, 21, 0, 111, 0, 10, 0, 10, + 86, 10, 134, 10, 1, 7, 0, 10, 0, 23, 0, 10, 0, 20, 12, 20, 108, 25, 0, 50, 0, 10, 0, 10, 0, + 10, 247, 10, 0, 9, 128, 10, 0, 59, 1, 3, 1, 4, 76, 45, 1, 15, 0, 13, 0, 10, 0, ]; pub fn lookup(c: char) -> bool { super::skip_search( @@ -468,38 +471,38 @@ pub mod n { #[rustfmt::skip] pub mod uppercase { - const BITSET_CHUNKS_MAP: &'static [u8; 125] = &[ + static BITSET_CHUNKS_MAP: [u8; 125] = [ 12, 15, 6, 6, 0, 6, 6, 2, 4, 11, 6, 16, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 8, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 5, 6, 14, 6, 10, 6, 6, 1, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 13, 6, 6, 6, 6, 9, 6, 3, ]; - const BITSET_INDEX_CHUNKS: &'static [[u8; 16]; 17] = &[ - [43, 43, 5, 34, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 5, 1], - [43, 43, 5, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43], - [43, 43, 39, 43, 43, 43, 43, 43, 17, 17, 62, 17, 42, 29, 24, 23], - [43, 43, 43, 43, 9, 8, 44, 43, 43, 43, 43, 43, 43, 43, 43, 43], - [43, 43, 43, 43, 36, 28, 66, 43, 43, 43, 43, 43, 43, 43, 43, 43], - [43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 0, 43, 43, 43], - [43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43], - [43, 43, 43, 43, 43, 43, 43, 43, 43, 54, 43, 43, 43, 43, 43, 43], - [43, 43, 43, 43, 43, 43, 43, 43, 43, 61, 60, 43, 20, 14, 16, 4], - [43, 43, 43, 43, 55, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43], - [43, 43, 58, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43], - [43, 43, 59, 45, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43], - [43, 48, 43, 31, 35, 21, 22, 15, 13, 33, 43, 43, 43, 11, 30, 38], - [51, 53, 26, 49, 12, 7, 25, 50, 40, 52, 6, 3, 65, 64, 63, 67], - [56, 43, 9, 46, 43, 41, 32, 43, 43, 43, 43, 43, 43, 43, 43, 43], - [57, 19, 2, 18, 10, 47, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43], - [57, 37, 17, 27, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43], + static BITSET_INDEX_CHUNKS: [[u8; 16]; 17] = [ + [44, 44, 5, 35, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 5, 1], + [44, 44, 5, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44], + [44, 44, 40, 44, 44, 44, 44, 44, 17, 17, 63, 17, 43, 29, 24, 23], + [44, 44, 44, 44, 9, 8, 45, 44, 44, 44, 44, 44, 44, 44, 44, 44], + [44, 44, 44, 44, 37, 28, 67, 44, 44, 44, 44, 44, 44, 44, 44, 44], + [44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 0, 44, 44, 44], + [44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44], + [44, 44, 44, 44, 44, 44, 44, 44, 44, 55, 44, 44, 44, 44, 44, 44], + [44, 44, 44, 44, 44, 44, 44, 44, 44, 62, 61, 44, 20, 14, 16, 4], + [44, 44, 44, 44, 56, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44], + [44, 44, 59, 44, 44, 31, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44], + [44, 44, 60, 46, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44], + [44, 49, 44, 32, 36, 21, 22, 15, 13, 34, 44, 44, 44, 11, 30, 39], + [52, 54, 26, 50, 12, 7, 25, 51, 41, 53, 6, 3, 66, 65, 64, 68], + [57, 44, 9, 47, 44, 42, 33, 44, 44, 44, 44, 44, 44, 44, 44, 44], + [58, 19, 2, 18, 10, 48, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44], + [58, 38, 17, 27, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44], ]; - const BITSET_CANONICAL: &'static [u64; 43] = &[ + static BITSET_CANONICAL: [u64; 44] = [ 0b0000011111111111111111111111111000000000000000000000000000000000, 0b0000000000111111111111111111111111111111111111111111111111111111, 0b0101010101010101010101010101010101010101010101010101010000000001, 0b0000011111111111111111111111110000000000000000000000000000000001, - 0b0000000000100000000000000000000000000001010000010000001011110101, + 0b0000000000100000000000000000000000010101010000010001101011110101, 0b1111111111111111111111111111111100000000000000000000000000000000, 0b1111111111111111111111110000000000000000000000000000001111111111, 0b1111111111111111111100000000000000000000000000011111110001011111, @@ -526,6 +529,7 @@ pub mod uppercase { 0b0000000000000000111111111111111100000000000000000000000000100000, 0b0000000000000000111111110000000010101010000000000011111100000000, 0b0000000000000000000011111111101111111111111111101101011101000000, + 0b0000000000000000000000000011111111111111111111110000000000000000, 0b0000000000000000000000000000000001111111011111111111111111111111, 0b0000000000000000000000000000000000000000001101111111011111111111, 0b0000000000000000000000000000000000000000000000000101010101111010, @@ -534,12 +538,12 @@ pub mod uppercase { 0b1100000000001111001111010101000000111110001001110011100010000100, 0b1100000000100101111010101001110100000000000000000000000000000000, 0b1110011010010000010101010101010101010101000111001000000000000000, - 0b1110011111111111111111111111111111111111111111110000000000000000, + 0b1110011111111111111111111111111111111111111111110000001000000000, 0b1111000000000000000000000000001111111111111111111111111100000000, 0b1111011111111111000000000000000000000000000000000000000000000000, 0b1111111100000000111111110000000000111111000000001111111100000000, ]; - const BITSET_MAPPING: &'static [(u8, u8); 25] = &[ + static BITSET_MAPPING: [(u8, u8); 25] = [ (0, 187), (0, 177), (0, 171), (0, 167), (0, 164), (0, 32), (0, 47), (0, 51), (0, 121), (0, 117), (0, 109), (1, 150), (1, 148), (1, 142), (1, 134), (1, 131), (1, 64), (2, 164), (2, 146), (2, 20), (3, 146), (3, 140), (3, 134), (4, 178), (4, 171), @@ -751,216 +755,223 @@ pub mod conversions { ('\u{13ea}', 43962), ('\u{13eb}', 43963), ('\u{13ec}', 43964), ('\u{13ed}', 43965), ('\u{13ee}', 43966), ('\u{13ef}', 43967), ('\u{13f0}', 5112), ('\u{13f1}', 5113), ('\u{13f2}', 5114), ('\u{13f3}', 5115), ('\u{13f4}', 5116), ('\u{13f5}', 5117), - ('\u{1c90}', 4304), ('\u{1c91}', 4305), ('\u{1c92}', 4306), ('\u{1c93}', 4307), - ('\u{1c94}', 4308), ('\u{1c95}', 4309), ('\u{1c96}', 4310), ('\u{1c97}', 4311), - ('\u{1c98}', 4312), ('\u{1c99}', 4313), ('\u{1c9a}', 4314), ('\u{1c9b}', 4315), - ('\u{1c9c}', 4316), ('\u{1c9d}', 4317), ('\u{1c9e}', 4318), ('\u{1c9f}', 4319), - ('\u{1ca0}', 4320), ('\u{1ca1}', 4321), ('\u{1ca2}', 4322), ('\u{1ca3}', 4323), - ('\u{1ca4}', 4324), ('\u{1ca5}', 4325), ('\u{1ca6}', 4326), ('\u{1ca7}', 4327), - ('\u{1ca8}', 4328), ('\u{1ca9}', 4329), ('\u{1caa}', 4330), ('\u{1cab}', 4331), - ('\u{1cac}', 4332), ('\u{1cad}', 4333), ('\u{1cae}', 4334), ('\u{1caf}', 4335), - ('\u{1cb0}', 4336), ('\u{1cb1}', 4337), ('\u{1cb2}', 4338), ('\u{1cb3}', 4339), - ('\u{1cb4}', 4340), ('\u{1cb5}', 4341), ('\u{1cb6}', 4342), ('\u{1cb7}', 4343), - ('\u{1cb8}', 4344), ('\u{1cb9}', 4345), ('\u{1cba}', 4346), ('\u{1cbd}', 4349), - ('\u{1cbe}', 4350), ('\u{1cbf}', 4351), ('\u{1e00}', 7681), ('\u{1e02}', 7683), - ('\u{1e04}', 7685), ('\u{1e06}', 7687), ('\u{1e08}', 7689), ('\u{1e0a}', 7691), - ('\u{1e0c}', 7693), ('\u{1e0e}', 7695), ('\u{1e10}', 7697), ('\u{1e12}', 7699), - ('\u{1e14}', 7701), ('\u{1e16}', 7703), ('\u{1e18}', 7705), ('\u{1e1a}', 7707), - ('\u{1e1c}', 7709), ('\u{1e1e}', 7711), ('\u{1e20}', 7713), ('\u{1e22}', 7715), - ('\u{1e24}', 7717), ('\u{1e26}', 7719), ('\u{1e28}', 7721), ('\u{1e2a}', 7723), - ('\u{1e2c}', 7725), ('\u{1e2e}', 7727), ('\u{1e30}', 7729), ('\u{1e32}', 7731), - ('\u{1e34}', 7733), ('\u{1e36}', 7735), ('\u{1e38}', 7737), ('\u{1e3a}', 7739), - ('\u{1e3c}', 7741), ('\u{1e3e}', 7743), ('\u{1e40}', 7745), ('\u{1e42}', 7747), - ('\u{1e44}', 7749), ('\u{1e46}', 7751), ('\u{1e48}', 7753), ('\u{1e4a}', 7755), - ('\u{1e4c}', 7757), ('\u{1e4e}', 7759), ('\u{1e50}', 7761), ('\u{1e52}', 7763), - ('\u{1e54}', 7765), ('\u{1e56}', 7767), ('\u{1e58}', 7769), ('\u{1e5a}', 7771), - ('\u{1e5c}', 7773), ('\u{1e5e}', 7775), ('\u{1e60}', 7777), ('\u{1e62}', 7779), - ('\u{1e64}', 7781), ('\u{1e66}', 7783), ('\u{1e68}', 7785), ('\u{1e6a}', 7787), - ('\u{1e6c}', 7789), ('\u{1e6e}', 7791), ('\u{1e70}', 7793), ('\u{1e72}', 7795), - ('\u{1e74}', 7797), ('\u{1e76}', 7799), ('\u{1e78}', 7801), ('\u{1e7a}', 7803), - ('\u{1e7c}', 7805), ('\u{1e7e}', 7807), ('\u{1e80}', 7809), ('\u{1e82}', 7811), - ('\u{1e84}', 7813), ('\u{1e86}', 7815), ('\u{1e88}', 7817), ('\u{1e8a}', 7819), - ('\u{1e8c}', 7821), ('\u{1e8e}', 7823), ('\u{1e90}', 7825), ('\u{1e92}', 7827), - ('\u{1e94}', 7829), ('\u{1e9e}', 223), ('\u{1ea0}', 7841), ('\u{1ea2}', 7843), - ('\u{1ea4}', 7845), ('\u{1ea6}', 7847), ('\u{1ea8}', 7849), ('\u{1eaa}', 7851), - ('\u{1eac}', 7853), ('\u{1eae}', 7855), ('\u{1eb0}', 7857), ('\u{1eb2}', 7859), - ('\u{1eb4}', 7861), ('\u{1eb6}', 7863), ('\u{1eb8}', 7865), ('\u{1eba}', 7867), - ('\u{1ebc}', 7869), ('\u{1ebe}', 7871), ('\u{1ec0}', 7873), ('\u{1ec2}', 7875), - ('\u{1ec4}', 7877), ('\u{1ec6}', 7879), ('\u{1ec8}', 7881), ('\u{1eca}', 7883), - ('\u{1ecc}', 7885), ('\u{1ece}', 7887), ('\u{1ed0}', 7889), ('\u{1ed2}', 7891), - ('\u{1ed4}', 7893), ('\u{1ed6}', 7895), ('\u{1ed8}', 7897), ('\u{1eda}', 7899), - ('\u{1edc}', 7901), ('\u{1ede}', 7903), ('\u{1ee0}', 7905), ('\u{1ee2}', 7907), - ('\u{1ee4}', 7909), ('\u{1ee6}', 7911), ('\u{1ee8}', 7913), ('\u{1eea}', 7915), - ('\u{1eec}', 7917), ('\u{1eee}', 7919), ('\u{1ef0}', 7921), ('\u{1ef2}', 7923), - ('\u{1ef4}', 7925), ('\u{1ef6}', 7927), ('\u{1ef8}', 7929), ('\u{1efa}', 7931), - ('\u{1efc}', 7933), ('\u{1efe}', 7935), ('\u{1f08}', 7936), ('\u{1f09}', 7937), - ('\u{1f0a}', 7938), ('\u{1f0b}', 7939), ('\u{1f0c}', 7940), ('\u{1f0d}', 7941), - ('\u{1f0e}', 7942), ('\u{1f0f}', 7943), ('\u{1f18}', 7952), ('\u{1f19}', 7953), - ('\u{1f1a}', 7954), ('\u{1f1b}', 7955), ('\u{1f1c}', 7956), ('\u{1f1d}', 7957), - ('\u{1f28}', 7968), ('\u{1f29}', 7969), ('\u{1f2a}', 7970), ('\u{1f2b}', 7971), - ('\u{1f2c}', 7972), ('\u{1f2d}', 7973), ('\u{1f2e}', 7974), ('\u{1f2f}', 7975), - ('\u{1f38}', 7984), ('\u{1f39}', 7985), ('\u{1f3a}', 7986), ('\u{1f3b}', 7987), - ('\u{1f3c}', 7988), ('\u{1f3d}', 7989), ('\u{1f3e}', 7990), ('\u{1f3f}', 7991), - ('\u{1f48}', 8000), ('\u{1f49}', 8001), ('\u{1f4a}', 8002), ('\u{1f4b}', 8003), - ('\u{1f4c}', 8004), ('\u{1f4d}', 8005), ('\u{1f59}', 8017), ('\u{1f5b}', 8019), - ('\u{1f5d}', 8021), ('\u{1f5f}', 8023), ('\u{1f68}', 8032), ('\u{1f69}', 8033), - ('\u{1f6a}', 8034), ('\u{1f6b}', 8035), ('\u{1f6c}', 8036), ('\u{1f6d}', 8037), - ('\u{1f6e}', 8038), ('\u{1f6f}', 8039), ('\u{1f88}', 8064), ('\u{1f89}', 8065), - ('\u{1f8a}', 8066), ('\u{1f8b}', 8067), ('\u{1f8c}', 8068), ('\u{1f8d}', 8069), - ('\u{1f8e}', 8070), ('\u{1f8f}', 8071), ('\u{1f98}', 8080), ('\u{1f99}', 8081), - ('\u{1f9a}', 8082), ('\u{1f9b}', 8083), ('\u{1f9c}', 8084), ('\u{1f9d}', 8085), - ('\u{1f9e}', 8086), ('\u{1f9f}', 8087), ('\u{1fa8}', 8096), ('\u{1fa9}', 8097), - ('\u{1faa}', 8098), ('\u{1fab}', 8099), ('\u{1fac}', 8100), ('\u{1fad}', 8101), - ('\u{1fae}', 8102), ('\u{1faf}', 8103), ('\u{1fb8}', 8112), ('\u{1fb9}', 8113), - ('\u{1fba}', 8048), ('\u{1fbb}', 8049), ('\u{1fbc}', 8115), ('\u{1fc8}', 8050), - ('\u{1fc9}', 8051), ('\u{1fca}', 8052), ('\u{1fcb}', 8053), ('\u{1fcc}', 8131), - ('\u{1fd8}', 8144), ('\u{1fd9}', 8145), ('\u{1fda}', 8054), ('\u{1fdb}', 8055), - ('\u{1fe8}', 8160), ('\u{1fe9}', 8161), ('\u{1fea}', 8058), ('\u{1feb}', 8059), - ('\u{1fec}', 8165), ('\u{1ff8}', 8056), ('\u{1ff9}', 8057), ('\u{1ffa}', 8060), - ('\u{1ffb}', 8061), ('\u{1ffc}', 8179), ('\u{2126}', 969), ('\u{212a}', 107), - ('\u{212b}', 229), ('\u{2132}', 8526), ('\u{2160}', 8560), ('\u{2161}', 8561), - ('\u{2162}', 8562), ('\u{2163}', 8563), ('\u{2164}', 8564), ('\u{2165}', 8565), - ('\u{2166}', 8566), ('\u{2167}', 8567), ('\u{2168}', 8568), ('\u{2169}', 8569), - ('\u{216a}', 8570), ('\u{216b}', 8571), ('\u{216c}', 8572), ('\u{216d}', 8573), - ('\u{216e}', 8574), ('\u{216f}', 8575), ('\u{2183}', 8580), ('\u{24b6}', 9424), - ('\u{24b7}', 9425), ('\u{24b8}', 9426), ('\u{24b9}', 9427), ('\u{24ba}', 9428), - ('\u{24bb}', 9429), ('\u{24bc}', 9430), ('\u{24bd}', 9431), ('\u{24be}', 9432), - ('\u{24bf}', 9433), ('\u{24c0}', 9434), ('\u{24c1}', 9435), ('\u{24c2}', 9436), - ('\u{24c3}', 9437), ('\u{24c4}', 9438), ('\u{24c5}', 9439), ('\u{24c6}', 9440), - ('\u{24c7}', 9441), ('\u{24c8}', 9442), ('\u{24c9}', 9443), ('\u{24ca}', 9444), - ('\u{24cb}', 9445), ('\u{24cc}', 9446), ('\u{24cd}', 9447), ('\u{24ce}', 9448), - ('\u{24cf}', 9449), ('\u{2c00}', 11312), ('\u{2c01}', 11313), ('\u{2c02}', 11314), - ('\u{2c03}', 11315), ('\u{2c04}', 11316), ('\u{2c05}', 11317), ('\u{2c06}', 11318), - ('\u{2c07}', 11319), ('\u{2c08}', 11320), ('\u{2c09}', 11321), ('\u{2c0a}', 11322), - ('\u{2c0b}', 11323), ('\u{2c0c}', 11324), ('\u{2c0d}', 11325), ('\u{2c0e}', 11326), - ('\u{2c0f}', 11327), ('\u{2c10}', 11328), ('\u{2c11}', 11329), ('\u{2c12}', 11330), - ('\u{2c13}', 11331), ('\u{2c14}', 11332), ('\u{2c15}', 11333), ('\u{2c16}', 11334), - ('\u{2c17}', 11335), ('\u{2c18}', 11336), ('\u{2c19}', 11337), ('\u{2c1a}', 11338), - ('\u{2c1b}', 11339), ('\u{2c1c}', 11340), ('\u{2c1d}', 11341), ('\u{2c1e}', 11342), - ('\u{2c1f}', 11343), ('\u{2c20}', 11344), ('\u{2c21}', 11345), ('\u{2c22}', 11346), - ('\u{2c23}', 11347), ('\u{2c24}', 11348), ('\u{2c25}', 11349), ('\u{2c26}', 11350), - ('\u{2c27}', 11351), ('\u{2c28}', 11352), ('\u{2c29}', 11353), ('\u{2c2a}', 11354), - ('\u{2c2b}', 11355), ('\u{2c2c}', 11356), ('\u{2c2d}', 11357), ('\u{2c2e}', 11358), - ('\u{2c2f}', 11359), ('\u{2c60}', 11361), ('\u{2c62}', 619), ('\u{2c63}', 7549), - ('\u{2c64}', 637), ('\u{2c67}', 11368), ('\u{2c69}', 11370), ('\u{2c6b}', 11372), - ('\u{2c6d}', 593), ('\u{2c6e}', 625), ('\u{2c6f}', 592), ('\u{2c70}', 594), - ('\u{2c72}', 11379), ('\u{2c75}', 11382), ('\u{2c7e}', 575), ('\u{2c7f}', 576), - ('\u{2c80}', 11393), ('\u{2c82}', 11395), ('\u{2c84}', 11397), ('\u{2c86}', 11399), - ('\u{2c88}', 11401), ('\u{2c8a}', 11403), ('\u{2c8c}', 11405), ('\u{2c8e}', 11407), - ('\u{2c90}', 11409), ('\u{2c92}', 11411), ('\u{2c94}', 11413), ('\u{2c96}', 11415), - ('\u{2c98}', 11417), ('\u{2c9a}', 11419), ('\u{2c9c}', 11421), ('\u{2c9e}', 11423), - ('\u{2ca0}', 11425), ('\u{2ca2}', 11427), ('\u{2ca4}', 11429), ('\u{2ca6}', 11431), - ('\u{2ca8}', 11433), ('\u{2caa}', 11435), ('\u{2cac}', 11437), ('\u{2cae}', 11439), - ('\u{2cb0}', 11441), ('\u{2cb2}', 11443), ('\u{2cb4}', 11445), ('\u{2cb6}', 11447), - ('\u{2cb8}', 11449), ('\u{2cba}', 11451), ('\u{2cbc}', 11453), ('\u{2cbe}', 11455), - ('\u{2cc0}', 11457), ('\u{2cc2}', 11459), ('\u{2cc4}', 11461), ('\u{2cc6}', 11463), - ('\u{2cc8}', 11465), ('\u{2cca}', 11467), ('\u{2ccc}', 11469), ('\u{2cce}', 11471), - ('\u{2cd0}', 11473), ('\u{2cd2}', 11475), ('\u{2cd4}', 11477), ('\u{2cd6}', 11479), - ('\u{2cd8}', 11481), ('\u{2cda}', 11483), ('\u{2cdc}', 11485), ('\u{2cde}', 11487), - ('\u{2ce0}', 11489), ('\u{2ce2}', 11491), ('\u{2ceb}', 11500), ('\u{2ced}', 11502), - ('\u{2cf2}', 11507), ('\u{a640}', 42561), ('\u{a642}', 42563), ('\u{a644}', 42565), - ('\u{a646}', 42567), ('\u{a648}', 42569), ('\u{a64a}', 42571), ('\u{a64c}', 42573), - ('\u{a64e}', 42575), ('\u{a650}', 42577), ('\u{a652}', 42579), ('\u{a654}', 42581), - ('\u{a656}', 42583), ('\u{a658}', 42585), ('\u{a65a}', 42587), ('\u{a65c}', 42589), - ('\u{a65e}', 42591), ('\u{a660}', 42593), ('\u{a662}', 42595), ('\u{a664}', 42597), - ('\u{a666}', 42599), ('\u{a668}', 42601), ('\u{a66a}', 42603), ('\u{a66c}', 42605), - ('\u{a680}', 42625), ('\u{a682}', 42627), ('\u{a684}', 42629), ('\u{a686}', 42631), - ('\u{a688}', 42633), ('\u{a68a}', 42635), ('\u{a68c}', 42637), ('\u{a68e}', 42639), - ('\u{a690}', 42641), ('\u{a692}', 42643), ('\u{a694}', 42645), ('\u{a696}', 42647), - ('\u{a698}', 42649), ('\u{a69a}', 42651), ('\u{a722}', 42787), ('\u{a724}', 42789), - ('\u{a726}', 42791), ('\u{a728}', 42793), ('\u{a72a}', 42795), ('\u{a72c}', 42797), - ('\u{a72e}', 42799), ('\u{a732}', 42803), ('\u{a734}', 42805), ('\u{a736}', 42807), - ('\u{a738}', 42809), ('\u{a73a}', 42811), ('\u{a73c}', 42813), ('\u{a73e}', 42815), - ('\u{a740}', 42817), ('\u{a742}', 42819), ('\u{a744}', 42821), ('\u{a746}', 42823), - ('\u{a748}', 42825), ('\u{a74a}', 42827), ('\u{a74c}', 42829), ('\u{a74e}', 42831), - ('\u{a750}', 42833), ('\u{a752}', 42835), ('\u{a754}', 42837), ('\u{a756}', 42839), - ('\u{a758}', 42841), ('\u{a75a}', 42843), ('\u{a75c}', 42845), ('\u{a75e}', 42847), - ('\u{a760}', 42849), ('\u{a762}', 42851), ('\u{a764}', 42853), ('\u{a766}', 42855), - ('\u{a768}', 42857), ('\u{a76a}', 42859), ('\u{a76c}', 42861), ('\u{a76e}', 42863), - ('\u{a779}', 42874), ('\u{a77b}', 42876), ('\u{a77d}', 7545), ('\u{a77e}', 42879), - ('\u{a780}', 42881), ('\u{a782}', 42883), ('\u{a784}', 42885), ('\u{a786}', 42887), - ('\u{a78b}', 42892), ('\u{a78d}', 613), ('\u{a790}', 42897), ('\u{a792}', 42899), - ('\u{a796}', 42903), ('\u{a798}', 42905), ('\u{a79a}', 42907), ('\u{a79c}', 42909), - ('\u{a79e}', 42911), ('\u{a7a0}', 42913), ('\u{a7a2}', 42915), ('\u{a7a4}', 42917), - ('\u{a7a6}', 42919), ('\u{a7a8}', 42921), ('\u{a7aa}', 614), ('\u{a7ab}', 604), - ('\u{a7ac}', 609), ('\u{a7ad}', 620), ('\u{a7ae}', 618), ('\u{a7b0}', 670), - ('\u{a7b1}', 647), ('\u{a7b2}', 669), ('\u{a7b3}', 43859), ('\u{a7b4}', 42933), - ('\u{a7b6}', 42935), ('\u{a7b8}', 42937), ('\u{a7ba}', 42939), ('\u{a7bc}', 42941), - ('\u{a7be}', 42943), ('\u{a7c0}', 42945), ('\u{a7c2}', 42947), ('\u{a7c4}', 42900), - ('\u{a7c5}', 642), ('\u{a7c6}', 7566), ('\u{a7c7}', 42952), ('\u{a7c9}', 42954), - ('\u{a7d0}', 42961), ('\u{a7d6}', 42967), ('\u{a7d8}', 42969), ('\u{a7f5}', 42998), - ('\u{ff21}', 65345), ('\u{ff22}', 65346), ('\u{ff23}', 65347), ('\u{ff24}', 65348), - ('\u{ff25}', 65349), ('\u{ff26}', 65350), ('\u{ff27}', 65351), ('\u{ff28}', 65352), - ('\u{ff29}', 65353), ('\u{ff2a}', 65354), ('\u{ff2b}', 65355), ('\u{ff2c}', 65356), - ('\u{ff2d}', 65357), ('\u{ff2e}', 65358), ('\u{ff2f}', 65359), ('\u{ff30}', 65360), - ('\u{ff31}', 65361), ('\u{ff32}', 65362), ('\u{ff33}', 65363), ('\u{ff34}', 65364), - ('\u{ff35}', 65365), ('\u{ff36}', 65366), ('\u{ff37}', 65367), ('\u{ff38}', 65368), - ('\u{ff39}', 65369), ('\u{ff3a}', 65370), ('\u{10400}', 66600), ('\u{10401}', 66601), - ('\u{10402}', 66602), ('\u{10403}', 66603), ('\u{10404}', 66604), ('\u{10405}', 66605), - ('\u{10406}', 66606), ('\u{10407}', 66607), ('\u{10408}', 66608), ('\u{10409}', 66609), - ('\u{1040a}', 66610), ('\u{1040b}', 66611), ('\u{1040c}', 66612), ('\u{1040d}', 66613), - ('\u{1040e}', 66614), ('\u{1040f}', 66615), ('\u{10410}', 66616), ('\u{10411}', 66617), - ('\u{10412}', 66618), ('\u{10413}', 66619), ('\u{10414}', 66620), ('\u{10415}', 66621), - ('\u{10416}', 66622), ('\u{10417}', 66623), ('\u{10418}', 66624), ('\u{10419}', 66625), - ('\u{1041a}', 66626), ('\u{1041b}', 66627), ('\u{1041c}', 66628), ('\u{1041d}', 66629), - ('\u{1041e}', 66630), ('\u{1041f}', 66631), ('\u{10420}', 66632), ('\u{10421}', 66633), - ('\u{10422}', 66634), ('\u{10423}', 66635), ('\u{10424}', 66636), ('\u{10425}', 66637), - ('\u{10426}', 66638), ('\u{10427}', 66639), ('\u{104b0}', 66776), ('\u{104b1}', 66777), - ('\u{104b2}', 66778), ('\u{104b3}', 66779), ('\u{104b4}', 66780), ('\u{104b5}', 66781), - ('\u{104b6}', 66782), ('\u{104b7}', 66783), ('\u{104b8}', 66784), ('\u{104b9}', 66785), - ('\u{104ba}', 66786), ('\u{104bb}', 66787), ('\u{104bc}', 66788), ('\u{104bd}', 66789), - ('\u{104be}', 66790), ('\u{104bf}', 66791), ('\u{104c0}', 66792), ('\u{104c1}', 66793), - ('\u{104c2}', 66794), ('\u{104c3}', 66795), ('\u{104c4}', 66796), ('\u{104c5}', 66797), - ('\u{104c6}', 66798), ('\u{104c7}', 66799), ('\u{104c8}', 66800), ('\u{104c9}', 66801), - ('\u{104ca}', 66802), ('\u{104cb}', 66803), ('\u{104cc}', 66804), ('\u{104cd}', 66805), - ('\u{104ce}', 66806), ('\u{104cf}', 66807), ('\u{104d0}', 66808), ('\u{104d1}', 66809), - ('\u{104d2}', 66810), ('\u{104d3}', 66811), ('\u{10570}', 66967), ('\u{10571}', 66968), - ('\u{10572}', 66969), ('\u{10573}', 66970), ('\u{10574}', 66971), ('\u{10575}', 66972), - ('\u{10576}', 66973), ('\u{10577}', 66974), ('\u{10578}', 66975), ('\u{10579}', 66976), - ('\u{1057a}', 66977), ('\u{1057c}', 66979), ('\u{1057d}', 66980), ('\u{1057e}', 66981), - ('\u{1057f}', 66982), ('\u{10580}', 66983), ('\u{10581}', 66984), ('\u{10582}', 66985), - ('\u{10583}', 66986), ('\u{10584}', 66987), ('\u{10585}', 66988), ('\u{10586}', 66989), - ('\u{10587}', 66990), ('\u{10588}', 66991), ('\u{10589}', 66992), ('\u{1058a}', 66993), - ('\u{1058c}', 66995), ('\u{1058d}', 66996), ('\u{1058e}', 66997), ('\u{1058f}', 66998), - ('\u{10590}', 66999), ('\u{10591}', 67000), ('\u{10592}', 67001), ('\u{10594}', 67003), - ('\u{10595}', 67004), ('\u{10c80}', 68800), ('\u{10c81}', 68801), ('\u{10c82}', 68802), - ('\u{10c83}', 68803), ('\u{10c84}', 68804), ('\u{10c85}', 68805), ('\u{10c86}', 68806), - ('\u{10c87}', 68807), ('\u{10c88}', 68808), ('\u{10c89}', 68809), ('\u{10c8a}', 68810), - ('\u{10c8b}', 68811), ('\u{10c8c}', 68812), ('\u{10c8d}', 68813), ('\u{10c8e}', 68814), - ('\u{10c8f}', 68815), ('\u{10c90}', 68816), ('\u{10c91}', 68817), ('\u{10c92}', 68818), - ('\u{10c93}', 68819), ('\u{10c94}', 68820), ('\u{10c95}', 68821), ('\u{10c96}', 68822), - ('\u{10c97}', 68823), ('\u{10c98}', 68824), ('\u{10c99}', 68825), ('\u{10c9a}', 68826), - ('\u{10c9b}', 68827), ('\u{10c9c}', 68828), ('\u{10c9d}', 68829), ('\u{10c9e}', 68830), - ('\u{10c9f}', 68831), ('\u{10ca0}', 68832), ('\u{10ca1}', 68833), ('\u{10ca2}', 68834), - ('\u{10ca3}', 68835), ('\u{10ca4}', 68836), ('\u{10ca5}', 68837), ('\u{10ca6}', 68838), - ('\u{10ca7}', 68839), ('\u{10ca8}', 68840), ('\u{10ca9}', 68841), ('\u{10caa}', 68842), - ('\u{10cab}', 68843), ('\u{10cac}', 68844), ('\u{10cad}', 68845), ('\u{10cae}', 68846), - ('\u{10caf}', 68847), ('\u{10cb0}', 68848), ('\u{10cb1}', 68849), ('\u{10cb2}', 68850), - ('\u{118a0}', 71872), ('\u{118a1}', 71873), ('\u{118a2}', 71874), ('\u{118a3}', 71875), - ('\u{118a4}', 71876), ('\u{118a5}', 71877), ('\u{118a6}', 71878), ('\u{118a7}', 71879), - ('\u{118a8}', 71880), ('\u{118a9}', 71881), ('\u{118aa}', 71882), ('\u{118ab}', 71883), - ('\u{118ac}', 71884), ('\u{118ad}', 71885), ('\u{118ae}', 71886), ('\u{118af}', 71887), - ('\u{118b0}', 71888), ('\u{118b1}', 71889), ('\u{118b2}', 71890), ('\u{118b3}', 71891), - ('\u{118b4}', 71892), ('\u{118b5}', 71893), ('\u{118b6}', 71894), ('\u{118b7}', 71895), - ('\u{118b8}', 71896), ('\u{118b9}', 71897), ('\u{118ba}', 71898), ('\u{118bb}', 71899), - ('\u{118bc}', 71900), ('\u{118bd}', 71901), ('\u{118be}', 71902), ('\u{118bf}', 71903), - ('\u{16e40}', 93792), ('\u{16e41}', 93793), ('\u{16e42}', 93794), ('\u{16e43}', 93795), - ('\u{16e44}', 93796), ('\u{16e45}', 93797), ('\u{16e46}', 93798), ('\u{16e47}', 93799), - ('\u{16e48}', 93800), ('\u{16e49}', 93801), ('\u{16e4a}', 93802), ('\u{16e4b}', 93803), - ('\u{16e4c}', 93804), ('\u{16e4d}', 93805), ('\u{16e4e}', 93806), ('\u{16e4f}', 93807), - ('\u{16e50}', 93808), ('\u{16e51}', 93809), ('\u{16e52}', 93810), ('\u{16e53}', 93811), - ('\u{16e54}', 93812), ('\u{16e55}', 93813), ('\u{16e56}', 93814), ('\u{16e57}', 93815), - ('\u{16e58}', 93816), ('\u{16e59}', 93817), ('\u{16e5a}', 93818), ('\u{16e5b}', 93819), - ('\u{16e5c}', 93820), ('\u{16e5d}', 93821), ('\u{16e5e}', 93822), ('\u{16e5f}', 93823), - ('\u{1e900}', 125218), ('\u{1e901}', 125219), ('\u{1e902}', 125220), ('\u{1e903}', 125221), - ('\u{1e904}', 125222), ('\u{1e905}', 125223), ('\u{1e906}', 125224), ('\u{1e907}', 125225), - ('\u{1e908}', 125226), ('\u{1e909}', 125227), ('\u{1e90a}', 125228), ('\u{1e90b}', 125229), - ('\u{1e90c}', 125230), ('\u{1e90d}', 125231), ('\u{1e90e}', 125232), ('\u{1e90f}', 125233), - ('\u{1e910}', 125234), ('\u{1e911}', 125235), ('\u{1e912}', 125236), ('\u{1e913}', 125237), - ('\u{1e914}', 125238), ('\u{1e915}', 125239), ('\u{1e916}', 125240), ('\u{1e917}', 125241), - ('\u{1e918}', 125242), ('\u{1e919}', 125243), ('\u{1e91a}', 125244), ('\u{1e91b}', 125245), - ('\u{1e91c}', 125246), ('\u{1e91d}', 125247), ('\u{1e91e}', 125248), ('\u{1e91f}', 125249), - ('\u{1e920}', 125250), ('\u{1e921}', 125251), + ('\u{1c89}', 7306), ('\u{1c90}', 4304), ('\u{1c91}', 4305), ('\u{1c92}', 4306), + ('\u{1c93}', 4307), ('\u{1c94}', 4308), ('\u{1c95}', 4309), ('\u{1c96}', 4310), + ('\u{1c97}', 4311), ('\u{1c98}', 4312), ('\u{1c99}', 4313), ('\u{1c9a}', 4314), + ('\u{1c9b}', 4315), ('\u{1c9c}', 4316), ('\u{1c9d}', 4317), ('\u{1c9e}', 4318), + ('\u{1c9f}', 4319), ('\u{1ca0}', 4320), ('\u{1ca1}', 4321), ('\u{1ca2}', 4322), + ('\u{1ca3}', 4323), ('\u{1ca4}', 4324), ('\u{1ca5}', 4325), ('\u{1ca6}', 4326), + ('\u{1ca7}', 4327), ('\u{1ca8}', 4328), ('\u{1ca9}', 4329), ('\u{1caa}', 4330), + ('\u{1cab}', 4331), ('\u{1cac}', 4332), ('\u{1cad}', 4333), ('\u{1cae}', 4334), + ('\u{1caf}', 4335), ('\u{1cb0}', 4336), ('\u{1cb1}', 4337), ('\u{1cb2}', 4338), + ('\u{1cb3}', 4339), ('\u{1cb4}', 4340), ('\u{1cb5}', 4341), ('\u{1cb6}', 4342), + ('\u{1cb7}', 4343), ('\u{1cb8}', 4344), ('\u{1cb9}', 4345), ('\u{1cba}', 4346), + ('\u{1cbd}', 4349), ('\u{1cbe}', 4350), ('\u{1cbf}', 4351), ('\u{1e00}', 7681), + ('\u{1e02}', 7683), ('\u{1e04}', 7685), ('\u{1e06}', 7687), ('\u{1e08}', 7689), + ('\u{1e0a}', 7691), ('\u{1e0c}', 7693), ('\u{1e0e}', 7695), ('\u{1e10}', 7697), + ('\u{1e12}', 7699), ('\u{1e14}', 7701), ('\u{1e16}', 7703), ('\u{1e18}', 7705), + ('\u{1e1a}', 7707), ('\u{1e1c}', 7709), ('\u{1e1e}', 7711), ('\u{1e20}', 7713), + ('\u{1e22}', 7715), ('\u{1e24}', 7717), ('\u{1e26}', 7719), ('\u{1e28}', 7721), + ('\u{1e2a}', 7723), ('\u{1e2c}', 7725), ('\u{1e2e}', 7727), ('\u{1e30}', 7729), + ('\u{1e32}', 7731), ('\u{1e34}', 7733), ('\u{1e36}', 7735), ('\u{1e38}', 7737), + ('\u{1e3a}', 7739), ('\u{1e3c}', 7741), ('\u{1e3e}', 7743), ('\u{1e40}', 7745), + ('\u{1e42}', 7747), ('\u{1e44}', 7749), ('\u{1e46}', 7751), ('\u{1e48}', 7753), + ('\u{1e4a}', 7755), ('\u{1e4c}', 7757), ('\u{1e4e}', 7759), ('\u{1e50}', 7761), + ('\u{1e52}', 7763), ('\u{1e54}', 7765), ('\u{1e56}', 7767), ('\u{1e58}', 7769), + ('\u{1e5a}', 7771), ('\u{1e5c}', 7773), ('\u{1e5e}', 7775), ('\u{1e60}', 7777), + ('\u{1e62}', 7779), ('\u{1e64}', 7781), ('\u{1e66}', 7783), ('\u{1e68}', 7785), + ('\u{1e6a}', 7787), ('\u{1e6c}', 7789), ('\u{1e6e}', 7791), ('\u{1e70}', 7793), + ('\u{1e72}', 7795), ('\u{1e74}', 7797), ('\u{1e76}', 7799), ('\u{1e78}', 7801), + ('\u{1e7a}', 7803), ('\u{1e7c}', 7805), ('\u{1e7e}', 7807), ('\u{1e80}', 7809), + ('\u{1e82}', 7811), ('\u{1e84}', 7813), ('\u{1e86}', 7815), ('\u{1e88}', 7817), + ('\u{1e8a}', 7819), ('\u{1e8c}', 7821), ('\u{1e8e}', 7823), ('\u{1e90}', 7825), + ('\u{1e92}', 7827), ('\u{1e94}', 7829), ('\u{1e9e}', 223), ('\u{1ea0}', 7841), + ('\u{1ea2}', 7843), ('\u{1ea4}', 7845), ('\u{1ea6}', 7847), ('\u{1ea8}', 7849), + ('\u{1eaa}', 7851), ('\u{1eac}', 7853), ('\u{1eae}', 7855), ('\u{1eb0}', 7857), + ('\u{1eb2}', 7859), ('\u{1eb4}', 7861), ('\u{1eb6}', 7863), ('\u{1eb8}', 7865), + ('\u{1eba}', 7867), ('\u{1ebc}', 7869), ('\u{1ebe}', 7871), ('\u{1ec0}', 7873), + ('\u{1ec2}', 7875), ('\u{1ec4}', 7877), ('\u{1ec6}', 7879), ('\u{1ec8}', 7881), + ('\u{1eca}', 7883), ('\u{1ecc}', 7885), ('\u{1ece}', 7887), ('\u{1ed0}', 7889), + ('\u{1ed2}', 7891), ('\u{1ed4}', 7893), ('\u{1ed6}', 7895), ('\u{1ed8}', 7897), + ('\u{1eda}', 7899), ('\u{1edc}', 7901), ('\u{1ede}', 7903), ('\u{1ee0}', 7905), + ('\u{1ee2}', 7907), ('\u{1ee4}', 7909), ('\u{1ee6}', 7911), ('\u{1ee8}', 7913), + ('\u{1eea}', 7915), ('\u{1eec}', 7917), ('\u{1eee}', 7919), ('\u{1ef0}', 7921), + ('\u{1ef2}', 7923), ('\u{1ef4}', 7925), ('\u{1ef6}', 7927), ('\u{1ef8}', 7929), + ('\u{1efa}', 7931), ('\u{1efc}', 7933), ('\u{1efe}', 7935), ('\u{1f08}', 7936), + ('\u{1f09}', 7937), ('\u{1f0a}', 7938), ('\u{1f0b}', 7939), ('\u{1f0c}', 7940), + ('\u{1f0d}', 7941), ('\u{1f0e}', 7942), ('\u{1f0f}', 7943), ('\u{1f18}', 7952), + ('\u{1f19}', 7953), ('\u{1f1a}', 7954), ('\u{1f1b}', 7955), ('\u{1f1c}', 7956), + ('\u{1f1d}', 7957), ('\u{1f28}', 7968), ('\u{1f29}', 7969), ('\u{1f2a}', 7970), + ('\u{1f2b}', 7971), ('\u{1f2c}', 7972), ('\u{1f2d}', 7973), ('\u{1f2e}', 7974), + ('\u{1f2f}', 7975), ('\u{1f38}', 7984), ('\u{1f39}', 7985), ('\u{1f3a}', 7986), + ('\u{1f3b}', 7987), ('\u{1f3c}', 7988), ('\u{1f3d}', 7989), ('\u{1f3e}', 7990), + ('\u{1f3f}', 7991), ('\u{1f48}', 8000), ('\u{1f49}', 8001), ('\u{1f4a}', 8002), + ('\u{1f4b}', 8003), ('\u{1f4c}', 8004), ('\u{1f4d}', 8005), ('\u{1f59}', 8017), + ('\u{1f5b}', 8019), ('\u{1f5d}', 8021), ('\u{1f5f}', 8023), ('\u{1f68}', 8032), + ('\u{1f69}', 8033), ('\u{1f6a}', 8034), ('\u{1f6b}', 8035), ('\u{1f6c}', 8036), + ('\u{1f6d}', 8037), ('\u{1f6e}', 8038), ('\u{1f6f}', 8039), ('\u{1f88}', 8064), + ('\u{1f89}', 8065), ('\u{1f8a}', 8066), ('\u{1f8b}', 8067), ('\u{1f8c}', 8068), + ('\u{1f8d}', 8069), ('\u{1f8e}', 8070), ('\u{1f8f}', 8071), ('\u{1f98}', 8080), + ('\u{1f99}', 8081), ('\u{1f9a}', 8082), ('\u{1f9b}', 8083), ('\u{1f9c}', 8084), + ('\u{1f9d}', 8085), ('\u{1f9e}', 8086), ('\u{1f9f}', 8087), ('\u{1fa8}', 8096), + ('\u{1fa9}', 8097), ('\u{1faa}', 8098), ('\u{1fab}', 8099), ('\u{1fac}', 8100), + ('\u{1fad}', 8101), ('\u{1fae}', 8102), ('\u{1faf}', 8103), ('\u{1fb8}', 8112), + ('\u{1fb9}', 8113), ('\u{1fba}', 8048), ('\u{1fbb}', 8049), ('\u{1fbc}', 8115), + ('\u{1fc8}', 8050), ('\u{1fc9}', 8051), ('\u{1fca}', 8052), ('\u{1fcb}', 8053), + ('\u{1fcc}', 8131), ('\u{1fd8}', 8144), ('\u{1fd9}', 8145), ('\u{1fda}', 8054), + ('\u{1fdb}', 8055), ('\u{1fe8}', 8160), ('\u{1fe9}', 8161), ('\u{1fea}', 8058), + ('\u{1feb}', 8059), ('\u{1fec}', 8165), ('\u{1ff8}', 8056), ('\u{1ff9}', 8057), + ('\u{1ffa}', 8060), ('\u{1ffb}', 8061), ('\u{1ffc}', 8179), ('\u{2126}', 969), + ('\u{212a}', 107), ('\u{212b}', 229), ('\u{2132}', 8526), ('\u{2160}', 8560), + ('\u{2161}', 8561), ('\u{2162}', 8562), ('\u{2163}', 8563), ('\u{2164}', 8564), + ('\u{2165}', 8565), ('\u{2166}', 8566), ('\u{2167}', 8567), ('\u{2168}', 8568), + ('\u{2169}', 8569), ('\u{216a}', 8570), ('\u{216b}', 8571), ('\u{216c}', 8572), + ('\u{216d}', 8573), ('\u{216e}', 8574), ('\u{216f}', 8575), ('\u{2183}', 8580), + ('\u{24b6}', 9424), ('\u{24b7}', 9425), ('\u{24b8}', 9426), ('\u{24b9}', 9427), + ('\u{24ba}', 9428), ('\u{24bb}', 9429), ('\u{24bc}', 9430), ('\u{24bd}', 9431), + ('\u{24be}', 9432), ('\u{24bf}', 9433), ('\u{24c0}', 9434), ('\u{24c1}', 9435), + ('\u{24c2}', 9436), ('\u{24c3}', 9437), ('\u{24c4}', 9438), ('\u{24c5}', 9439), + ('\u{24c6}', 9440), ('\u{24c7}', 9441), ('\u{24c8}', 9442), ('\u{24c9}', 9443), + ('\u{24ca}', 9444), ('\u{24cb}', 9445), ('\u{24cc}', 9446), ('\u{24cd}', 9447), + ('\u{24ce}', 9448), ('\u{24cf}', 9449), ('\u{2c00}', 11312), ('\u{2c01}', 11313), + ('\u{2c02}', 11314), ('\u{2c03}', 11315), ('\u{2c04}', 11316), ('\u{2c05}', 11317), + ('\u{2c06}', 11318), ('\u{2c07}', 11319), ('\u{2c08}', 11320), ('\u{2c09}', 11321), + ('\u{2c0a}', 11322), ('\u{2c0b}', 11323), ('\u{2c0c}', 11324), ('\u{2c0d}', 11325), + ('\u{2c0e}', 11326), ('\u{2c0f}', 11327), ('\u{2c10}', 11328), ('\u{2c11}', 11329), + ('\u{2c12}', 11330), ('\u{2c13}', 11331), ('\u{2c14}', 11332), ('\u{2c15}', 11333), + ('\u{2c16}', 11334), ('\u{2c17}', 11335), ('\u{2c18}', 11336), ('\u{2c19}', 11337), + ('\u{2c1a}', 11338), ('\u{2c1b}', 11339), ('\u{2c1c}', 11340), ('\u{2c1d}', 11341), + ('\u{2c1e}', 11342), ('\u{2c1f}', 11343), ('\u{2c20}', 11344), ('\u{2c21}', 11345), + ('\u{2c22}', 11346), ('\u{2c23}', 11347), ('\u{2c24}', 11348), ('\u{2c25}', 11349), + ('\u{2c26}', 11350), ('\u{2c27}', 11351), ('\u{2c28}', 11352), ('\u{2c29}', 11353), + ('\u{2c2a}', 11354), ('\u{2c2b}', 11355), ('\u{2c2c}', 11356), ('\u{2c2d}', 11357), + ('\u{2c2e}', 11358), ('\u{2c2f}', 11359), ('\u{2c60}', 11361), ('\u{2c62}', 619), + ('\u{2c63}', 7549), ('\u{2c64}', 637), ('\u{2c67}', 11368), ('\u{2c69}', 11370), + ('\u{2c6b}', 11372), ('\u{2c6d}', 593), ('\u{2c6e}', 625), ('\u{2c6f}', 592), + ('\u{2c70}', 594), ('\u{2c72}', 11379), ('\u{2c75}', 11382), ('\u{2c7e}', 575), + ('\u{2c7f}', 576), ('\u{2c80}', 11393), ('\u{2c82}', 11395), ('\u{2c84}', 11397), + ('\u{2c86}', 11399), ('\u{2c88}', 11401), ('\u{2c8a}', 11403), ('\u{2c8c}', 11405), + ('\u{2c8e}', 11407), ('\u{2c90}', 11409), ('\u{2c92}', 11411), ('\u{2c94}', 11413), + ('\u{2c96}', 11415), ('\u{2c98}', 11417), ('\u{2c9a}', 11419), ('\u{2c9c}', 11421), + ('\u{2c9e}', 11423), ('\u{2ca0}', 11425), ('\u{2ca2}', 11427), ('\u{2ca4}', 11429), + ('\u{2ca6}', 11431), ('\u{2ca8}', 11433), ('\u{2caa}', 11435), ('\u{2cac}', 11437), + ('\u{2cae}', 11439), ('\u{2cb0}', 11441), ('\u{2cb2}', 11443), ('\u{2cb4}', 11445), + ('\u{2cb6}', 11447), ('\u{2cb8}', 11449), ('\u{2cba}', 11451), ('\u{2cbc}', 11453), + ('\u{2cbe}', 11455), ('\u{2cc0}', 11457), ('\u{2cc2}', 11459), ('\u{2cc4}', 11461), + ('\u{2cc6}', 11463), ('\u{2cc8}', 11465), ('\u{2cca}', 11467), ('\u{2ccc}', 11469), + ('\u{2cce}', 11471), ('\u{2cd0}', 11473), ('\u{2cd2}', 11475), ('\u{2cd4}', 11477), + ('\u{2cd6}', 11479), ('\u{2cd8}', 11481), ('\u{2cda}', 11483), ('\u{2cdc}', 11485), + ('\u{2cde}', 11487), ('\u{2ce0}', 11489), ('\u{2ce2}', 11491), ('\u{2ceb}', 11500), + ('\u{2ced}', 11502), ('\u{2cf2}', 11507), ('\u{a640}', 42561), ('\u{a642}', 42563), + ('\u{a644}', 42565), ('\u{a646}', 42567), ('\u{a648}', 42569), ('\u{a64a}', 42571), + ('\u{a64c}', 42573), ('\u{a64e}', 42575), ('\u{a650}', 42577), ('\u{a652}', 42579), + ('\u{a654}', 42581), ('\u{a656}', 42583), ('\u{a658}', 42585), ('\u{a65a}', 42587), + ('\u{a65c}', 42589), ('\u{a65e}', 42591), ('\u{a660}', 42593), ('\u{a662}', 42595), + ('\u{a664}', 42597), ('\u{a666}', 42599), ('\u{a668}', 42601), ('\u{a66a}', 42603), + ('\u{a66c}', 42605), ('\u{a680}', 42625), ('\u{a682}', 42627), ('\u{a684}', 42629), + ('\u{a686}', 42631), ('\u{a688}', 42633), ('\u{a68a}', 42635), ('\u{a68c}', 42637), + ('\u{a68e}', 42639), ('\u{a690}', 42641), ('\u{a692}', 42643), ('\u{a694}', 42645), + ('\u{a696}', 42647), ('\u{a698}', 42649), ('\u{a69a}', 42651), ('\u{a722}', 42787), + ('\u{a724}', 42789), ('\u{a726}', 42791), ('\u{a728}', 42793), ('\u{a72a}', 42795), + ('\u{a72c}', 42797), ('\u{a72e}', 42799), ('\u{a732}', 42803), ('\u{a734}', 42805), + ('\u{a736}', 42807), ('\u{a738}', 42809), ('\u{a73a}', 42811), ('\u{a73c}', 42813), + ('\u{a73e}', 42815), ('\u{a740}', 42817), ('\u{a742}', 42819), ('\u{a744}', 42821), + ('\u{a746}', 42823), ('\u{a748}', 42825), ('\u{a74a}', 42827), ('\u{a74c}', 42829), + ('\u{a74e}', 42831), ('\u{a750}', 42833), ('\u{a752}', 42835), ('\u{a754}', 42837), + ('\u{a756}', 42839), ('\u{a758}', 42841), ('\u{a75a}', 42843), ('\u{a75c}', 42845), + ('\u{a75e}', 42847), ('\u{a760}', 42849), ('\u{a762}', 42851), ('\u{a764}', 42853), + ('\u{a766}', 42855), ('\u{a768}', 42857), ('\u{a76a}', 42859), ('\u{a76c}', 42861), + ('\u{a76e}', 42863), ('\u{a779}', 42874), ('\u{a77b}', 42876), ('\u{a77d}', 7545), + ('\u{a77e}', 42879), ('\u{a780}', 42881), ('\u{a782}', 42883), ('\u{a784}', 42885), + ('\u{a786}', 42887), ('\u{a78b}', 42892), ('\u{a78d}', 613), ('\u{a790}', 42897), + ('\u{a792}', 42899), ('\u{a796}', 42903), ('\u{a798}', 42905), ('\u{a79a}', 42907), + ('\u{a79c}', 42909), ('\u{a79e}', 42911), ('\u{a7a0}', 42913), ('\u{a7a2}', 42915), + ('\u{a7a4}', 42917), ('\u{a7a6}', 42919), ('\u{a7a8}', 42921), ('\u{a7aa}', 614), + ('\u{a7ab}', 604), ('\u{a7ac}', 609), ('\u{a7ad}', 620), ('\u{a7ae}', 618), + ('\u{a7b0}', 670), ('\u{a7b1}', 647), ('\u{a7b2}', 669), ('\u{a7b3}', 43859), + ('\u{a7b4}', 42933), ('\u{a7b6}', 42935), ('\u{a7b8}', 42937), ('\u{a7ba}', 42939), + ('\u{a7bc}', 42941), ('\u{a7be}', 42943), ('\u{a7c0}', 42945), ('\u{a7c2}', 42947), + ('\u{a7c4}', 42900), ('\u{a7c5}', 642), ('\u{a7c6}', 7566), ('\u{a7c7}', 42952), + ('\u{a7c9}', 42954), ('\u{a7cb}', 612), ('\u{a7cc}', 42957), ('\u{a7d0}', 42961), + ('\u{a7d6}', 42967), ('\u{a7d8}', 42969), ('\u{a7da}', 42971), ('\u{a7dc}', 411), + ('\u{a7f5}', 42998), ('\u{ff21}', 65345), ('\u{ff22}', 65346), ('\u{ff23}', 65347), + ('\u{ff24}', 65348), ('\u{ff25}', 65349), ('\u{ff26}', 65350), ('\u{ff27}', 65351), + ('\u{ff28}', 65352), ('\u{ff29}', 65353), ('\u{ff2a}', 65354), ('\u{ff2b}', 65355), + ('\u{ff2c}', 65356), ('\u{ff2d}', 65357), ('\u{ff2e}', 65358), ('\u{ff2f}', 65359), + ('\u{ff30}', 65360), ('\u{ff31}', 65361), ('\u{ff32}', 65362), ('\u{ff33}', 65363), + ('\u{ff34}', 65364), ('\u{ff35}', 65365), ('\u{ff36}', 65366), ('\u{ff37}', 65367), + ('\u{ff38}', 65368), ('\u{ff39}', 65369), ('\u{ff3a}', 65370), ('\u{10400}', 66600), + ('\u{10401}', 66601), ('\u{10402}', 66602), ('\u{10403}', 66603), ('\u{10404}', 66604), + ('\u{10405}', 66605), ('\u{10406}', 66606), ('\u{10407}', 66607), ('\u{10408}', 66608), + ('\u{10409}', 66609), ('\u{1040a}', 66610), ('\u{1040b}', 66611), ('\u{1040c}', 66612), + ('\u{1040d}', 66613), ('\u{1040e}', 66614), ('\u{1040f}', 66615), ('\u{10410}', 66616), + ('\u{10411}', 66617), ('\u{10412}', 66618), ('\u{10413}', 66619), ('\u{10414}', 66620), + ('\u{10415}', 66621), ('\u{10416}', 66622), ('\u{10417}', 66623), ('\u{10418}', 66624), + ('\u{10419}', 66625), ('\u{1041a}', 66626), ('\u{1041b}', 66627), ('\u{1041c}', 66628), + ('\u{1041d}', 66629), ('\u{1041e}', 66630), ('\u{1041f}', 66631), ('\u{10420}', 66632), + ('\u{10421}', 66633), ('\u{10422}', 66634), ('\u{10423}', 66635), ('\u{10424}', 66636), + ('\u{10425}', 66637), ('\u{10426}', 66638), ('\u{10427}', 66639), ('\u{104b0}', 66776), + ('\u{104b1}', 66777), ('\u{104b2}', 66778), ('\u{104b3}', 66779), ('\u{104b4}', 66780), + ('\u{104b5}', 66781), ('\u{104b6}', 66782), ('\u{104b7}', 66783), ('\u{104b8}', 66784), + ('\u{104b9}', 66785), ('\u{104ba}', 66786), ('\u{104bb}', 66787), ('\u{104bc}', 66788), + ('\u{104bd}', 66789), ('\u{104be}', 66790), ('\u{104bf}', 66791), ('\u{104c0}', 66792), + ('\u{104c1}', 66793), ('\u{104c2}', 66794), ('\u{104c3}', 66795), ('\u{104c4}', 66796), + ('\u{104c5}', 66797), ('\u{104c6}', 66798), ('\u{104c7}', 66799), ('\u{104c8}', 66800), + ('\u{104c9}', 66801), ('\u{104ca}', 66802), ('\u{104cb}', 66803), ('\u{104cc}', 66804), + ('\u{104cd}', 66805), ('\u{104ce}', 66806), ('\u{104cf}', 66807), ('\u{104d0}', 66808), + ('\u{104d1}', 66809), ('\u{104d2}', 66810), ('\u{104d3}', 66811), ('\u{10570}', 66967), + ('\u{10571}', 66968), ('\u{10572}', 66969), ('\u{10573}', 66970), ('\u{10574}', 66971), + ('\u{10575}', 66972), ('\u{10576}', 66973), ('\u{10577}', 66974), ('\u{10578}', 66975), + ('\u{10579}', 66976), ('\u{1057a}', 66977), ('\u{1057c}', 66979), ('\u{1057d}', 66980), + ('\u{1057e}', 66981), ('\u{1057f}', 66982), ('\u{10580}', 66983), ('\u{10581}', 66984), + ('\u{10582}', 66985), ('\u{10583}', 66986), ('\u{10584}', 66987), ('\u{10585}', 66988), + ('\u{10586}', 66989), ('\u{10587}', 66990), ('\u{10588}', 66991), ('\u{10589}', 66992), + ('\u{1058a}', 66993), ('\u{1058c}', 66995), ('\u{1058d}', 66996), ('\u{1058e}', 66997), + ('\u{1058f}', 66998), ('\u{10590}', 66999), ('\u{10591}', 67000), ('\u{10592}', 67001), + ('\u{10594}', 67003), ('\u{10595}', 67004), ('\u{10c80}', 68800), ('\u{10c81}', 68801), + ('\u{10c82}', 68802), ('\u{10c83}', 68803), ('\u{10c84}', 68804), ('\u{10c85}', 68805), + ('\u{10c86}', 68806), ('\u{10c87}', 68807), ('\u{10c88}', 68808), ('\u{10c89}', 68809), + ('\u{10c8a}', 68810), ('\u{10c8b}', 68811), ('\u{10c8c}', 68812), ('\u{10c8d}', 68813), + ('\u{10c8e}', 68814), ('\u{10c8f}', 68815), ('\u{10c90}', 68816), ('\u{10c91}', 68817), + ('\u{10c92}', 68818), ('\u{10c93}', 68819), ('\u{10c94}', 68820), ('\u{10c95}', 68821), + ('\u{10c96}', 68822), ('\u{10c97}', 68823), ('\u{10c98}', 68824), ('\u{10c99}', 68825), + ('\u{10c9a}', 68826), ('\u{10c9b}', 68827), ('\u{10c9c}', 68828), ('\u{10c9d}', 68829), + ('\u{10c9e}', 68830), ('\u{10c9f}', 68831), ('\u{10ca0}', 68832), ('\u{10ca1}', 68833), + ('\u{10ca2}', 68834), ('\u{10ca3}', 68835), ('\u{10ca4}', 68836), ('\u{10ca5}', 68837), + ('\u{10ca6}', 68838), ('\u{10ca7}', 68839), ('\u{10ca8}', 68840), ('\u{10ca9}', 68841), + ('\u{10caa}', 68842), ('\u{10cab}', 68843), ('\u{10cac}', 68844), ('\u{10cad}', 68845), + ('\u{10cae}', 68846), ('\u{10caf}', 68847), ('\u{10cb0}', 68848), ('\u{10cb1}', 68849), + ('\u{10cb2}', 68850), ('\u{10d50}', 68976), ('\u{10d51}', 68977), ('\u{10d52}', 68978), + ('\u{10d53}', 68979), ('\u{10d54}', 68980), ('\u{10d55}', 68981), ('\u{10d56}', 68982), + ('\u{10d57}', 68983), ('\u{10d58}', 68984), ('\u{10d59}', 68985), ('\u{10d5a}', 68986), + ('\u{10d5b}', 68987), ('\u{10d5c}', 68988), ('\u{10d5d}', 68989), ('\u{10d5e}', 68990), + ('\u{10d5f}', 68991), ('\u{10d60}', 68992), ('\u{10d61}', 68993), ('\u{10d62}', 68994), + ('\u{10d63}', 68995), ('\u{10d64}', 68996), ('\u{10d65}', 68997), ('\u{118a0}', 71872), + ('\u{118a1}', 71873), ('\u{118a2}', 71874), ('\u{118a3}', 71875), ('\u{118a4}', 71876), + ('\u{118a5}', 71877), ('\u{118a6}', 71878), ('\u{118a7}', 71879), ('\u{118a8}', 71880), + ('\u{118a9}', 71881), ('\u{118aa}', 71882), ('\u{118ab}', 71883), ('\u{118ac}', 71884), + ('\u{118ad}', 71885), ('\u{118ae}', 71886), ('\u{118af}', 71887), ('\u{118b0}', 71888), + ('\u{118b1}', 71889), ('\u{118b2}', 71890), ('\u{118b3}', 71891), ('\u{118b4}', 71892), + ('\u{118b5}', 71893), ('\u{118b6}', 71894), ('\u{118b7}', 71895), ('\u{118b8}', 71896), + ('\u{118b9}', 71897), ('\u{118ba}', 71898), ('\u{118bb}', 71899), ('\u{118bc}', 71900), + ('\u{118bd}', 71901), ('\u{118be}', 71902), ('\u{118bf}', 71903), ('\u{16e40}', 93792), + ('\u{16e41}', 93793), ('\u{16e42}', 93794), ('\u{16e43}', 93795), ('\u{16e44}', 93796), + ('\u{16e45}', 93797), ('\u{16e46}', 93798), ('\u{16e47}', 93799), ('\u{16e48}', 93800), + ('\u{16e49}', 93801), ('\u{16e4a}', 93802), ('\u{16e4b}', 93803), ('\u{16e4c}', 93804), + ('\u{16e4d}', 93805), ('\u{16e4e}', 93806), ('\u{16e4f}', 93807), ('\u{16e50}', 93808), + ('\u{16e51}', 93809), ('\u{16e52}', 93810), ('\u{16e53}', 93811), ('\u{16e54}', 93812), + ('\u{16e55}', 93813), ('\u{16e56}', 93814), ('\u{16e57}', 93815), ('\u{16e58}', 93816), + ('\u{16e59}', 93817), ('\u{16e5a}', 93818), ('\u{16e5b}', 93819), ('\u{16e5c}', 93820), + ('\u{16e5d}', 93821), ('\u{16e5e}', 93822), ('\u{16e5f}', 93823), ('\u{1e900}', 125218), + ('\u{1e901}', 125219), ('\u{1e902}', 125220), ('\u{1e903}', 125221), ('\u{1e904}', 125222), + ('\u{1e905}', 125223), ('\u{1e906}', 125224), ('\u{1e907}', 125225), ('\u{1e908}', 125226), + ('\u{1e909}', 125227), ('\u{1e90a}', 125228), ('\u{1e90b}', 125229), ('\u{1e90c}', 125230), + ('\u{1e90d}', 125231), ('\u{1e90e}', 125232), ('\u{1e90f}', 125233), ('\u{1e910}', 125234), + ('\u{1e911}', 125235), ('\u{1e912}', 125236), ('\u{1e913}', 125237), ('\u{1e914}', 125238), + ('\u{1e915}', 125239), ('\u{1e916}', 125240), ('\u{1e917}', 125241), ('\u{1e918}', 125242), + ('\u{1e919}', 125243), ('\u{1e91a}', 125244), ('\u{1e91b}', 125245), ('\u{1e91c}', 125246), + ('\u{1e91d}', 125247), ('\u{1e91e}', 125248), ('\u{1e91f}', 125249), ('\u{1e920}', 125250), + ('\u{1e921}', 125251), ]; static LOWERCASE_TABLE_MULTI: &[[char; 3]] = &[ @@ -989,14 +1000,14 @@ pub mod conversions { ('\u{16f}', 366), ('\u{171}', 368), ('\u{173}', 370), ('\u{175}', 372), ('\u{177}', 374), ('\u{17a}', 377), ('\u{17c}', 379), ('\u{17e}', 381), ('\u{17f}', 83), ('\u{180}', 579), ('\u{183}', 386), ('\u{185}', 388), ('\u{188}', 391), ('\u{18c}', 395), ('\u{192}', 401), - ('\u{195}', 502), ('\u{199}', 408), ('\u{19a}', 573), ('\u{19e}', 544), ('\u{1a1}', 416), - ('\u{1a3}', 418), ('\u{1a5}', 420), ('\u{1a8}', 423), ('\u{1ad}', 428), ('\u{1b0}', 431), - ('\u{1b4}', 435), ('\u{1b6}', 437), ('\u{1b9}', 440), ('\u{1bd}', 444), ('\u{1bf}', 503), - ('\u{1c5}', 452), ('\u{1c6}', 452), ('\u{1c8}', 455), ('\u{1c9}', 455), ('\u{1cb}', 458), - ('\u{1cc}', 458), ('\u{1ce}', 461), ('\u{1d0}', 463), ('\u{1d2}', 465), ('\u{1d4}', 467), - ('\u{1d6}', 469), ('\u{1d8}', 471), ('\u{1da}', 473), ('\u{1dc}', 475), ('\u{1dd}', 398), - ('\u{1df}', 478), ('\u{1e1}', 480), ('\u{1e3}', 482), ('\u{1e5}', 484), ('\u{1e7}', 486), - ('\u{1e9}', 488), ('\u{1eb}', 490), ('\u{1ed}', 492), ('\u{1ef}', 494), + ('\u{195}', 502), ('\u{199}', 408), ('\u{19a}', 573), ('\u{19b}', 42972), ('\u{19e}', 544), + ('\u{1a1}', 416), ('\u{1a3}', 418), ('\u{1a5}', 420), ('\u{1a8}', 423), ('\u{1ad}', 428), + ('\u{1b0}', 431), ('\u{1b4}', 435), ('\u{1b6}', 437), ('\u{1b9}', 440), ('\u{1bd}', 444), + ('\u{1bf}', 503), ('\u{1c5}', 452), ('\u{1c6}', 452), ('\u{1c8}', 455), ('\u{1c9}', 455), + ('\u{1cb}', 458), ('\u{1cc}', 458), ('\u{1ce}', 461), ('\u{1d0}', 463), ('\u{1d2}', 465), + ('\u{1d4}', 467), ('\u{1d6}', 469), ('\u{1d8}', 471), ('\u{1da}', 473), ('\u{1dc}', 475), + ('\u{1dd}', 398), ('\u{1df}', 478), ('\u{1e1}', 480), ('\u{1e3}', 482), ('\u{1e5}', 484), + ('\u{1e7}', 486), ('\u{1e9}', 488), ('\u{1eb}', 490), ('\u{1ed}', 492), ('\u{1ef}', 494), ('\u{1f0}', 4194306), ('\u{1f2}', 497), ('\u{1f3}', 497), ('\u{1f5}', 500), ('\u{1f9}', 504), ('\u{1fb}', 506), ('\u{1fd}', 508), ('\u{1ff}', 510), ('\u{201}', 512), ('\u{203}', 514), ('\u{205}', 516), ('\u{207}', 518), ('\u{209}', 520), ('\u{20b}', 522), @@ -1008,25 +1019,25 @@ pub mod conversions { ('\u{249}', 584), ('\u{24b}', 586), ('\u{24d}', 588), ('\u{24f}', 590), ('\u{250}', 11375), ('\u{251}', 11373), ('\u{252}', 11376), ('\u{253}', 385), ('\u{254}', 390), ('\u{256}', 393), ('\u{257}', 394), ('\u{259}', 399), ('\u{25b}', 400), ('\u{25c}', 42923), - ('\u{260}', 403), ('\u{261}', 42924), ('\u{263}', 404), ('\u{265}', 42893), - ('\u{266}', 42922), ('\u{268}', 407), ('\u{269}', 406), ('\u{26a}', 42926), - ('\u{26b}', 11362), ('\u{26c}', 42925), ('\u{26f}', 412), ('\u{271}', 11374), - ('\u{272}', 413), ('\u{275}', 415), ('\u{27d}', 11364), ('\u{280}', 422), - ('\u{282}', 42949), ('\u{283}', 425), ('\u{287}', 42929), ('\u{288}', 430), - ('\u{289}', 580), ('\u{28a}', 433), ('\u{28b}', 434), ('\u{28c}', 581), ('\u{292}', 439), - ('\u{29d}', 42930), ('\u{29e}', 42928), ('\u{345}', 921), ('\u{371}', 880), - ('\u{373}', 882), ('\u{377}', 886), ('\u{37b}', 1021), ('\u{37c}', 1022), ('\u{37d}', 1023), - ('\u{390}', 4194307), ('\u{3ac}', 902), ('\u{3ad}', 904), ('\u{3ae}', 905), - ('\u{3af}', 906), ('\u{3b0}', 4194308), ('\u{3b1}', 913), ('\u{3b2}', 914), - ('\u{3b3}', 915), ('\u{3b4}', 916), ('\u{3b5}', 917), ('\u{3b6}', 918), ('\u{3b7}', 919), - ('\u{3b8}', 920), ('\u{3b9}', 921), ('\u{3ba}', 922), ('\u{3bb}', 923), ('\u{3bc}', 924), - ('\u{3bd}', 925), ('\u{3be}', 926), ('\u{3bf}', 927), ('\u{3c0}', 928), ('\u{3c1}', 929), - ('\u{3c2}', 931), ('\u{3c3}', 931), ('\u{3c4}', 932), ('\u{3c5}', 933), ('\u{3c6}', 934), - ('\u{3c7}', 935), ('\u{3c8}', 936), ('\u{3c9}', 937), ('\u{3ca}', 938), ('\u{3cb}', 939), - ('\u{3cc}', 908), ('\u{3cd}', 910), ('\u{3ce}', 911), ('\u{3d0}', 914), ('\u{3d1}', 920), - ('\u{3d5}', 934), ('\u{3d6}', 928), ('\u{3d7}', 975), ('\u{3d9}', 984), ('\u{3db}', 986), - ('\u{3dd}', 988), ('\u{3df}', 990), ('\u{3e1}', 992), ('\u{3e3}', 994), ('\u{3e5}', 996), - ('\u{3e7}', 998), ('\u{3e9}', 1000), ('\u{3eb}', 1002), ('\u{3ed}', 1004), + ('\u{260}', 403), ('\u{261}', 42924), ('\u{263}', 404), ('\u{264}', 42955), + ('\u{265}', 42893), ('\u{266}', 42922), ('\u{268}', 407), ('\u{269}', 406), + ('\u{26a}', 42926), ('\u{26b}', 11362), ('\u{26c}', 42925), ('\u{26f}', 412), + ('\u{271}', 11374), ('\u{272}', 413), ('\u{275}', 415), ('\u{27d}', 11364), + ('\u{280}', 422), ('\u{282}', 42949), ('\u{283}', 425), ('\u{287}', 42929), + ('\u{288}', 430), ('\u{289}', 580), ('\u{28a}', 433), ('\u{28b}', 434), ('\u{28c}', 581), + ('\u{292}', 439), ('\u{29d}', 42930), ('\u{29e}', 42928), ('\u{345}', 921), + ('\u{371}', 880), ('\u{373}', 882), ('\u{377}', 886), ('\u{37b}', 1021), ('\u{37c}', 1022), + ('\u{37d}', 1023), ('\u{390}', 4194307), ('\u{3ac}', 902), ('\u{3ad}', 904), + ('\u{3ae}', 905), ('\u{3af}', 906), ('\u{3b0}', 4194308), ('\u{3b1}', 913), + ('\u{3b2}', 914), ('\u{3b3}', 915), ('\u{3b4}', 916), ('\u{3b5}', 917), ('\u{3b6}', 918), + ('\u{3b7}', 919), ('\u{3b8}', 920), ('\u{3b9}', 921), ('\u{3ba}', 922), ('\u{3bb}', 923), + ('\u{3bc}', 924), ('\u{3bd}', 925), ('\u{3be}', 926), ('\u{3bf}', 927), ('\u{3c0}', 928), + ('\u{3c1}', 929), ('\u{3c2}', 931), ('\u{3c3}', 931), ('\u{3c4}', 932), ('\u{3c5}', 933), + ('\u{3c6}', 934), ('\u{3c7}', 935), ('\u{3c8}', 936), ('\u{3c9}', 937), ('\u{3ca}', 938), + ('\u{3cb}', 939), ('\u{3cc}', 908), ('\u{3cd}', 910), ('\u{3ce}', 911), ('\u{3d0}', 914), + ('\u{3d1}', 920), ('\u{3d5}', 934), ('\u{3d6}', 928), ('\u{3d7}', 975), ('\u{3d9}', 984), + ('\u{3db}', 986), ('\u{3dd}', 988), ('\u{3df}', 990), ('\u{3e1}', 992), ('\u{3e3}', 994), + ('\u{3e5}', 996), ('\u{3e7}', 998), ('\u{3e9}', 1000), ('\u{3eb}', 1002), ('\u{3ed}', 1004), ('\u{3ef}', 1006), ('\u{3f0}', 922), ('\u{3f1}', 929), ('\u{3f2}', 1017), ('\u{3f3}', 895), ('\u{3f5}', 917), ('\u{3f8}', 1015), ('\u{3fb}', 1018), ('\u{430}', 1040), ('\u{431}', 1041), ('\u{432}', 1042), ('\u{433}', 1043), ('\u{434}', 1044), @@ -1090,248 +1101,254 @@ pub mod conversions { ('\u{13f8}', 5104), ('\u{13f9}', 5105), ('\u{13fa}', 5106), ('\u{13fb}', 5107), ('\u{13fc}', 5108), ('\u{13fd}', 5109), ('\u{1c80}', 1042), ('\u{1c81}', 1044), ('\u{1c82}', 1054), ('\u{1c83}', 1057), ('\u{1c84}', 1058), ('\u{1c85}', 1058), - ('\u{1c86}', 1066), ('\u{1c87}', 1122), ('\u{1c88}', 42570), ('\u{1d79}', 42877), - ('\u{1d7d}', 11363), ('\u{1d8e}', 42950), ('\u{1e01}', 7680), ('\u{1e03}', 7682), - ('\u{1e05}', 7684), ('\u{1e07}', 7686), ('\u{1e09}', 7688), ('\u{1e0b}', 7690), - ('\u{1e0d}', 7692), ('\u{1e0f}', 7694), ('\u{1e11}', 7696), ('\u{1e13}', 7698), - ('\u{1e15}', 7700), ('\u{1e17}', 7702), ('\u{1e19}', 7704), ('\u{1e1b}', 7706), - ('\u{1e1d}', 7708), ('\u{1e1f}', 7710), ('\u{1e21}', 7712), ('\u{1e23}', 7714), - ('\u{1e25}', 7716), ('\u{1e27}', 7718), ('\u{1e29}', 7720), ('\u{1e2b}', 7722), - ('\u{1e2d}', 7724), ('\u{1e2f}', 7726), ('\u{1e31}', 7728), ('\u{1e33}', 7730), - ('\u{1e35}', 7732), ('\u{1e37}', 7734), ('\u{1e39}', 7736), ('\u{1e3b}', 7738), - ('\u{1e3d}', 7740), ('\u{1e3f}', 7742), ('\u{1e41}', 7744), ('\u{1e43}', 7746), - ('\u{1e45}', 7748), ('\u{1e47}', 7750), ('\u{1e49}', 7752), ('\u{1e4b}', 7754), - ('\u{1e4d}', 7756), ('\u{1e4f}', 7758), ('\u{1e51}', 7760), ('\u{1e53}', 7762), - ('\u{1e55}', 7764), ('\u{1e57}', 7766), ('\u{1e59}', 7768), ('\u{1e5b}', 7770), - ('\u{1e5d}', 7772), ('\u{1e5f}', 7774), ('\u{1e61}', 7776), ('\u{1e63}', 7778), - ('\u{1e65}', 7780), ('\u{1e67}', 7782), ('\u{1e69}', 7784), ('\u{1e6b}', 7786), - ('\u{1e6d}', 7788), ('\u{1e6f}', 7790), ('\u{1e71}', 7792), ('\u{1e73}', 7794), - ('\u{1e75}', 7796), ('\u{1e77}', 7798), ('\u{1e79}', 7800), ('\u{1e7b}', 7802), - ('\u{1e7d}', 7804), ('\u{1e7f}', 7806), ('\u{1e81}', 7808), ('\u{1e83}', 7810), - ('\u{1e85}', 7812), ('\u{1e87}', 7814), ('\u{1e89}', 7816), ('\u{1e8b}', 7818), - ('\u{1e8d}', 7820), ('\u{1e8f}', 7822), ('\u{1e91}', 7824), ('\u{1e93}', 7826), - ('\u{1e95}', 7828), ('\u{1e96}', 4194310), ('\u{1e97}', 4194311), ('\u{1e98}', 4194312), - ('\u{1e99}', 4194313), ('\u{1e9a}', 4194314), ('\u{1e9b}', 7776), ('\u{1ea1}', 7840), - ('\u{1ea3}', 7842), ('\u{1ea5}', 7844), ('\u{1ea7}', 7846), ('\u{1ea9}', 7848), - ('\u{1eab}', 7850), ('\u{1ead}', 7852), ('\u{1eaf}', 7854), ('\u{1eb1}', 7856), - ('\u{1eb3}', 7858), ('\u{1eb5}', 7860), ('\u{1eb7}', 7862), ('\u{1eb9}', 7864), - ('\u{1ebb}', 7866), ('\u{1ebd}', 7868), ('\u{1ebf}', 7870), ('\u{1ec1}', 7872), - ('\u{1ec3}', 7874), ('\u{1ec5}', 7876), ('\u{1ec7}', 7878), ('\u{1ec9}', 7880), - ('\u{1ecb}', 7882), ('\u{1ecd}', 7884), ('\u{1ecf}', 7886), ('\u{1ed1}', 7888), - ('\u{1ed3}', 7890), ('\u{1ed5}', 7892), ('\u{1ed7}', 7894), ('\u{1ed9}', 7896), - ('\u{1edb}', 7898), ('\u{1edd}', 7900), ('\u{1edf}', 7902), ('\u{1ee1}', 7904), - ('\u{1ee3}', 7906), ('\u{1ee5}', 7908), ('\u{1ee7}', 7910), ('\u{1ee9}', 7912), - ('\u{1eeb}', 7914), ('\u{1eed}', 7916), ('\u{1eef}', 7918), ('\u{1ef1}', 7920), - ('\u{1ef3}', 7922), ('\u{1ef5}', 7924), ('\u{1ef7}', 7926), ('\u{1ef9}', 7928), - ('\u{1efb}', 7930), ('\u{1efd}', 7932), ('\u{1eff}', 7934), ('\u{1f00}', 7944), - ('\u{1f01}', 7945), ('\u{1f02}', 7946), ('\u{1f03}', 7947), ('\u{1f04}', 7948), - ('\u{1f05}', 7949), ('\u{1f06}', 7950), ('\u{1f07}', 7951), ('\u{1f10}', 7960), - ('\u{1f11}', 7961), ('\u{1f12}', 7962), ('\u{1f13}', 7963), ('\u{1f14}', 7964), - ('\u{1f15}', 7965), ('\u{1f20}', 7976), ('\u{1f21}', 7977), ('\u{1f22}', 7978), - ('\u{1f23}', 7979), ('\u{1f24}', 7980), ('\u{1f25}', 7981), ('\u{1f26}', 7982), - ('\u{1f27}', 7983), ('\u{1f30}', 7992), ('\u{1f31}', 7993), ('\u{1f32}', 7994), - ('\u{1f33}', 7995), ('\u{1f34}', 7996), ('\u{1f35}', 7997), ('\u{1f36}', 7998), - ('\u{1f37}', 7999), ('\u{1f40}', 8008), ('\u{1f41}', 8009), ('\u{1f42}', 8010), - ('\u{1f43}', 8011), ('\u{1f44}', 8012), ('\u{1f45}', 8013), ('\u{1f50}', 4194315), - ('\u{1f51}', 8025), ('\u{1f52}', 4194316), ('\u{1f53}', 8027), ('\u{1f54}', 4194317), - ('\u{1f55}', 8029), ('\u{1f56}', 4194318), ('\u{1f57}', 8031), ('\u{1f60}', 8040), - ('\u{1f61}', 8041), ('\u{1f62}', 8042), ('\u{1f63}', 8043), ('\u{1f64}', 8044), - ('\u{1f65}', 8045), ('\u{1f66}', 8046), ('\u{1f67}', 8047), ('\u{1f70}', 8122), - ('\u{1f71}', 8123), ('\u{1f72}', 8136), ('\u{1f73}', 8137), ('\u{1f74}', 8138), - ('\u{1f75}', 8139), ('\u{1f76}', 8154), ('\u{1f77}', 8155), ('\u{1f78}', 8184), - ('\u{1f79}', 8185), ('\u{1f7a}', 8170), ('\u{1f7b}', 8171), ('\u{1f7c}', 8186), - ('\u{1f7d}', 8187), ('\u{1f80}', 4194319), ('\u{1f81}', 4194320), ('\u{1f82}', 4194321), - ('\u{1f83}', 4194322), ('\u{1f84}', 4194323), ('\u{1f85}', 4194324), ('\u{1f86}', 4194325), - ('\u{1f87}', 4194326), ('\u{1f88}', 4194327), ('\u{1f89}', 4194328), ('\u{1f8a}', 4194329), - ('\u{1f8b}', 4194330), ('\u{1f8c}', 4194331), ('\u{1f8d}', 4194332), ('\u{1f8e}', 4194333), - ('\u{1f8f}', 4194334), ('\u{1f90}', 4194335), ('\u{1f91}', 4194336), ('\u{1f92}', 4194337), - ('\u{1f93}', 4194338), ('\u{1f94}', 4194339), ('\u{1f95}', 4194340), ('\u{1f96}', 4194341), - ('\u{1f97}', 4194342), ('\u{1f98}', 4194343), ('\u{1f99}', 4194344), ('\u{1f9a}', 4194345), - ('\u{1f9b}', 4194346), ('\u{1f9c}', 4194347), ('\u{1f9d}', 4194348), ('\u{1f9e}', 4194349), - ('\u{1f9f}', 4194350), ('\u{1fa0}', 4194351), ('\u{1fa1}', 4194352), ('\u{1fa2}', 4194353), - ('\u{1fa3}', 4194354), ('\u{1fa4}', 4194355), ('\u{1fa5}', 4194356), ('\u{1fa6}', 4194357), - ('\u{1fa7}', 4194358), ('\u{1fa8}', 4194359), ('\u{1fa9}', 4194360), ('\u{1faa}', 4194361), - ('\u{1fab}', 4194362), ('\u{1fac}', 4194363), ('\u{1fad}', 4194364), ('\u{1fae}', 4194365), - ('\u{1faf}', 4194366), ('\u{1fb0}', 8120), ('\u{1fb1}', 8121), ('\u{1fb2}', 4194367), - ('\u{1fb3}', 4194368), ('\u{1fb4}', 4194369), ('\u{1fb6}', 4194370), ('\u{1fb7}', 4194371), - ('\u{1fbc}', 4194372), ('\u{1fbe}', 921), ('\u{1fc2}', 4194373), ('\u{1fc3}', 4194374), - ('\u{1fc4}', 4194375), ('\u{1fc6}', 4194376), ('\u{1fc7}', 4194377), ('\u{1fcc}', 4194378), - ('\u{1fd0}', 8152), ('\u{1fd1}', 8153), ('\u{1fd2}', 4194379), ('\u{1fd3}', 4194380), - ('\u{1fd6}', 4194381), ('\u{1fd7}', 4194382), ('\u{1fe0}', 8168), ('\u{1fe1}', 8169), - ('\u{1fe2}', 4194383), ('\u{1fe3}', 4194384), ('\u{1fe4}', 4194385), ('\u{1fe5}', 8172), - ('\u{1fe6}', 4194386), ('\u{1fe7}', 4194387), ('\u{1ff2}', 4194388), ('\u{1ff3}', 4194389), - ('\u{1ff4}', 4194390), ('\u{1ff6}', 4194391), ('\u{1ff7}', 4194392), ('\u{1ffc}', 4194393), - ('\u{214e}', 8498), ('\u{2170}', 8544), ('\u{2171}', 8545), ('\u{2172}', 8546), - ('\u{2173}', 8547), ('\u{2174}', 8548), ('\u{2175}', 8549), ('\u{2176}', 8550), - ('\u{2177}', 8551), ('\u{2178}', 8552), ('\u{2179}', 8553), ('\u{217a}', 8554), - ('\u{217b}', 8555), ('\u{217c}', 8556), ('\u{217d}', 8557), ('\u{217e}', 8558), - ('\u{217f}', 8559), ('\u{2184}', 8579), ('\u{24d0}', 9398), ('\u{24d1}', 9399), - ('\u{24d2}', 9400), ('\u{24d3}', 9401), ('\u{24d4}', 9402), ('\u{24d5}', 9403), - ('\u{24d6}', 9404), ('\u{24d7}', 9405), ('\u{24d8}', 9406), ('\u{24d9}', 9407), - ('\u{24da}', 9408), ('\u{24db}', 9409), ('\u{24dc}', 9410), ('\u{24dd}', 9411), - ('\u{24de}', 9412), ('\u{24df}', 9413), ('\u{24e0}', 9414), ('\u{24e1}', 9415), - ('\u{24e2}', 9416), ('\u{24e3}', 9417), ('\u{24e4}', 9418), ('\u{24e5}', 9419), - ('\u{24e6}', 9420), ('\u{24e7}', 9421), ('\u{24e8}', 9422), ('\u{24e9}', 9423), - ('\u{2c30}', 11264), ('\u{2c31}', 11265), ('\u{2c32}', 11266), ('\u{2c33}', 11267), - ('\u{2c34}', 11268), ('\u{2c35}', 11269), ('\u{2c36}', 11270), ('\u{2c37}', 11271), - ('\u{2c38}', 11272), ('\u{2c39}', 11273), ('\u{2c3a}', 11274), ('\u{2c3b}', 11275), - ('\u{2c3c}', 11276), ('\u{2c3d}', 11277), ('\u{2c3e}', 11278), ('\u{2c3f}', 11279), - ('\u{2c40}', 11280), ('\u{2c41}', 11281), ('\u{2c42}', 11282), ('\u{2c43}', 11283), - ('\u{2c44}', 11284), ('\u{2c45}', 11285), ('\u{2c46}', 11286), ('\u{2c47}', 11287), - ('\u{2c48}', 11288), ('\u{2c49}', 11289), ('\u{2c4a}', 11290), ('\u{2c4b}', 11291), - ('\u{2c4c}', 11292), ('\u{2c4d}', 11293), ('\u{2c4e}', 11294), ('\u{2c4f}', 11295), - ('\u{2c50}', 11296), ('\u{2c51}', 11297), ('\u{2c52}', 11298), ('\u{2c53}', 11299), - ('\u{2c54}', 11300), ('\u{2c55}', 11301), ('\u{2c56}', 11302), ('\u{2c57}', 11303), - ('\u{2c58}', 11304), ('\u{2c59}', 11305), ('\u{2c5a}', 11306), ('\u{2c5b}', 11307), - ('\u{2c5c}', 11308), ('\u{2c5d}', 11309), ('\u{2c5e}', 11310), ('\u{2c5f}', 11311), - ('\u{2c61}', 11360), ('\u{2c65}', 570), ('\u{2c66}', 574), ('\u{2c68}', 11367), - ('\u{2c6a}', 11369), ('\u{2c6c}', 11371), ('\u{2c73}', 11378), ('\u{2c76}', 11381), - ('\u{2c81}', 11392), ('\u{2c83}', 11394), ('\u{2c85}', 11396), ('\u{2c87}', 11398), - ('\u{2c89}', 11400), ('\u{2c8b}', 11402), ('\u{2c8d}', 11404), ('\u{2c8f}', 11406), - ('\u{2c91}', 11408), ('\u{2c93}', 11410), ('\u{2c95}', 11412), ('\u{2c97}', 11414), - ('\u{2c99}', 11416), ('\u{2c9b}', 11418), ('\u{2c9d}', 11420), ('\u{2c9f}', 11422), - ('\u{2ca1}', 11424), ('\u{2ca3}', 11426), ('\u{2ca5}', 11428), ('\u{2ca7}', 11430), - ('\u{2ca9}', 11432), ('\u{2cab}', 11434), ('\u{2cad}', 11436), ('\u{2caf}', 11438), - ('\u{2cb1}', 11440), ('\u{2cb3}', 11442), ('\u{2cb5}', 11444), ('\u{2cb7}', 11446), - ('\u{2cb9}', 11448), ('\u{2cbb}', 11450), ('\u{2cbd}', 11452), ('\u{2cbf}', 11454), - ('\u{2cc1}', 11456), ('\u{2cc3}', 11458), ('\u{2cc5}', 11460), ('\u{2cc7}', 11462), - ('\u{2cc9}', 11464), ('\u{2ccb}', 11466), ('\u{2ccd}', 11468), ('\u{2ccf}', 11470), - ('\u{2cd1}', 11472), ('\u{2cd3}', 11474), ('\u{2cd5}', 11476), ('\u{2cd7}', 11478), - ('\u{2cd9}', 11480), ('\u{2cdb}', 11482), ('\u{2cdd}', 11484), ('\u{2cdf}', 11486), - ('\u{2ce1}', 11488), ('\u{2ce3}', 11490), ('\u{2cec}', 11499), ('\u{2cee}', 11501), - ('\u{2cf3}', 11506), ('\u{2d00}', 4256), ('\u{2d01}', 4257), ('\u{2d02}', 4258), - ('\u{2d03}', 4259), ('\u{2d04}', 4260), ('\u{2d05}', 4261), ('\u{2d06}', 4262), - ('\u{2d07}', 4263), ('\u{2d08}', 4264), ('\u{2d09}', 4265), ('\u{2d0a}', 4266), - ('\u{2d0b}', 4267), ('\u{2d0c}', 4268), ('\u{2d0d}', 4269), ('\u{2d0e}', 4270), - ('\u{2d0f}', 4271), ('\u{2d10}', 4272), ('\u{2d11}', 4273), ('\u{2d12}', 4274), - ('\u{2d13}', 4275), ('\u{2d14}', 4276), ('\u{2d15}', 4277), ('\u{2d16}', 4278), - ('\u{2d17}', 4279), ('\u{2d18}', 4280), ('\u{2d19}', 4281), ('\u{2d1a}', 4282), - ('\u{2d1b}', 4283), ('\u{2d1c}', 4284), ('\u{2d1d}', 4285), ('\u{2d1e}', 4286), - ('\u{2d1f}', 4287), ('\u{2d20}', 4288), ('\u{2d21}', 4289), ('\u{2d22}', 4290), - ('\u{2d23}', 4291), ('\u{2d24}', 4292), ('\u{2d25}', 4293), ('\u{2d27}', 4295), - ('\u{2d2d}', 4301), ('\u{a641}', 42560), ('\u{a643}', 42562), ('\u{a645}', 42564), - ('\u{a647}', 42566), ('\u{a649}', 42568), ('\u{a64b}', 42570), ('\u{a64d}', 42572), - ('\u{a64f}', 42574), ('\u{a651}', 42576), ('\u{a653}', 42578), ('\u{a655}', 42580), - ('\u{a657}', 42582), ('\u{a659}', 42584), ('\u{a65b}', 42586), ('\u{a65d}', 42588), - ('\u{a65f}', 42590), ('\u{a661}', 42592), ('\u{a663}', 42594), ('\u{a665}', 42596), - ('\u{a667}', 42598), ('\u{a669}', 42600), ('\u{a66b}', 42602), ('\u{a66d}', 42604), - ('\u{a681}', 42624), ('\u{a683}', 42626), ('\u{a685}', 42628), ('\u{a687}', 42630), - ('\u{a689}', 42632), ('\u{a68b}', 42634), ('\u{a68d}', 42636), ('\u{a68f}', 42638), - ('\u{a691}', 42640), ('\u{a693}', 42642), ('\u{a695}', 42644), ('\u{a697}', 42646), - ('\u{a699}', 42648), ('\u{a69b}', 42650), ('\u{a723}', 42786), ('\u{a725}', 42788), - ('\u{a727}', 42790), ('\u{a729}', 42792), ('\u{a72b}', 42794), ('\u{a72d}', 42796), - ('\u{a72f}', 42798), ('\u{a733}', 42802), ('\u{a735}', 42804), ('\u{a737}', 42806), - ('\u{a739}', 42808), ('\u{a73b}', 42810), ('\u{a73d}', 42812), ('\u{a73f}', 42814), - ('\u{a741}', 42816), ('\u{a743}', 42818), ('\u{a745}', 42820), ('\u{a747}', 42822), - ('\u{a749}', 42824), ('\u{a74b}', 42826), ('\u{a74d}', 42828), ('\u{a74f}', 42830), - ('\u{a751}', 42832), ('\u{a753}', 42834), ('\u{a755}', 42836), ('\u{a757}', 42838), - ('\u{a759}', 42840), ('\u{a75b}', 42842), ('\u{a75d}', 42844), ('\u{a75f}', 42846), - ('\u{a761}', 42848), ('\u{a763}', 42850), ('\u{a765}', 42852), ('\u{a767}', 42854), - ('\u{a769}', 42856), ('\u{a76b}', 42858), ('\u{a76d}', 42860), ('\u{a76f}', 42862), - ('\u{a77a}', 42873), ('\u{a77c}', 42875), ('\u{a77f}', 42878), ('\u{a781}', 42880), - ('\u{a783}', 42882), ('\u{a785}', 42884), ('\u{a787}', 42886), ('\u{a78c}', 42891), - ('\u{a791}', 42896), ('\u{a793}', 42898), ('\u{a794}', 42948), ('\u{a797}', 42902), - ('\u{a799}', 42904), ('\u{a79b}', 42906), ('\u{a79d}', 42908), ('\u{a79f}', 42910), - ('\u{a7a1}', 42912), ('\u{a7a3}', 42914), ('\u{a7a5}', 42916), ('\u{a7a7}', 42918), - ('\u{a7a9}', 42920), ('\u{a7b5}', 42932), ('\u{a7b7}', 42934), ('\u{a7b9}', 42936), - ('\u{a7bb}', 42938), ('\u{a7bd}', 42940), ('\u{a7bf}', 42942), ('\u{a7c1}', 42944), - ('\u{a7c3}', 42946), ('\u{a7c8}', 42951), ('\u{a7ca}', 42953), ('\u{a7d1}', 42960), - ('\u{a7d7}', 42966), ('\u{a7d9}', 42968), ('\u{a7f6}', 42997), ('\u{ab53}', 42931), - ('\u{ab70}', 5024), ('\u{ab71}', 5025), ('\u{ab72}', 5026), ('\u{ab73}', 5027), - ('\u{ab74}', 5028), ('\u{ab75}', 5029), ('\u{ab76}', 5030), ('\u{ab77}', 5031), - ('\u{ab78}', 5032), ('\u{ab79}', 5033), ('\u{ab7a}', 5034), ('\u{ab7b}', 5035), - ('\u{ab7c}', 5036), ('\u{ab7d}', 5037), ('\u{ab7e}', 5038), ('\u{ab7f}', 5039), - ('\u{ab80}', 5040), ('\u{ab81}', 5041), ('\u{ab82}', 5042), ('\u{ab83}', 5043), - ('\u{ab84}', 5044), ('\u{ab85}', 5045), ('\u{ab86}', 5046), ('\u{ab87}', 5047), - ('\u{ab88}', 5048), ('\u{ab89}', 5049), ('\u{ab8a}', 5050), ('\u{ab8b}', 5051), - ('\u{ab8c}', 5052), ('\u{ab8d}', 5053), ('\u{ab8e}', 5054), ('\u{ab8f}', 5055), - ('\u{ab90}', 5056), ('\u{ab91}', 5057), ('\u{ab92}', 5058), ('\u{ab93}', 5059), - ('\u{ab94}', 5060), ('\u{ab95}', 5061), ('\u{ab96}', 5062), ('\u{ab97}', 5063), - ('\u{ab98}', 5064), ('\u{ab99}', 5065), ('\u{ab9a}', 5066), ('\u{ab9b}', 5067), - ('\u{ab9c}', 5068), ('\u{ab9d}', 5069), ('\u{ab9e}', 5070), ('\u{ab9f}', 5071), - ('\u{aba0}', 5072), ('\u{aba1}', 5073), ('\u{aba2}', 5074), ('\u{aba3}', 5075), - ('\u{aba4}', 5076), ('\u{aba5}', 5077), ('\u{aba6}', 5078), ('\u{aba7}', 5079), - ('\u{aba8}', 5080), ('\u{aba9}', 5081), ('\u{abaa}', 5082), ('\u{abab}', 5083), - ('\u{abac}', 5084), ('\u{abad}', 5085), ('\u{abae}', 5086), ('\u{abaf}', 5087), - ('\u{abb0}', 5088), ('\u{abb1}', 5089), ('\u{abb2}', 5090), ('\u{abb3}', 5091), - ('\u{abb4}', 5092), ('\u{abb5}', 5093), ('\u{abb6}', 5094), ('\u{abb7}', 5095), - ('\u{abb8}', 5096), ('\u{abb9}', 5097), ('\u{abba}', 5098), ('\u{abbb}', 5099), - ('\u{abbc}', 5100), ('\u{abbd}', 5101), ('\u{abbe}', 5102), ('\u{abbf}', 5103), - ('\u{fb00}', 4194394), ('\u{fb01}', 4194395), ('\u{fb02}', 4194396), ('\u{fb03}', 4194397), - ('\u{fb04}', 4194398), ('\u{fb05}', 4194399), ('\u{fb06}', 4194400), ('\u{fb13}', 4194401), - ('\u{fb14}', 4194402), ('\u{fb15}', 4194403), ('\u{fb16}', 4194404), ('\u{fb17}', 4194405), - ('\u{ff41}', 65313), ('\u{ff42}', 65314), ('\u{ff43}', 65315), ('\u{ff44}', 65316), - ('\u{ff45}', 65317), ('\u{ff46}', 65318), ('\u{ff47}', 65319), ('\u{ff48}', 65320), - ('\u{ff49}', 65321), ('\u{ff4a}', 65322), ('\u{ff4b}', 65323), ('\u{ff4c}', 65324), - ('\u{ff4d}', 65325), ('\u{ff4e}', 65326), ('\u{ff4f}', 65327), ('\u{ff50}', 65328), - ('\u{ff51}', 65329), ('\u{ff52}', 65330), ('\u{ff53}', 65331), ('\u{ff54}', 65332), - ('\u{ff55}', 65333), ('\u{ff56}', 65334), ('\u{ff57}', 65335), ('\u{ff58}', 65336), - ('\u{ff59}', 65337), ('\u{ff5a}', 65338), ('\u{10428}', 66560), ('\u{10429}', 66561), - ('\u{1042a}', 66562), ('\u{1042b}', 66563), ('\u{1042c}', 66564), ('\u{1042d}', 66565), - ('\u{1042e}', 66566), ('\u{1042f}', 66567), ('\u{10430}', 66568), ('\u{10431}', 66569), - ('\u{10432}', 66570), ('\u{10433}', 66571), ('\u{10434}', 66572), ('\u{10435}', 66573), - ('\u{10436}', 66574), ('\u{10437}', 66575), ('\u{10438}', 66576), ('\u{10439}', 66577), - ('\u{1043a}', 66578), ('\u{1043b}', 66579), ('\u{1043c}', 66580), ('\u{1043d}', 66581), - ('\u{1043e}', 66582), ('\u{1043f}', 66583), ('\u{10440}', 66584), ('\u{10441}', 66585), - ('\u{10442}', 66586), ('\u{10443}', 66587), ('\u{10444}', 66588), ('\u{10445}', 66589), - ('\u{10446}', 66590), ('\u{10447}', 66591), ('\u{10448}', 66592), ('\u{10449}', 66593), - ('\u{1044a}', 66594), ('\u{1044b}', 66595), ('\u{1044c}', 66596), ('\u{1044d}', 66597), - ('\u{1044e}', 66598), ('\u{1044f}', 66599), ('\u{104d8}', 66736), ('\u{104d9}', 66737), - ('\u{104da}', 66738), ('\u{104db}', 66739), ('\u{104dc}', 66740), ('\u{104dd}', 66741), - ('\u{104de}', 66742), ('\u{104df}', 66743), ('\u{104e0}', 66744), ('\u{104e1}', 66745), - ('\u{104e2}', 66746), ('\u{104e3}', 66747), ('\u{104e4}', 66748), ('\u{104e5}', 66749), - ('\u{104e6}', 66750), ('\u{104e7}', 66751), ('\u{104e8}', 66752), ('\u{104e9}', 66753), - ('\u{104ea}', 66754), ('\u{104eb}', 66755), ('\u{104ec}', 66756), ('\u{104ed}', 66757), - ('\u{104ee}', 66758), ('\u{104ef}', 66759), ('\u{104f0}', 66760), ('\u{104f1}', 66761), - ('\u{104f2}', 66762), ('\u{104f3}', 66763), ('\u{104f4}', 66764), ('\u{104f5}', 66765), - ('\u{104f6}', 66766), ('\u{104f7}', 66767), ('\u{104f8}', 66768), ('\u{104f9}', 66769), - ('\u{104fa}', 66770), ('\u{104fb}', 66771), ('\u{10597}', 66928), ('\u{10598}', 66929), - ('\u{10599}', 66930), ('\u{1059a}', 66931), ('\u{1059b}', 66932), ('\u{1059c}', 66933), - ('\u{1059d}', 66934), ('\u{1059e}', 66935), ('\u{1059f}', 66936), ('\u{105a0}', 66937), - ('\u{105a1}', 66938), ('\u{105a3}', 66940), ('\u{105a4}', 66941), ('\u{105a5}', 66942), - ('\u{105a6}', 66943), ('\u{105a7}', 66944), ('\u{105a8}', 66945), ('\u{105a9}', 66946), - ('\u{105aa}', 66947), ('\u{105ab}', 66948), ('\u{105ac}', 66949), ('\u{105ad}', 66950), - ('\u{105ae}', 66951), ('\u{105af}', 66952), ('\u{105b0}', 66953), ('\u{105b1}', 66954), - ('\u{105b3}', 66956), ('\u{105b4}', 66957), ('\u{105b5}', 66958), ('\u{105b6}', 66959), - ('\u{105b7}', 66960), ('\u{105b8}', 66961), ('\u{105b9}', 66962), ('\u{105bb}', 66964), - ('\u{105bc}', 66965), ('\u{10cc0}', 68736), ('\u{10cc1}', 68737), ('\u{10cc2}', 68738), - ('\u{10cc3}', 68739), ('\u{10cc4}', 68740), ('\u{10cc5}', 68741), ('\u{10cc6}', 68742), - ('\u{10cc7}', 68743), ('\u{10cc8}', 68744), ('\u{10cc9}', 68745), ('\u{10cca}', 68746), - ('\u{10ccb}', 68747), ('\u{10ccc}', 68748), ('\u{10ccd}', 68749), ('\u{10cce}', 68750), - ('\u{10ccf}', 68751), ('\u{10cd0}', 68752), ('\u{10cd1}', 68753), ('\u{10cd2}', 68754), - ('\u{10cd3}', 68755), ('\u{10cd4}', 68756), ('\u{10cd5}', 68757), ('\u{10cd6}', 68758), - ('\u{10cd7}', 68759), ('\u{10cd8}', 68760), ('\u{10cd9}', 68761), ('\u{10cda}', 68762), - ('\u{10cdb}', 68763), ('\u{10cdc}', 68764), ('\u{10cdd}', 68765), ('\u{10cde}', 68766), - ('\u{10cdf}', 68767), ('\u{10ce0}', 68768), ('\u{10ce1}', 68769), ('\u{10ce2}', 68770), - ('\u{10ce3}', 68771), ('\u{10ce4}', 68772), ('\u{10ce5}', 68773), ('\u{10ce6}', 68774), - ('\u{10ce7}', 68775), ('\u{10ce8}', 68776), ('\u{10ce9}', 68777), ('\u{10cea}', 68778), - ('\u{10ceb}', 68779), ('\u{10cec}', 68780), ('\u{10ced}', 68781), ('\u{10cee}', 68782), - ('\u{10cef}', 68783), ('\u{10cf0}', 68784), ('\u{10cf1}', 68785), ('\u{10cf2}', 68786), - ('\u{118c0}', 71840), ('\u{118c1}', 71841), ('\u{118c2}', 71842), ('\u{118c3}', 71843), - ('\u{118c4}', 71844), ('\u{118c5}', 71845), ('\u{118c6}', 71846), ('\u{118c7}', 71847), - ('\u{118c8}', 71848), ('\u{118c9}', 71849), ('\u{118ca}', 71850), ('\u{118cb}', 71851), - ('\u{118cc}', 71852), ('\u{118cd}', 71853), ('\u{118ce}', 71854), ('\u{118cf}', 71855), - ('\u{118d0}', 71856), ('\u{118d1}', 71857), ('\u{118d2}', 71858), ('\u{118d3}', 71859), - ('\u{118d4}', 71860), ('\u{118d5}', 71861), ('\u{118d6}', 71862), ('\u{118d7}', 71863), - ('\u{118d8}', 71864), ('\u{118d9}', 71865), ('\u{118da}', 71866), ('\u{118db}', 71867), - ('\u{118dc}', 71868), ('\u{118dd}', 71869), ('\u{118de}', 71870), ('\u{118df}', 71871), - ('\u{16e60}', 93760), ('\u{16e61}', 93761), ('\u{16e62}', 93762), ('\u{16e63}', 93763), - ('\u{16e64}', 93764), ('\u{16e65}', 93765), ('\u{16e66}', 93766), ('\u{16e67}', 93767), - ('\u{16e68}', 93768), ('\u{16e69}', 93769), ('\u{16e6a}', 93770), ('\u{16e6b}', 93771), - ('\u{16e6c}', 93772), ('\u{16e6d}', 93773), ('\u{16e6e}', 93774), ('\u{16e6f}', 93775), - ('\u{16e70}', 93776), ('\u{16e71}', 93777), ('\u{16e72}', 93778), ('\u{16e73}', 93779), - ('\u{16e74}', 93780), ('\u{16e75}', 93781), ('\u{16e76}', 93782), ('\u{16e77}', 93783), - ('\u{16e78}', 93784), ('\u{16e79}', 93785), ('\u{16e7a}', 93786), ('\u{16e7b}', 93787), - ('\u{16e7c}', 93788), ('\u{16e7d}', 93789), ('\u{16e7e}', 93790), ('\u{16e7f}', 93791), - ('\u{1e922}', 125184), ('\u{1e923}', 125185), ('\u{1e924}', 125186), ('\u{1e925}', 125187), - ('\u{1e926}', 125188), ('\u{1e927}', 125189), ('\u{1e928}', 125190), ('\u{1e929}', 125191), - ('\u{1e92a}', 125192), ('\u{1e92b}', 125193), ('\u{1e92c}', 125194), ('\u{1e92d}', 125195), - ('\u{1e92e}', 125196), ('\u{1e92f}', 125197), ('\u{1e930}', 125198), ('\u{1e931}', 125199), - ('\u{1e932}', 125200), ('\u{1e933}', 125201), ('\u{1e934}', 125202), ('\u{1e935}', 125203), - ('\u{1e936}', 125204), ('\u{1e937}', 125205), ('\u{1e938}', 125206), ('\u{1e939}', 125207), - ('\u{1e93a}', 125208), ('\u{1e93b}', 125209), ('\u{1e93c}', 125210), ('\u{1e93d}', 125211), - ('\u{1e93e}', 125212), ('\u{1e93f}', 125213), ('\u{1e940}', 125214), ('\u{1e941}', 125215), - ('\u{1e942}', 125216), ('\u{1e943}', 125217), + ('\u{1c86}', 1066), ('\u{1c87}', 1122), ('\u{1c88}', 42570), ('\u{1c8a}', 7305), + ('\u{1d79}', 42877), ('\u{1d7d}', 11363), ('\u{1d8e}', 42950), ('\u{1e01}', 7680), + ('\u{1e03}', 7682), ('\u{1e05}', 7684), ('\u{1e07}', 7686), ('\u{1e09}', 7688), + ('\u{1e0b}', 7690), ('\u{1e0d}', 7692), ('\u{1e0f}', 7694), ('\u{1e11}', 7696), + ('\u{1e13}', 7698), ('\u{1e15}', 7700), ('\u{1e17}', 7702), ('\u{1e19}', 7704), + ('\u{1e1b}', 7706), ('\u{1e1d}', 7708), ('\u{1e1f}', 7710), ('\u{1e21}', 7712), + ('\u{1e23}', 7714), ('\u{1e25}', 7716), ('\u{1e27}', 7718), ('\u{1e29}', 7720), + ('\u{1e2b}', 7722), ('\u{1e2d}', 7724), ('\u{1e2f}', 7726), ('\u{1e31}', 7728), + ('\u{1e33}', 7730), ('\u{1e35}', 7732), ('\u{1e37}', 7734), ('\u{1e39}', 7736), + ('\u{1e3b}', 7738), ('\u{1e3d}', 7740), ('\u{1e3f}', 7742), ('\u{1e41}', 7744), + ('\u{1e43}', 7746), ('\u{1e45}', 7748), ('\u{1e47}', 7750), ('\u{1e49}', 7752), + ('\u{1e4b}', 7754), ('\u{1e4d}', 7756), ('\u{1e4f}', 7758), ('\u{1e51}', 7760), + ('\u{1e53}', 7762), ('\u{1e55}', 7764), ('\u{1e57}', 7766), ('\u{1e59}', 7768), + ('\u{1e5b}', 7770), ('\u{1e5d}', 7772), ('\u{1e5f}', 7774), ('\u{1e61}', 7776), + ('\u{1e63}', 7778), ('\u{1e65}', 7780), ('\u{1e67}', 7782), ('\u{1e69}', 7784), + ('\u{1e6b}', 7786), ('\u{1e6d}', 7788), ('\u{1e6f}', 7790), ('\u{1e71}', 7792), + ('\u{1e73}', 7794), ('\u{1e75}', 7796), ('\u{1e77}', 7798), ('\u{1e79}', 7800), + ('\u{1e7b}', 7802), ('\u{1e7d}', 7804), ('\u{1e7f}', 7806), ('\u{1e81}', 7808), + ('\u{1e83}', 7810), ('\u{1e85}', 7812), ('\u{1e87}', 7814), ('\u{1e89}', 7816), + ('\u{1e8b}', 7818), ('\u{1e8d}', 7820), ('\u{1e8f}', 7822), ('\u{1e91}', 7824), + ('\u{1e93}', 7826), ('\u{1e95}', 7828), ('\u{1e96}', 4194310), ('\u{1e97}', 4194311), + ('\u{1e98}', 4194312), ('\u{1e99}', 4194313), ('\u{1e9a}', 4194314), ('\u{1e9b}', 7776), + ('\u{1ea1}', 7840), ('\u{1ea3}', 7842), ('\u{1ea5}', 7844), ('\u{1ea7}', 7846), + ('\u{1ea9}', 7848), ('\u{1eab}', 7850), ('\u{1ead}', 7852), ('\u{1eaf}', 7854), + ('\u{1eb1}', 7856), ('\u{1eb3}', 7858), ('\u{1eb5}', 7860), ('\u{1eb7}', 7862), + ('\u{1eb9}', 7864), ('\u{1ebb}', 7866), ('\u{1ebd}', 7868), ('\u{1ebf}', 7870), + ('\u{1ec1}', 7872), ('\u{1ec3}', 7874), ('\u{1ec5}', 7876), ('\u{1ec7}', 7878), + ('\u{1ec9}', 7880), ('\u{1ecb}', 7882), ('\u{1ecd}', 7884), ('\u{1ecf}', 7886), + ('\u{1ed1}', 7888), ('\u{1ed3}', 7890), ('\u{1ed5}', 7892), ('\u{1ed7}', 7894), + ('\u{1ed9}', 7896), ('\u{1edb}', 7898), ('\u{1edd}', 7900), ('\u{1edf}', 7902), + ('\u{1ee1}', 7904), ('\u{1ee3}', 7906), ('\u{1ee5}', 7908), ('\u{1ee7}', 7910), + ('\u{1ee9}', 7912), ('\u{1eeb}', 7914), ('\u{1eed}', 7916), ('\u{1eef}', 7918), + ('\u{1ef1}', 7920), ('\u{1ef3}', 7922), ('\u{1ef5}', 7924), ('\u{1ef7}', 7926), + ('\u{1ef9}', 7928), ('\u{1efb}', 7930), ('\u{1efd}', 7932), ('\u{1eff}', 7934), + ('\u{1f00}', 7944), ('\u{1f01}', 7945), ('\u{1f02}', 7946), ('\u{1f03}', 7947), + ('\u{1f04}', 7948), ('\u{1f05}', 7949), ('\u{1f06}', 7950), ('\u{1f07}', 7951), + ('\u{1f10}', 7960), ('\u{1f11}', 7961), ('\u{1f12}', 7962), ('\u{1f13}', 7963), + ('\u{1f14}', 7964), ('\u{1f15}', 7965), ('\u{1f20}', 7976), ('\u{1f21}', 7977), + ('\u{1f22}', 7978), ('\u{1f23}', 7979), ('\u{1f24}', 7980), ('\u{1f25}', 7981), + ('\u{1f26}', 7982), ('\u{1f27}', 7983), ('\u{1f30}', 7992), ('\u{1f31}', 7993), + ('\u{1f32}', 7994), ('\u{1f33}', 7995), ('\u{1f34}', 7996), ('\u{1f35}', 7997), + ('\u{1f36}', 7998), ('\u{1f37}', 7999), ('\u{1f40}', 8008), ('\u{1f41}', 8009), + ('\u{1f42}', 8010), ('\u{1f43}', 8011), ('\u{1f44}', 8012), ('\u{1f45}', 8013), + ('\u{1f50}', 4194315), ('\u{1f51}', 8025), ('\u{1f52}', 4194316), ('\u{1f53}', 8027), + ('\u{1f54}', 4194317), ('\u{1f55}', 8029), ('\u{1f56}', 4194318), ('\u{1f57}', 8031), + ('\u{1f60}', 8040), ('\u{1f61}', 8041), ('\u{1f62}', 8042), ('\u{1f63}', 8043), + ('\u{1f64}', 8044), ('\u{1f65}', 8045), ('\u{1f66}', 8046), ('\u{1f67}', 8047), + ('\u{1f70}', 8122), ('\u{1f71}', 8123), ('\u{1f72}', 8136), ('\u{1f73}', 8137), + ('\u{1f74}', 8138), ('\u{1f75}', 8139), ('\u{1f76}', 8154), ('\u{1f77}', 8155), + ('\u{1f78}', 8184), ('\u{1f79}', 8185), ('\u{1f7a}', 8170), ('\u{1f7b}', 8171), + ('\u{1f7c}', 8186), ('\u{1f7d}', 8187), ('\u{1f80}', 4194319), ('\u{1f81}', 4194320), + ('\u{1f82}', 4194321), ('\u{1f83}', 4194322), ('\u{1f84}', 4194323), ('\u{1f85}', 4194324), + ('\u{1f86}', 4194325), ('\u{1f87}', 4194326), ('\u{1f88}', 4194327), ('\u{1f89}', 4194328), + ('\u{1f8a}', 4194329), ('\u{1f8b}', 4194330), ('\u{1f8c}', 4194331), ('\u{1f8d}', 4194332), + ('\u{1f8e}', 4194333), ('\u{1f8f}', 4194334), ('\u{1f90}', 4194335), ('\u{1f91}', 4194336), + ('\u{1f92}', 4194337), ('\u{1f93}', 4194338), ('\u{1f94}', 4194339), ('\u{1f95}', 4194340), + ('\u{1f96}', 4194341), ('\u{1f97}', 4194342), ('\u{1f98}', 4194343), ('\u{1f99}', 4194344), + ('\u{1f9a}', 4194345), ('\u{1f9b}', 4194346), ('\u{1f9c}', 4194347), ('\u{1f9d}', 4194348), + ('\u{1f9e}', 4194349), ('\u{1f9f}', 4194350), ('\u{1fa0}', 4194351), ('\u{1fa1}', 4194352), + ('\u{1fa2}', 4194353), ('\u{1fa3}', 4194354), ('\u{1fa4}', 4194355), ('\u{1fa5}', 4194356), + ('\u{1fa6}', 4194357), ('\u{1fa7}', 4194358), ('\u{1fa8}', 4194359), ('\u{1fa9}', 4194360), + ('\u{1faa}', 4194361), ('\u{1fab}', 4194362), ('\u{1fac}', 4194363), ('\u{1fad}', 4194364), + ('\u{1fae}', 4194365), ('\u{1faf}', 4194366), ('\u{1fb0}', 8120), ('\u{1fb1}', 8121), + ('\u{1fb2}', 4194367), ('\u{1fb3}', 4194368), ('\u{1fb4}', 4194369), ('\u{1fb6}', 4194370), + ('\u{1fb7}', 4194371), ('\u{1fbc}', 4194372), ('\u{1fbe}', 921), ('\u{1fc2}', 4194373), + ('\u{1fc3}', 4194374), ('\u{1fc4}', 4194375), ('\u{1fc6}', 4194376), ('\u{1fc7}', 4194377), + ('\u{1fcc}', 4194378), ('\u{1fd0}', 8152), ('\u{1fd1}', 8153), ('\u{1fd2}', 4194379), + ('\u{1fd3}', 4194380), ('\u{1fd6}', 4194381), ('\u{1fd7}', 4194382), ('\u{1fe0}', 8168), + ('\u{1fe1}', 8169), ('\u{1fe2}', 4194383), ('\u{1fe3}', 4194384), ('\u{1fe4}', 4194385), + ('\u{1fe5}', 8172), ('\u{1fe6}', 4194386), ('\u{1fe7}', 4194387), ('\u{1ff2}', 4194388), + ('\u{1ff3}', 4194389), ('\u{1ff4}', 4194390), ('\u{1ff6}', 4194391), ('\u{1ff7}', 4194392), + ('\u{1ffc}', 4194393), ('\u{214e}', 8498), ('\u{2170}', 8544), ('\u{2171}', 8545), + ('\u{2172}', 8546), ('\u{2173}', 8547), ('\u{2174}', 8548), ('\u{2175}', 8549), + ('\u{2176}', 8550), ('\u{2177}', 8551), ('\u{2178}', 8552), ('\u{2179}', 8553), + ('\u{217a}', 8554), ('\u{217b}', 8555), ('\u{217c}', 8556), ('\u{217d}', 8557), + ('\u{217e}', 8558), ('\u{217f}', 8559), ('\u{2184}', 8579), ('\u{24d0}', 9398), + ('\u{24d1}', 9399), ('\u{24d2}', 9400), ('\u{24d3}', 9401), ('\u{24d4}', 9402), + ('\u{24d5}', 9403), ('\u{24d6}', 9404), ('\u{24d7}', 9405), ('\u{24d8}', 9406), + ('\u{24d9}', 9407), ('\u{24da}', 9408), ('\u{24db}', 9409), ('\u{24dc}', 9410), + ('\u{24dd}', 9411), ('\u{24de}', 9412), ('\u{24df}', 9413), ('\u{24e0}', 9414), + ('\u{24e1}', 9415), ('\u{24e2}', 9416), ('\u{24e3}', 9417), ('\u{24e4}', 9418), + ('\u{24e5}', 9419), ('\u{24e6}', 9420), ('\u{24e7}', 9421), ('\u{24e8}', 9422), + ('\u{24e9}', 9423), ('\u{2c30}', 11264), ('\u{2c31}', 11265), ('\u{2c32}', 11266), + ('\u{2c33}', 11267), ('\u{2c34}', 11268), ('\u{2c35}', 11269), ('\u{2c36}', 11270), + ('\u{2c37}', 11271), ('\u{2c38}', 11272), ('\u{2c39}', 11273), ('\u{2c3a}', 11274), + ('\u{2c3b}', 11275), ('\u{2c3c}', 11276), ('\u{2c3d}', 11277), ('\u{2c3e}', 11278), + ('\u{2c3f}', 11279), ('\u{2c40}', 11280), ('\u{2c41}', 11281), ('\u{2c42}', 11282), + ('\u{2c43}', 11283), ('\u{2c44}', 11284), ('\u{2c45}', 11285), ('\u{2c46}', 11286), + ('\u{2c47}', 11287), ('\u{2c48}', 11288), ('\u{2c49}', 11289), ('\u{2c4a}', 11290), + ('\u{2c4b}', 11291), ('\u{2c4c}', 11292), ('\u{2c4d}', 11293), ('\u{2c4e}', 11294), + ('\u{2c4f}', 11295), ('\u{2c50}', 11296), ('\u{2c51}', 11297), ('\u{2c52}', 11298), + ('\u{2c53}', 11299), ('\u{2c54}', 11300), ('\u{2c55}', 11301), ('\u{2c56}', 11302), + ('\u{2c57}', 11303), ('\u{2c58}', 11304), ('\u{2c59}', 11305), ('\u{2c5a}', 11306), + ('\u{2c5b}', 11307), ('\u{2c5c}', 11308), ('\u{2c5d}', 11309), ('\u{2c5e}', 11310), + ('\u{2c5f}', 11311), ('\u{2c61}', 11360), ('\u{2c65}', 570), ('\u{2c66}', 574), + ('\u{2c68}', 11367), ('\u{2c6a}', 11369), ('\u{2c6c}', 11371), ('\u{2c73}', 11378), + ('\u{2c76}', 11381), ('\u{2c81}', 11392), ('\u{2c83}', 11394), ('\u{2c85}', 11396), + ('\u{2c87}', 11398), ('\u{2c89}', 11400), ('\u{2c8b}', 11402), ('\u{2c8d}', 11404), + ('\u{2c8f}', 11406), ('\u{2c91}', 11408), ('\u{2c93}', 11410), ('\u{2c95}', 11412), + ('\u{2c97}', 11414), ('\u{2c99}', 11416), ('\u{2c9b}', 11418), ('\u{2c9d}', 11420), + ('\u{2c9f}', 11422), ('\u{2ca1}', 11424), ('\u{2ca3}', 11426), ('\u{2ca5}', 11428), + ('\u{2ca7}', 11430), ('\u{2ca9}', 11432), ('\u{2cab}', 11434), ('\u{2cad}', 11436), + ('\u{2caf}', 11438), ('\u{2cb1}', 11440), ('\u{2cb3}', 11442), ('\u{2cb5}', 11444), + ('\u{2cb7}', 11446), ('\u{2cb9}', 11448), ('\u{2cbb}', 11450), ('\u{2cbd}', 11452), + ('\u{2cbf}', 11454), ('\u{2cc1}', 11456), ('\u{2cc3}', 11458), ('\u{2cc5}', 11460), + ('\u{2cc7}', 11462), ('\u{2cc9}', 11464), ('\u{2ccb}', 11466), ('\u{2ccd}', 11468), + ('\u{2ccf}', 11470), ('\u{2cd1}', 11472), ('\u{2cd3}', 11474), ('\u{2cd5}', 11476), + ('\u{2cd7}', 11478), ('\u{2cd9}', 11480), ('\u{2cdb}', 11482), ('\u{2cdd}', 11484), + ('\u{2cdf}', 11486), ('\u{2ce1}', 11488), ('\u{2ce3}', 11490), ('\u{2cec}', 11499), + ('\u{2cee}', 11501), ('\u{2cf3}', 11506), ('\u{2d00}', 4256), ('\u{2d01}', 4257), + ('\u{2d02}', 4258), ('\u{2d03}', 4259), ('\u{2d04}', 4260), ('\u{2d05}', 4261), + ('\u{2d06}', 4262), ('\u{2d07}', 4263), ('\u{2d08}', 4264), ('\u{2d09}', 4265), + ('\u{2d0a}', 4266), ('\u{2d0b}', 4267), ('\u{2d0c}', 4268), ('\u{2d0d}', 4269), + ('\u{2d0e}', 4270), ('\u{2d0f}', 4271), ('\u{2d10}', 4272), ('\u{2d11}', 4273), + ('\u{2d12}', 4274), ('\u{2d13}', 4275), ('\u{2d14}', 4276), ('\u{2d15}', 4277), + ('\u{2d16}', 4278), ('\u{2d17}', 4279), ('\u{2d18}', 4280), ('\u{2d19}', 4281), + ('\u{2d1a}', 4282), ('\u{2d1b}', 4283), ('\u{2d1c}', 4284), ('\u{2d1d}', 4285), + ('\u{2d1e}', 4286), ('\u{2d1f}', 4287), ('\u{2d20}', 4288), ('\u{2d21}', 4289), + ('\u{2d22}', 4290), ('\u{2d23}', 4291), ('\u{2d24}', 4292), ('\u{2d25}', 4293), + ('\u{2d27}', 4295), ('\u{2d2d}', 4301), ('\u{a641}', 42560), ('\u{a643}', 42562), + ('\u{a645}', 42564), ('\u{a647}', 42566), ('\u{a649}', 42568), ('\u{a64b}', 42570), + ('\u{a64d}', 42572), ('\u{a64f}', 42574), ('\u{a651}', 42576), ('\u{a653}', 42578), + ('\u{a655}', 42580), ('\u{a657}', 42582), ('\u{a659}', 42584), ('\u{a65b}', 42586), + ('\u{a65d}', 42588), ('\u{a65f}', 42590), ('\u{a661}', 42592), ('\u{a663}', 42594), + ('\u{a665}', 42596), ('\u{a667}', 42598), ('\u{a669}', 42600), ('\u{a66b}', 42602), + ('\u{a66d}', 42604), ('\u{a681}', 42624), ('\u{a683}', 42626), ('\u{a685}', 42628), + ('\u{a687}', 42630), ('\u{a689}', 42632), ('\u{a68b}', 42634), ('\u{a68d}', 42636), + ('\u{a68f}', 42638), ('\u{a691}', 42640), ('\u{a693}', 42642), ('\u{a695}', 42644), + ('\u{a697}', 42646), ('\u{a699}', 42648), ('\u{a69b}', 42650), ('\u{a723}', 42786), + ('\u{a725}', 42788), ('\u{a727}', 42790), ('\u{a729}', 42792), ('\u{a72b}', 42794), + ('\u{a72d}', 42796), ('\u{a72f}', 42798), ('\u{a733}', 42802), ('\u{a735}', 42804), + ('\u{a737}', 42806), ('\u{a739}', 42808), ('\u{a73b}', 42810), ('\u{a73d}', 42812), + ('\u{a73f}', 42814), ('\u{a741}', 42816), ('\u{a743}', 42818), ('\u{a745}', 42820), + ('\u{a747}', 42822), ('\u{a749}', 42824), ('\u{a74b}', 42826), ('\u{a74d}', 42828), + ('\u{a74f}', 42830), ('\u{a751}', 42832), ('\u{a753}', 42834), ('\u{a755}', 42836), + ('\u{a757}', 42838), ('\u{a759}', 42840), ('\u{a75b}', 42842), ('\u{a75d}', 42844), + ('\u{a75f}', 42846), ('\u{a761}', 42848), ('\u{a763}', 42850), ('\u{a765}', 42852), + ('\u{a767}', 42854), ('\u{a769}', 42856), ('\u{a76b}', 42858), ('\u{a76d}', 42860), + ('\u{a76f}', 42862), ('\u{a77a}', 42873), ('\u{a77c}', 42875), ('\u{a77f}', 42878), + ('\u{a781}', 42880), ('\u{a783}', 42882), ('\u{a785}', 42884), ('\u{a787}', 42886), + ('\u{a78c}', 42891), ('\u{a791}', 42896), ('\u{a793}', 42898), ('\u{a794}', 42948), + ('\u{a797}', 42902), ('\u{a799}', 42904), ('\u{a79b}', 42906), ('\u{a79d}', 42908), + ('\u{a79f}', 42910), ('\u{a7a1}', 42912), ('\u{a7a3}', 42914), ('\u{a7a5}', 42916), + ('\u{a7a7}', 42918), ('\u{a7a9}', 42920), ('\u{a7b5}', 42932), ('\u{a7b7}', 42934), + ('\u{a7b9}', 42936), ('\u{a7bb}', 42938), ('\u{a7bd}', 42940), ('\u{a7bf}', 42942), + ('\u{a7c1}', 42944), ('\u{a7c3}', 42946), ('\u{a7c8}', 42951), ('\u{a7ca}', 42953), + ('\u{a7cd}', 42956), ('\u{a7d1}', 42960), ('\u{a7d7}', 42966), ('\u{a7d9}', 42968), + ('\u{a7db}', 42970), ('\u{a7f6}', 42997), ('\u{ab53}', 42931), ('\u{ab70}', 5024), + ('\u{ab71}', 5025), ('\u{ab72}', 5026), ('\u{ab73}', 5027), ('\u{ab74}', 5028), + ('\u{ab75}', 5029), ('\u{ab76}', 5030), ('\u{ab77}', 5031), ('\u{ab78}', 5032), + ('\u{ab79}', 5033), ('\u{ab7a}', 5034), ('\u{ab7b}', 5035), ('\u{ab7c}', 5036), + ('\u{ab7d}', 5037), ('\u{ab7e}', 5038), ('\u{ab7f}', 5039), ('\u{ab80}', 5040), + ('\u{ab81}', 5041), ('\u{ab82}', 5042), ('\u{ab83}', 5043), ('\u{ab84}', 5044), + ('\u{ab85}', 5045), ('\u{ab86}', 5046), ('\u{ab87}', 5047), ('\u{ab88}', 5048), + ('\u{ab89}', 5049), ('\u{ab8a}', 5050), ('\u{ab8b}', 5051), ('\u{ab8c}', 5052), + ('\u{ab8d}', 5053), ('\u{ab8e}', 5054), ('\u{ab8f}', 5055), ('\u{ab90}', 5056), + ('\u{ab91}', 5057), ('\u{ab92}', 5058), ('\u{ab93}', 5059), ('\u{ab94}', 5060), + ('\u{ab95}', 5061), ('\u{ab96}', 5062), ('\u{ab97}', 5063), ('\u{ab98}', 5064), + ('\u{ab99}', 5065), ('\u{ab9a}', 5066), ('\u{ab9b}', 5067), ('\u{ab9c}', 5068), + ('\u{ab9d}', 5069), ('\u{ab9e}', 5070), ('\u{ab9f}', 5071), ('\u{aba0}', 5072), + ('\u{aba1}', 5073), ('\u{aba2}', 5074), ('\u{aba3}', 5075), ('\u{aba4}', 5076), + ('\u{aba5}', 5077), ('\u{aba6}', 5078), ('\u{aba7}', 5079), ('\u{aba8}', 5080), + ('\u{aba9}', 5081), ('\u{abaa}', 5082), ('\u{abab}', 5083), ('\u{abac}', 5084), + ('\u{abad}', 5085), ('\u{abae}', 5086), ('\u{abaf}', 5087), ('\u{abb0}', 5088), + ('\u{abb1}', 5089), ('\u{abb2}', 5090), ('\u{abb3}', 5091), ('\u{abb4}', 5092), + ('\u{abb5}', 5093), ('\u{abb6}', 5094), ('\u{abb7}', 5095), ('\u{abb8}', 5096), + ('\u{abb9}', 5097), ('\u{abba}', 5098), ('\u{abbb}', 5099), ('\u{abbc}', 5100), + ('\u{abbd}', 5101), ('\u{abbe}', 5102), ('\u{abbf}', 5103), ('\u{fb00}', 4194394), + ('\u{fb01}', 4194395), ('\u{fb02}', 4194396), ('\u{fb03}', 4194397), ('\u{fb04}', 4194398), + ('\u{fb05}', 4194399), ('\u{fb06}', 4194400), ('\u{fb13}', 4194401), ('\u{fb14}', 4194402), + ('\u{fb15}', 4194403), ('\u{fb16}', 4194404), ('\u{fb17}', 4194405), ('\u{ff41}', 65313), + ('\u{ff42}', 65314), ('\u{ff43}', 65315), ('\u{ff44}', 65316), ('\u{ff45}', 65317), + ('\u{ff46}', 65318), ('\u{ff47}', 65319), ('\u{ff48}', 65320), ('\u{ff49}', 65321), + ('\u{ff4a}', 65322), ('\u{ff4b}', 65323), ('\u{ff4c}', 65324), ('\u{ff4d}', 65325), + ('\u{ff4e}', 65326), ('\u{ff4f}', 65327), ('\u{ff50}', 65328), ('\u{ff51}', 65329), + ('\u{ff52}', 65330), ('\u{ff53}', 65331), ('\u{ff54}', 65332), ('\u{ff55}', 65333), + ('\u{ff56}', 65334), ('\u{ff57}', 65335), ('\u{ff58}', 65336), ('\u{ff59}', 65337), + ('\u{ff5a}', 65338), ('\u{10428}', 66560), ('\u{10429}', 66561), ('\u{1042a}', 66562), + ('\u{1042b}', 66563), ('\u{1042c}', 66564), ('\u{1042d}', 66565), ('\u{1042e}', 66566), + ('\u{1042f}', 66567), ('\u{10430}', 66568), ('\u{10431}', 66569), ('\u{10432}', 66570), + ('\u{10433}', 66571), ('\u{10434}', 66572), ('\u{10435}', 66573), ('\u{10436}', 66574), + ('\u{10437}', 66575), ('\u{10438}', 66576), ('\u{10439}', 66577), ('\u{1043a}', 66578), + ('\u{1043b}', 66579), ('\u{1043c}', 66580), ('\u{1043d}', 66581), ('\u{1043e}', 66582), + ('\u{1043f}', 66583), ('\u{10440}', 66584), ('\u{10441}', 66585), ('\u{10442}', 66586), + ('\u{10443}', 66587), ('\u{10444}', 66588), ('\u{10445}', 66589), ('\u{10446}', 66590), + ('\u{10447}', 66591), ('\u{10448}', 66592), ('\u{10449}', 66593), ('\u{1044a}', 66594), + ('\u{1044b}', 66595), ('\u{1044c}', 66596), ('\u{1044d}', 66597), ('\u{1044e}', 66598), + ('\u{1044f}', 66599), ('\u{104d8}', 66736), ('\u{104d9}', 66737), ('\u{104da}', 66738), + ('\u{104db}', 66739), ('\u{104dc}', 66740), ('\u{104dd}', 66741), ('\u{104de}', 66742), + ('\u{104df}', 66743), ('\u{104e0}', 66744), ('\u{104e1}', 66745), ('\u{104e2}', 66746), + ('\u{104e3}', 66747), ('\u{104e4}', 66748), ('\u{104e5}', 66749), ('\u{104e6}', 66750), + ('\u{104e7}', 66751), ('\u{104e8}', 66752), ('\u{104e9}', 66753), ('\u{104ea}', 66754), + ('\u{104eb}', 66755), ('\u{104ec}', 66756), ('\u{104ed}', 66757), ('\u{104ee}', 66758), + ('\u{104ef}', 66759), ('\u{104f0}', 66760), ('\u{104f1}', 66761), ('\u{104f2}', 66762), + ('\u{104f3}', 66763), ('\u{104f4}', 66764), ('\u{104f5}', 66765), ('\u{104f6}', 66766), + ('\u{104f7}', 66767), ('\u{104f8}', 66768), ('\u{104f9}', 66769), ('\u{104fa}', 66770), + ('\u{104fb}', 66771), ('\u{10597}', 66928), ('\u{10598}', 66929), ('\u{10599}', 66930), + ('\u{1059a}', 66931), ('\u{1059b}', 66932), ('\u{1059c}', 66933), ('\u{1059d}', 66934), + ('\u{1059e}', 66935), ('\u{1059f}', 66936), ('\u{105a0}', 66937), ('\u{105a1}', 66938), + ('\u{105a3}', 66940), ('\u{105a4}', 66941), ('\u{105a5}', 66942), ('\u{105a6}', 66943), + ('\u{105a7}', 66944), ('\u{105a8}', 66945), ('\u{105a9}', 66946), ('\u{105aa}', 66947), + ('\u{105ab}', 66948), ('\u{105ac}', 66949), ('\u{105ad}', 66950), ('\u{105ae}', 66951), + ('\u{105af}', 66952), ('\u{105b0}', 66953), ('\u{105b1}', 66954), ('\u{105b3}', 66956), + ('\u{105b4}', 66957), ('\u{105b5}', 66958), ('\u{105b6}', 66959), ('\u{105b7}', 66960), + ('\u{105b8}', 66961), ('\u{105b9}', 66962), ('\u{105bb}', 66964), ('\u{105bc}', 66965), + ('\u{10cc0}', 68736), ('\u{10cc1}', 68737), ('\u{10cc2}', 68738), ('\u{10cc3}', 68739), + ('\u{10cc4}', 68740), ('\u{10cc5}', 68741), ('\u{10cc6}', 68742), ('\u{10cc7}', 68743), + ('\u{10cc8}', 68744), ('\u{10cc9}', 68745), ('\u{10cca}', 68746), ('\u{10ccb}', 68747), + ('\u{10ccc}', 68748), ('\u{10ccd}', 68749), ('\u{10cce}', 68750), ('\u{10ccf}', 68751), + ('\u{10cd0}', 68752), ('\u{10cd1}', 68753), ('\u{10cd2}', 68754), ('\u{10cd3}', 68755), + ('\u{10cd4}', 68756), ('\u{10cd5}', 68757), ('\u{10cd6}', 68758), ('\u{10cd7}', 68759), + ('\u{10cd8}', 68760), ('\u{10cd9}', 68761), ('\u{10cda}', 68762), ('\u{10cdb}', 68763), + ('\u{10cdc}', 68764), ('\u{10cdd}', 68765), ('\u{10cde}', 68766), ('\u{10cdf}', 68767), + ('\u{10ce0}', 68768), ('\u{10ce1}', 68769), ('\u{10ce2}', 68770), ('\u{10ce3}', 68771), + ('\u{10ce4}', 68772), ('\u{10ce5}', 68773), ('\u{10ce6}', 68774), ('\u{10ce7}', 68775), + ('\u{10ce8}', 68776), ('\u{10ce9}', 68777), ('\u{10cea}', 68778), ('\u{10ceb}', 68779), + ('\u{10cec}', 68780), ('\u{10ced}', 68781), ('\u{10cee}', 68782), ('\u{10cef}', 68783), + ('\u{10cf0}', 68784), ('\u{10cf1}', 68785), ('\u{10cf2}', 68786), ('\u{10d70}', 68944), + ('\u{10d71}', 68945), ('\u{10d72}', 68946), ('\u{10d73}', 68947), ('\u{10d74}', 68948), + ('\u{10d75}', 68949), ('\u{10d76}', 68950), ('\u{10d77}', 68951), ('\u{10d78}', 68952), + ('\u{10d79}', 68953), ('\u{10d7a}', 68954), ('\u{10d7b}', 68955), ('\u{10d7c}', 68956), + ('\u{10d7d}', 68957), ('\u{10d7e}', 68958), ('\u{10d7f}', 68959), ('\u{10d80}', 68960), + ('\u{10d81}', 68961), ('\u{10d82}', 68962), ('\u{10d83}', 68963), ('\u{10d84}', 68964), + ('\u{10d85}', 68965), ('\u{118c0}', 71840), ('\u{118c1}', 71841), ('\u{118c2}', 71842), + ('\u{118c3}', 71843), ('\u{118c4}', 71844), ('\u{118c5}', 71845), ('\u{118c6}', 71846), + ('\u{118c7}', 71847), ('\u{118c8}', 71848), ('\u{118c9}', 71849), ('\u{118ca}', 71850), + ('\u{118cb}', 71851), ('\u{118cc}', 71852), ('\u{118cd}', 71853), ('\u{118ce}', 71854), + ('\u{118cf}', 71855), ('\u{118d0}', 71856), ('\u{118d1}', 71857), ('\u{118d2}', 71858), + ('\u{118d3}', 71859), ('\u{118d4}', 71860), ('\u{118d5}', 71861), ('\u{118d6}', 71862), + ('\u{118d7}', 71863), ('\u{118d8}', 71864), ('\u{118d9}', 71865), ('\u{118da}', 71866), + ('\u{118db}', 71867), ('\u{118dc}', 71868), ('\u{118dd}', 71869), ('\u{118de}', 71870), + ('\u{118df}', 71871), ('\u{16e60}', 93760), ('\u{16e61}', 93761), ('\u{16e62}', 93762), + ('\u{16e63}', 93763), ('\u{16e64}', 93764), ('\u{16e65}', 93765), ('\u{16e66}', 93766), + ('\u{16e67}', 93767), ('\u{16e68}', 93768), ('\u{16e69}', 93769), ('\u{16e6a}', 93770), + ('\u{16e6b}', 93771), ('\u{16e6c}', 93772), ('\u{16e6d}', 93773), ('\u{16e6e}', 93774), + ('\u{16e6f}', 93775), ('\u{16e70}', 93776), ('\u{16e71}', 93777), ('\u{16e72}', 93778), + ('\u{16e73}', 93779), ('\u{16e74}', 93780), ('\u{16e75}', 93781), ('\u{16e76}', 93782), + ('\u{16e77}', 93783), ('\u{16e78}', 93784), ('\u{16e79}', 93785), ('\u{16e7a}', 93786), + ('\u{16e7b}', 93787), ('\u{16e7c}', 93788), ('\u{16e7d}', 93789), ('\u{16e7e}', 93790), + ('\u{16e7f}', 93791), ('\u{1e922}', 125184), ('\u{1e923}', 125185), ('\u{1e924}', 125186), + ('\u{1e925}', 125187), ('\u{1e926}', 125188), ('\u{1e927}', 125189), ('\u{1e928}', 125190), + ('\u{1e929}', 125191), ('\u{1e92a}', 125192), ('\u{1e92b}', 125193), ('\u{1e92c}', 125194), + ('\u{1e92d}', 125195), ('\u{1e92e}', 125196), ('\u{1e92f}', 125197), ('\u{1e930}', 125198), + ('\u{1e931}', 125199), ('\u{1e932}', 125200), ('\u{1e933}', 125201), ('\u{1e934}', 125202), + ('\u{1e935}', 125203), ('\u{1e936}', 125204), ('\u{1e937}', 125205), ('\u{1e938}', 125206), + ('\u{1e939}', 125207), ('\u{1e93a}', 125208), ('\u{1e93b}', 125209), ('\u{1e93c}', 125210), + ('\u{1e93d}', 125211), ('\u{1e93e}', 125212), ('\u{1e93f}', 125213), ('\u{1e940}', 125214), + ('\u{1e941}', 125215), ('\u{1e942}', 125216), ('\u{1e943}', 125217), ]; static UPPERCASE_TABLE_MULTI: &[[char; 3]] = &[ diff --git a/library/core/tests/ascii.rs b/library/core/tests/ascii.rs index 3d3f8ac10c603..ce09ee507f11f 100644 --- a/library/core/tests/ascii.rs +++ b/library/core/tests/ascii.rs @@ -481,9 +481,25 @@ fn ascii_ctype_const() { } #[test] -fn test_ascii_display() { - assert_eq!(b"foo'bar".escape_ascii().to_string(), r#"foo\'bar"#); - assert_eq!(b"\0\xff".escape_ascii().to_string(), r#"\x00\xff"#); +fn test_escape_ascii() { + let mut buf = [0u8; 0x1F + 7]; // 0..=0x1F plus two quotes, slash, \x7F, \x80, \xFF + for idx in 0..=0x1F { + buf[idx] = idx as u8; + } + buf[0x20] = b'\''; + buf[0x21] = b'"'; + buf[0x22] = b'\\'; + buf[0x23] = 0x7F; + buf[0x24] = 0x80; + buf[0x25] = 0xff; + assert_eq!( + buf.escape_ascii().to_string(), + r#"\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\'\"\\\x7f\x80\xff"# + ); +} + +#[test] +fn test_escape_ascii_iter() { let mut it = b"\0fastpath\xffremainder\xff".escape_ascii(); let _ = it.advance_by(4); let _ = it.advance_back_by(4); diff --git a/library/core/tests/atomic.rs b/library/core/tests/atomic.rs index 0d1c72a689291..0ffba538b2074 100644 --- a/library/core/tests/atomic.rs +++ b/library/core/tests/atomic.rs @@ -228,6 +228,8 @@ fn static_init() { } #[test] +// FIXME(static_mut_refs): Do not allow `static_mut_refs` lint +#[allow(static_mut_refs)] fn atomic_access_bool() { static mut ATOMIC: AtomicBool = AtomicBool::new(false); diff --git a/library/core/tests/error.rs b/library/core/tests/error.rs index 5e20c34ca6ced..996566d3848bd 100644 --- a/library/core/tests/error.rs +++ b/library/core/tests/error.rs @@ -1,4 +1,4 @@ -use core::error::{request_ref, request_value, Request}; +use core::error::{Request, request_ref, request_value}; // Test the `Request` API. #[derive(Debug)] diff --git a/library/core/tests/future.rs b/library/core/tests/future.rs index 93aca72d59082..ebfe5a0a66dc5 100644 --- a/library/core/tests/future.rs +++ b/library/core/tests/future.rs @@ -1,4 +1,4 @@ -use std::future::{join, Future}; +use std::future::{Future, join}; use std::pin::Pin; use std::sync::Arc; use std::task::{Context, Poll, Wake}; diff --git a/library/core/tests/hash/mod.rs b/library/core/tests/hash/mod.rs index bdd1c2579ded8..bf91e9e5df0e2 100644 --- a/library/core/tests/hash/mod.rs +++ b/library/core/tests/hash/mod.rs @@ -16,11 +16,8 @@ impl Default for MyHasher { impl Hasher for MyHasher { fn write(&mut self, buf: &[u8]) { - // FIXME(const_trait_impl): change to for loop - let mut i = 0; - while i < buf.len() { - self.hash += buf[i] as u64; - i += 1; + for byte in buf { + self.hash += *byte as u64; } } fn write_str(&mut self, s: &str) { @@ -167,7 +164,7 @@ fn test_indirect_hasher() { } #[test] -fn test_build_hasher_object_safe() { +fn test_build_hasher_dyn_compatible() { use std::hash::{DefaultHasher, RandomState}; let _: &dyn BuildHasher = &RandomState::new(); diff --git a/library/core/tests/iter/adapters/map_windows.rs b/library/core/tests/iter/adapters/map_windows.rs index 01cebc9b27fd8..b677f1cfd55e7 100644 --- a/library/core/tests/iter/adapters/map_windows.rs +++ b/library/core/tests/iter/adapters/map_windows.rs @@ -159,10 +159,11 @@ fn output_n2() { >::new(), ); assert_eq!("ab".chars().map_windows(|a: &[_; 2]| *a).collect::>(), vec![['a', 'b']]); - assert_eq!( - "abcd".chars().map_windows(|a: &[_; 2]| *a).collect::>(), - vec![['a', 'b'], ['b', 'c'], ['c', 'd']], - ); + assert_eq!("abcd".chars().map_windows(|a: &[_; 2]| *a).collect::>(), vec![ + ['a', 'b'], + ['b', 'c'], + ['c', 'd'] + ],); } #[test] diff --git a/library/core/tests/iter/adapters/zip.rs b/library/core/tests/iter/adapters/zip.rs index 94b49bac45337..70392dca0fa17 100644 --- a/library/core/tests/iter/adapters/zip.rs +++ b/library/core/tests/iter/adapters/zip.rs @@ -240,7 +240,7 @@ fn test_zip_trusted_random_access_composition() { #[test] #[cfg(panic = "unwind")] fn test_zip_trusted_random_access_next_back_drop() { - use std::panic::{catch_unwind, AssertUnwindSafe}; + use std::panic::{AssertUnwindSafe, catch_unwind}; let mut counter = 0; diff --git a/library/core/tests/iter/sources.rs b/library/core/tests/iter/sources.rs index eb8c80dd08724..506febaa056a8 100644 --- a/library/core/tests/iter/sources.rs +++ b/library/core/tests/iter/sources.rs @@ -156,3 +156,27 @@ fn test_repeat_n_drop() { drop((x0, x1, x2)); assert_eq!(count.get(), 3); } + +#[test] +fn test_repeat_n_soundness() { + let x = std::iter::repeat_n(String::from("use after free"), 0); + println!("{x:?}"); + + pub struct PanicOnClone; + + impl Clone for PanicOnClone { + fn clone(&self) -> Self { + unreachable!() + } + } + + // `repeat_n` should drop the element immediately if `count` is zero. + // `Clone` should then not try to clone the element. + let x = std::iter::repeat_n(PanicOnClone, 0); + let _ = x.clone(); + + let mut y = std::iter::repeat_n(Box::new(0), 1); + let x = y.next().unwrap(); + let _z = y; + assert_eq!(0, *x); +} diff --git a/library/core/tests/lazy.rs b/library/core/tests/lazy.rs index a3b89f15b3f81..32d0ac51f0336 100644 --- a/library/core/tests/lazy.rs +++ b/library/core/tests/lazy.rs @@ -113,6 +113,28 @@ fn lazy_type_inference() { let _ = LazyCell::new(|| ()); } +#[test] +#[cfg(panic = "unwind")] +#[should_panic = "LazyCell instance has previously been poisoned"] +fn lazy_force_mut_panic() { + let mut lazy = LazyCell::::new(|| panic!()); + std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { + let _ = LazyCell::force_mut(&mut lazy); + })) + .unwrap_err(); + let _ = &*lazy; +} + +#[test] +fn lazy_force_mut() { + let s = "abc".to_owned(); + let mut lazy = LazyCell::new(move || s); + LazyCell::force_mut(&mut lazy); + let p = LazyCell::force_mut(&mut lazy); + p.clear(); + LazyCell::force_mut(&mut lazy); +} + #[test] fn aliasing_in_get() { let x = OnceCell::new(); diff --git a/library/core/tests/lib.rs b/library/core/tests/lib.rs index dbceb8abafc84..2a9f1660a629e 100644 --- a/library/core/tests/lib.rs +++ b/library/core/tests/lib.rs @@ -1,4 +1,6 @@ // tidy-alphabetical-start +#![cfg_attr(bootstrap, feature(strict_provenance))] +#![cfg_attr(not(bootstrap), feature(strict_provenance_lints))] #![cfg_attr(target_has_atomic = "128", feature(integer_atomics))] #![cfg_attr(test, feature(cfg_match))] #![feature(alloc_layout_extra)] @@ -15,38 +17,25 @@ #![feature(clone_to_uninit)] #![feature(const_align_of_val_raw)] #![feature(const_align_offset)] -#![feature(const_array_from_ref)] +#![feature(const_bigint_helper_methods)] #![feature(const_black_box)] -#![feature(const_cell_into_inner)] +#![feature(const_eval_select)] #![feature(const_hash)] #![feature(const_heap)] -#![feature(const_intrinsic_copy)] -#![feature(const_ip)] -#![feature(const_ipv4)] -#![feature(const_ipv6)] -#![feature(const_likely)] -#![feature(const_maybe_uninit_as_mut_ptr)] -#![feature(const_mut_refs)] #![feature(const_nonnull_new)] -#![feature(const_option)] +#![feature(const_num_midpoint)] #![feature(const_option_ext)] -#![feature(const_pin)] +#![feature(const_pin_2)] #![feature(const_pointer_is_aligned)] -#![feature(const_ptr_as_ref)] -#![feature(const_ptr_write)] -#![feature(const_result)] -#![feature(const_slice_from_ref)] #![feature(const_three_way_compare)] #![feature(const_trait_impl)] #![feature(core_intrinsics)] #![feature(core_io_borrowed_buf)] #![feature(core_private_bignum)] #![feature(core_private_diy_float)] -#![feature(debug_more_non_exhaustive)] #![feature(dec2flt)] #![feature(duration_constants)] #![feature(duration_constructors)] -#![feature(duration_consts_float)] #![feature(error_generic_member_access)] #![feature(exact_size_is_empty)] #![feature(extern_types)] @@ -59,10 +48,11 @@ #![feature(get_many_mut)] #![feature(hasher_prefixfree_extras)] #![feature(hashmap_internals)] +#![feature(inline_const_pat)] #![feature(int_roundings)] #![feature(ip)] +#![feature(ip_from)] #![feature(is_ascii_octdigit)] -#![feature(isqrt)] #![feature(iter_advance_by)] #![feature(iter_array_chunks)] #![feature(iter_chain)] @@ -76,6 +66,7 @@ #![feature(iterator_try_collect)] #![feature(iterator_try_reduce)] #![feature(layout_for_ptr)] +#![feature(lazy_get)] #![feature(maybe_uninit_fill)] #![feature(maybe_uninit_uninit_array_transpose)] #![feature(maybe_uninit_write_slice)] @@ -98,7 +89,6 @@ #![feature(std_internals)] #![feature(step_trait)] #![feature(str_internals)] -#![feature(strict_provenance)] #![feature(strict_provenance_atomic_ptr)] #![feature(test)] #![feature(trait_upcasting)] @@ -116,6 +106,37 @@ #![deny(fuzzy_provenance_casts)] #![deny(unsafe_op_in_unsafe_fn)] +/// Version of `assert_matches` that ignores fancy runtime printing in const context and uses structural equality. +macro_rules! assert_eq_const_safe { + ($left:expr, $right:expr$(, $($arg:tt)+)?) => { + { + fn runtime() { + assert_eq!($left, $right, $($arg)*); + } + const fn compiletime() { + assert!(matches!($left, const { $right })); + } + core::intrinsics::const_eval_select((), compiletime, runtime) + } + }; +} + +/// Creates a test for runtime and a test for constant-time. +macro_rules! test_runtime_and_compiletime { + ($( + $(#[$attr:meta])* + fn $test:ident() $block:block + )*) => { + $( + $(#[$attr])* + #[test] + fn $test() $block + $(#[$attr])* + const _: () = $block; + )* + } +} + mod alloc; mod any; mod array; diff --git a/library/core/tests/macros.rs b/library/core/tests/macros.rs index 09994fbcbdb78..fdb4ea2941285 100644 --- a/library/core/tests/macros.rs +++ b/library/core/tests/macros.rs @@ -1,3 +1,5 @@ +#![allow(unused_must_use)] + #[allow(dead_code)] trait Trait { fn blah(&self); @@ -173,3 +175,21 @@ fn cfg_match_two_functions() { bar2(); } } + +fn _accepts_expressions() -> i32 { + cfg_match! { + cfg(unix) => { 1 } + _ => { 2 } + } +} + +// The current implementation expands to a macro call, which allows the use of expression +// statements. +fn _allows_stmt_expr_attributes() { + let one = 1; + let two = 2; + cfg_match! { + cfg(unix) => { one * two; } + _ => { one + two; } + } +} diff --git a/library/core/tests/mem.rs b/library/core/tests/mem.rs index b7eee10ec3f9c..f3b4387f6a898 100644 --- a/library/core/tests/mem.rs +++ b/library/core/tests/mem.rs @@ -773,15 +773,20 @@ fn offset_of_addr() { #[test] fn const_maybe_uninit_zeroed() { // Sanity check for `MaybeUninit::zeroed` in a realistic const situation (plugin array term) + + // It is crucial that this type has no padding! #[repr(C)] struct Foo { - a: Option<&'static str>, + a: Option<&'static u8>, b: Bar, c: f32, + _pad: u32, d: *const u8, } + #[repr(C)] struct Bar(usize); + struct FooPtr(*const Foo); unsafe impl Sync for FooPtr {} diff --git a/library/core/tests/net/ip_addr.rs b/library/core/tests/net/ip_addr.rs index a10b51c550d5b..707f9a160e127 100644 --- a/library/core/tests/net/ip_addr.rs +++ b/library/core/tests/net/ip_addr.rs @@ -494,6 +494,7 @@ fn ipv6_properties() { let octets = &[$($octet),*]; assert_eq!(&ip!($s).octets(), octets); assert_eq!(Ipv6Addr::from(*octets), ip!($s)); + assert_eq!(Ipv6Addr::from_octets(*octets), ip!($s)); let unspecified: u32 = 1 << 0; let loopback: u32 = 1 << 1; @@ -846,15 +847,19 @@ fn ipv6_from_constructors() { #[test] fn ipv4_from_octets() { - assert_eq!(Ipv4Addr::from([127, 0, 0, 1]), Ipv4Addr::new(127, 0, 0, 1)) + assert_eq!(Ipv4Addr::from([127, 0, 0, 1]), Ipv4Addr::new(127, 0, 0, 1)); + assert_eq!(Ipv4Addr::from_octets([127, 0, 0, 1]), Ipv4Addr::new(127, 0, 0, 1)); } #[test] fn ipv6_from_segments() { let from_u16s = Ipv6Addr::from([0x0011, 0x2233, 0x4455, 0x6677, 0x8899, 0xaabb, 0xccdd, 0xeeff]); + let from_u16s_explicit = + Ipv6Addr::from_segments([0x0011, 0x2233, 0x4455, 0x6677, 0x8899, 0xaabb, 0xccdd, 0xeeff]); let new = Ipv6Addr::new(0x0011, 0x2233, 0x4455, 0x6677, 0x8899, 0xaabb, 0xccdd, 0xeeff); assert_eq!(new, from_u16s); + assert_eq!(new, from_u16s_explicit); } #[test] @@ -865,7 +870,15 @@ fn ipv6_from_octets() { 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff, ]); + let from_u16s_explicit = + Ipv6Addr::from_segments([0x0011, 0x2233, 0x4455, 0x6677, 0x8899, 0xaabb, 0xccdd, 0xeeff]); + let from_u8s_explicit = Ipv6Addr::from_octets([ + 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, + 0xff, + ]); assert_eq!(from_u16s, from_u8s); + assert_eq!(from_u16s, from_u16s_explicit); + assert_eq!(from_u16s_explicit, from_u8s_explicit); } #[test] @@ -915,6 +928,9 @@ fn ipv4_const() { const OCTETS: [u8; 4] = IP_ADDRESS.octets(); assert_eq!(OCTETS, [127, 0, 0, 1]); + const FROM_OCTETS: Ipv4Addr = Ipv4Addr::from_octets(OCTETS); + assert_eq!(IP_ADDRESS, FROM_OCTETS); + const IS_UNSPECIFIED: bool = IP_ADDRESS.is_unspecified(); assert!(!IS_UNSPECIFIED); @@ -971,9 +987,15 @@ fn ipv6_const() { const SEGMENTS: [u16; 8] = IP_ADDRESS.segments(); assert_eq!(SEGMENTS, [0, 0, 0, 0, 0, 0, 0, 1]); + const FROM_SEGMENTS: Ipv6Addr = Ipv6Addr::from_segments(SEGMENTS); + assert_eq!(IP_ADDRESS, FROM_SEGMENTS); + const OCTETS: [u8; 16] = IP_ADDRESS.octets(); assert_eq!(OCTETS, [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]); + const FROM_OCTETS: Ipv6Addr = Ipv6Addr::from_octets(OCTETS); + assert_eq!(IP_ADDRESS, FROM_OCTETS); + const IS_UNSPECIFIED: bool = IP_ADDRESS.is_unspecified(); assert!(!IS_UNSPECIFIED); diff --git a/library/core/tests/num/bignum.rs b/library/core/tests/num/bignum.rs index 416e7cea7a67b..f213fd5366cbf 100644 --- a/library/core/tests/num/bignum.rs +++ b/library/core/tests/num/bignum.rs @@ -1,5 +1,5 @@ -use core::num::bignum::tests::Big8x3 as Big; use core::num::bignum::Big32x40; +use core::num::bignum::tests::Big8x3 as Big; #[test] #[should_panic] diff --git a/library/core/tests/num/flt2dec/mod.rs b/library/core/tests/num/flt2dec/mod.rs index 070a53edc2e0f..3d82522481316 100644 --- a/library/core/tests/num/flt2dec/mod.rs +++ b/library/core/tests/num/flt2dec/mod.rs @@ -1,6 +1,6 @@ use core::num::flt2dec::{ - decode, round_up, to_exact_exp_str, to_exact_fixed_str, to_shortest_exp_str, to_shortest_str, - DecodableFloat, Decoded, FullDecoded, Sign, MAX_SIG_DIGITS, + DecodableFloat, Decoded, FullDecoded, MAX_SIG_DIGITS, Sign, decode, round_up, to_exact_exp_str, + to_exact_fixed_str, to_shortest_exp_str, to_shortest_str, }; use core::num::fmt::{Formatted, Part}; use std::mem::MaybeUninit; diff --git a/library/core/tests/num/flt2dec/random.rs b/library/core/tests/num/flt2dec/random.rs index 4817a66638391..99fc23af7ea9d 100644 --- a/library/core/tests/num/flt2dec/random.rs +++ b/library/core/tests/num/flt2dec/random.rs @@ -1,7 +1,7 @@ #![cfg(not(target_arch = "wasm32"))] use core::num::flt2dec::strategy::grisu::{format_exact_opt, format_shortest_opt}; -use core::num::flt2dec::{decode, DecodableFloat, Decoded, FullDecoded, MAX_SIG_DIGITS}; +use core::num::flt2dec::{DecodableFloat, Decoded, FullDecoded, MAX_SIG_DIGITS, decode}; use std::mem::MaybeUninit; use std::str; diff --git a/library/core/tests/num/int_macros.rs b/library/core/tests/num/int_macros.rs index 830a96204ca03..474d57049ab65 100644 --- a/library/core/tests/num/int_macros.rs +++ b/library/core/tests/num/int_macros.rs @@ -17,42 +17,6 @@ macro_rules! int_module { num::test_num(10 as $T, 2 as $T); } - #[test] - fn test_rem_euclid() { - assert_eq!((-1 as $T).rem_euclid(MIN), MAX); - } - - #[test] - pub fn test_abs() { - assert_eq!((1 as $T).abs(), 1 as $T); - assert_eq!((0 as $T).abs(), 0 as $T); - assert_eq!((-1 as $T).abs(), 1 as $T); - } - - #[test] - fn test_signum() { - assert_eq!((1 as $T).signum(), 1 as $T); - assert_eq!((0 as $T).signum(), 0 as $T); - assert_eq!((-0 as $T).signum(), 0 as $T); - assert_eq!((-1 as $T).signum(), -1 as $T); - } - - #[test] - fn test_is_positive() { - assert!((1 as $T).is_positive()); - assert!(!(0 as $T).is_positive()); - assert!(!(-0 as $T).is_positive()); - assert!(!(-1 as $T).is_positive()); - } - - #[test] - fn test_is_negative() { - assert!(!(1 as $T).is_negative()); - assert!(!(0 as $T).is_negative()); - assert!(!(-0 as $T).is_negative()); - assert!((-1 as $T).is_negative()); - } - #[test] fn test_bitwise_operators() { assert_eq!(0b1110 as $T, (0b1100 as $T).bitor(0b1010 as $T)); @@ -63,6 +27,40 @@ macro_rules! int_module { assert_eq!(-(0b11 as $T) - (1 as $T), (0b11 as $T).not()); } + test_runtime_and_compiletime! { + + fn test_rem_euclid() { + assert_eq_const_safe!((-1 as $T).rem_euclid(MIN), MAX); + } + + fn test_abs() { + assert_eq_const_safe!((1 as $T).abs(), 1 as $T); + assert_eq_const_safe!((0 as $T).abs(), 0 as $T); + assert_eq_const_safe!((-1 as $T).abs(), 1 as $T); + } + + fn test_signum() { + assert_eq_const_safe!((1 as $T).signum(), 1 as $T); + assert_eq_const_safe!((0 as $T).signum(), 0 as $T); + assert_eq_const_safe!((-0 as $T).signum(), 0 as $T); + assert_eq_const_safe!((-1 as $T).signum(), -1 as $T); + } + + fn test_is_positive() { + assert!((1 as $T).is_positive()); + assert!(!(0 as $T).is_positive()); + assert!(!(-0 as $T).is_positive()); + assert!(!(-1 as $T).is_positive()); + } + + fn test_is_negative() { + assert!(!(1 as $T).is_negative()); + assert!(!(0 as $T).is_negative()); + assert!(!(-0 as $T).is_negative()); + assert!((-1 as $T).is_negative()); + } + } + const A: $T = 0b0101100; const B: $T = 0b0100001; const C: $T = 0b1111001; @@ -70,134 +68,126 @@ macro_rules! int_module { const _0: $T = 0; const _1: $T = !0; - #[test] - fn test_count_ones() { - assert_eq!(A.count_ones(), 3); - assert_eq!(B.count_ones(), 2); - assert_eq!(C.count_ones(), 5); - } + test_runtime_and_compiletime! { + fn test_count_ones() { + assert_eq_const_safe!(A.count_ones(), 3); + assert_eq_const_safe!(B.count_ones(), 2); + assert_eq_const_safe!(C.count_ones(), 5); + } - #[test] - fn test_count_zeros() { - assert_eq!(A.count_zeros(), $T::BITS - 3); - assert_eq!(B.count_zeros(), $T::BITS - 2); - assert_eq!(C.count_zeros(), $T::BITS - 5); - } + fn test_count_zeros() { + assert_eq_const_safe!(A.count_zeros(), $T::BITS - 3); + assert_eq_const_safe!(B.count_zeros(), $T::BITS - 2); + assert_eq_const_safe!(C.count_zeros(), $T::BITS - 5); + } - #[test] - fn test_leading_trailing_ones() { - let a: $T = 0b0101_1111; - assert_eq!(a.trailing_ones(), 5); - assert_eq!((!a).leading_ones(), $T::BITS - 7); + fn test_leading_trailing_ones() { + const A: $T = 0b0101_1111; + assert_eq_const_safe!(A.trailing_ones(), 5); + assert_eq_const_safe!((!A).leading_ones(), $T::BITS - 7); - assert_eq!(a.reverse_bits().leading_ones(), 5); + assert_eq_const_safe!(A.reverse_bits().leading_ones(), 5); - assert_eq!(_1.leading_ones(), $T::BITS); - assert_eq!(_1.trailing_ones(), $T::BITS); + assert_eq_const_safe!(_1.leading_ones(), $T::BITS); + assert_eq_const_safe!(_1.trailing_ones(), $T::BITS); - assert_eq!((_1 << 1).trailing_ones(), 0); - assert_eq!(MAX.leading_ones(), 0); + assert_eq_const_safe!((_1 << 1).trailing_ones(), 0); + assert_eq_const_safe!(MAX.leading_ones(), 0); - assert_eq!((_1 << 1).leading_ones(), $T::BITS - 1); - assert_eq!(MAX.trailing_ones(), $T::BITS - 1); + assert_eq_const_safe!((_1 << 1).leading_ones(), $T::BITS - 1); + assert_eq_const_safe!(MAX.trailing_ones(), $T::BITS - 1); - assert_eq!(_0.leading_ones(), 0); - assert_eq!(_0.trailing_ones(), 0); + assert_eq_const_safe!(_0.leading_ones(), 0); + assert_eq_const_safe!(_0.trailing_ones(), 0); - let x: $T = 0b0010_1100; - assert_eq!(x.leading_ones(), 0); - assert_eq!(x.trailing_ones(), 0); - } + const X: $T = 0b0010_1100; + assert_eq_const_safe!(X.leading_ones(), 0); + assert_eq_const_safe!(X.trailing_ones(), 0); + } - #[test] - fn test_rotate() { - assert_eq!(A.rotate_left(6).rotate_right(2).rotate_right(4), A); - assert_eq!(B.rotate_left(3).rotate_left(2).rotate_right(5), B); - assert_eq!(C.rotate_left(6).rotate_right(2).rotate_right(4), C); - - // Rotating these should make no difference - // - // We test using 124 bits because to ensure that overlong bit shifts do - // not cause undefined behaviour. See #10183. - assert_eq!(_0.rotate_left(124), _0); - assert_eq!(_1.rotate_left(124), _1); - assert_eq!(_0.rotate_right(124), _0); - assert_eq!(_1.rotate_right(124), _1); - - // Rotating by 0 should have no effect - assert_eq!(A.rotate_left(0), A); - assert_eq!(B.rotate_left(0), B); - assert_eq!(C.rotate_left(0), C); - // Rotating by a multiple of word size should also have no effect - assert_eq!(A.rotate_left(128), A); - assert_eq!(B.rotate_left(128), B); - assert_eq!(C.rotate_left(128), C); - } + fn test_rotate() { + assert_eq_const_safe!(A.rotate_left(6).rotate_right(2).rotate_right(4), A); + assert_eq_const_safe!(B.rotate_left(3).rotate_left(2).rotate_right(5), B); + assert_eq_const_safe!(C.rotate_left(6).rotate_right(2).rotate_right(4), C); + + // Rotating these should make no difference + // + // We test using 124 bits because to ensure that overlong bit shifts do + // not cause undefined behavior. See #10183. + assert_eq_const_safe!(_0.rotate_left(124), _0); + assert_eq_const_safe!(_1.rotate_left(124), _1); + assert_eq_const_safe!(_0.rotate_right(124), _0); + assert_eq_const_safe!(_1.rotate_right(124), _1); + + // Rotating by 0 should have no effect + assert_eq_const_safe!(A.rotate_left(0), A); + assert_eq_const_safe!(B.rotate_left(0), B); + assert_eq_const_safe!(C.rotate_left(0), C); + // Rotating by a multiple of word size should also have no effect + assert_eq_const_safe!(A.rotate_left(128), A); + assert_eq_const_safe!(B.rotate_left(128), B); + assert_eq_const_safe!(C.rotate_left(128), C); + } - #[test] - fn test_swap_bytes() { - assert_eq!(A.swap_bytes().swap_bytes(), A); - assert_eq!(B.swap_bytes().swap_bytes(), B); - assert_eq!(C.swap_bytes().swap_bytes(), C); - - // Swapping these should make no difference - assert_eq!(_0.swap_bytes(), _0); - assert_eq!(_1.swap_bytes(), _1); - } + fn test_swap_bytes() { + assert_eq_const_safe!(A.swap_bytes().swap_bytes(), A); + assert_eq_const_safe!(B.swap_bytes().swap_bytes(), B); + assert_eq_const_safe!(C.swap_bytes().swap_bytes(), C); - #[test] - fn test_le() { - assert_eq!($T::from_le(A.to_le()), A); - assert_eq!($T::from_le(B.to_le()), B); - assert_eq!($T::from_le(C.to_le()), C); - assert_eq!($T::from_le(_0), _0); - assert_eq!($T::from_le(_1), _1); - assert_eq!(_0.to_le(), _0); - assert_eq!(_1.to_le(), _1); - } + // Swapping these should make no difference + assert_eq_const_safe!(_0.swap_bytes(), _0); + assert_eq_const_safe!(_1.swap_bytes(), _1); + } - #[test] - fn test_be() { - assert_eq!($T::from_be(A.to_be()), A); - assert_eq!($T::from_be(B.to_be()), B); - assert_eq!($T::from_be(C.to_be()), C); - assert_eq!($T::from_be(_0), _0); - assert_eq!($T::from_be(_1), _1); - assert_eq!(_0.to_be(), _0); - assert_eq!(_1.to_be(), _1); - } + fn test_le() { + assert_eq_const_safe!($T::from_le(A.to_le()), A); + assert_eq_const_safe!($T::from_le(B.to_le()), B); + assert_eq_const_safe!($T::from_le(C.to_le()), C); + assert_eq_const_safe!($T::from_le(_0), _0); + assert_eq_const_safe!($T::from_le(_1), _1); + assert_eq_const_safe!(_0.to_le(), _0); + assert_eq_const_safe!(_1.to_le(), _1); + } - #[test] - fn test_signed_checked_div() { - assert_eq!((10 as $T).checked_div(2), Some(5)); - assert_eq!((5 as $T).checked_div(0), None); - assert_eq!(isize::MIN.checked_div(-1), None); - } + fn test_be() { + assert_eq_const_safe!($T::from_be(A.to_be()), A); + assert_eq_const_safe!($T::from_be(B.to_be()), B); + assert_eq_const_safe!($T::from_be(C.to_be()), C); + assert_eq_const_safe!($T::from_be(_0), _0); + assert_eq_const_safe!($T::from_be(_1), _1); + assert_eq_const_safe!(_0.to_be(), _0); + assert_eq_const_safe!(_1.to_be(), _1); + } - #[test] - fn test_saturating_abs() { - assert_eq!((0 as $T).saturating_abs(), 0); - assert_eq!((123 as $T).saturating_abs(), 123); - assert_eq!((-123 as $T).saturating_abs(), 123); - assert_eq!((MAX - 2).saturating_abs(), MAX - 2); - assert_eq!((MAX - 1).saturating_abs(), MAX - 1); - assert_eq!(MAX.saturating_abs(), MAX); - assert_eq!((MIN + 2).saturating_abs(), MAX - 1); - assert_eq!((MIN + 1).saturating_abs(), MAX); - assert_eq!(MIN.saturating_abs(), MAX); - } + fn test_signed_checked_div() { + assert_eq_const_safe!((10 as $T).checked_div(2), Some(5)); + assert_eq_const_safe!((5 as $T).checked_div(0), None); + assert_eq_const_safe!(isize::MIN.checked_div(-1), None); + } - #[test] - fn test_saturating_neg() { - assert_eq!((0 as $T).saturating_neg(), 0); - assert_eq!((123 as $T).saturating_neg(), -123); - assert_eq!((-123 as $T).saturating_neg(), 123); - assert_eq!((MAX - 2).saturating_neg(), MIN + 3); - assert_eq!((MAX - 1).saturating_neg(), MIN + 2); - assert_eq!(MAX.saturating_neg(), MIN + 1); - assert_eq!((MIN + 2).saturating_neg(), MAX - 1); - assert_eq!((MIN + 1).saturating_neg(), MAX); - assert_eq!(MIN.saturating_neg(), MAX); + fn test_saturating_abs() { + assert_eq_const_safe!((0 as $T).saturating_abs(), 0); + assert_eq_const_safe!((123 as $T).saturating_abs(), 123); + assert_eq_const_safe!((-123 as $T).saturating_abs(), 123); + assert_eq_const_safe!((MAX - 2).saturating_abs(), MAX - 2); + assert_eq_const_safe!((MAX - 1).saturating_abs(), MAX - 1); + assert_eq_const_safe!(MAX.saturating_abs(), MAX); + assert_eq_const_safe!((MIN + 2).saturating_abs(), MAX - 1); + assert_eq_const_safe!((MIN + 1).saturating_abs(), MAX); + assert_eq_const_safe!(MIN.saturating_abs(), MAX); + } + + fn test_saturating_neg() { + assert_eq_const_safe!((0 as $T).saturating_neg(), 0); + assert_eq_const_safe!((123 as $T).saturating_neg(), -123); + assert_eq_const_safe!((-123 as $T).saturating_neg(), 123); + assert_eq_const_safe!((MAX - 2).saturating_neg(), MIN + 3); + assert_eq_const_safe!((MAX - 1).saturating_neg(), MIN + 2); + assert_eq_const_safe!(MAX.saturating_neg(), MIN + 1); + assert_eq_const_safe!((MIN + 2).saturating_neg(), MAX - 1); + assert_eq_const_safe!((MIN + 1).saturating_neg(), MAX); + assert_eq_const_safe!(MIN.saturating_neg(), MAX); + } } #[test] @@ -222,171 +212,173 @@ macro_rules! int_module { assert_eq!(from_str::<$T>("x"), None); } - #[test] - fn test_from_str_radix() { - assert_eq!($T::from_str_radix("123", 10), Ok(123 as $T)); - assert_eq!($T::from_str_radix("1001", 2), Ok(9 as $T)); - assert_eq!($T::from_str_radix("123", 8), Ok(83 as $T)); - assert_eq!(i32::from_str_radix("123", 16), Ok(291 as i32)); - assert_eq!(i32::from_str_radix("ffff", 16), Ok(65535 as i32)); - assert_eq!(i32::from_str_radix("FFFF", 16), Ok(65535 as i32)); - assert_eq!($T::from_str_radix("z", 36), Ok(35 as $T)); - assert_eq!($T::from_str_radix("Z", 36), Ok(35 as $T)); - - assert_eq!($T::from_str_radix("-123", 10), Ok(-123 as $T)); - assert_eq!($T::from_str_radix("-1001", 2), Ok(-9 as $T)); - assert_eq!($T::from_str_radix("-123", 8), Ok(-83 as $T)); - assert_eq!(i32::from_str_radix("-123", 16), Ok(-291 as i32)); - assert_eq!(i32::from_str_radix("-ffff", 16), Ok(-65535 as i32)); - assert_eq!(i32::from_str_radix("-FFFF", 16), Ok(-65535 as i32)); - assert_eq!($T::from_str_radix("-z", 36), Ok(-35 as $T)); - assert_eq!($T::from_str_radix("-Z", 36), Ok(-35 as $T)); - - assert_eq!($T::from_str_radix("Z", 35).ok(), None::<$T>); - assert_eq!($T::from_str_radix("-9", 2).ok(), None::<$T>); - } + test_runtime_and_compiletime! { + fn test_from_str_radix() { + assert_eq_const_safe!($T::from_str_radix("123", 10), Ok(123 as $T)); + assert_eq_const_safe!($T::from_str_radix("1001", 2), Ok(9 as $T)); + assert_eq_const_safe!($T::from_str_radix("123", 8), Ok(83 as $T)); + assert_eq_const_safe!(i32::from_str_radix("123", 16), Ok(291 as i32)); + assert_eq_const_safe!(i32::from_str_radix("ffff", 16), Ok(65535 as i32)); + assert_eq_const_safe!(i32::from_str_radix("FFFF", 16), Ok(65535 as i32)); + assert_eq_const_safe!($T::from_str_radix("z", 36), Ok(35 as $T)); + assert_eq_const_safe!($T::from_str_radix("Z", 36), Ok(35 as $T)); + + assert_eq_const_safe!($T::from_str_radix("-123", 10), Ok(-123 as $T)); + assert_eq_const_safe!($T::from_str_radix("-1001", 2), Ok(-9 as $T)); + assert_eq_const_safe!($T::from_str_radix("-123", 8), Ok(-83 as $T)); + assert_eq_const_safe!(i32::from_str_radix("-123", 16), Ok(-291 as i32)); + assert_eq_const_safe!(i32::from_str_radix("-ffff", 16), Ok(-65535 as i32)); + assert_eq_const_safe!(i32::from_str_radix("-FFFF", 16), Ok(-65535 as i32)); + assert_eq_const_safe!($T::from_str_radix("-z", 36), Ok(-35 as $T)); + assert_eq_const_safe!($T::from_str_radix("-Z", 36), Ok(-35 as $T)); + + assert!($T::from_str_radix("Z", 35).is_err()); + assert!($T::from_str_radix("-9", 2).is_err()); + assert!($T::from_str_radix("10_0", 10).is_err()); + assert!(u32::from_str_radix("-9", 10).is_err()); + } - #[test] - fn test_pow() { - let mut r = 2 as $T; - assert_eq!(r.pow(2), 4 as $T); - assert_eq!(r.pow(0), 1 as $T); - assert_eq!(r.wrapping_pow(2), 4 as $T); - assert_eq!(r.wrapping_pow(0), 1 as $T); - assert_eq!(r.checked_pow(2), Some(4 as $T)); - assert_eq!(r.checked_pow(0), Some(1 as $T)); - assert_eq!(r.overflowing_pow(2), (4 as $T, false)); - assert_eq!(r.overflowing_pow(0), (1 as $T, false)); - assert_eq!(r.saturating_pow(2), 4 as $T); - assert_eq!(r.saturating_pow(0), 1 as $T); - - r = MAX; - // use `^` to represent .pow() with no overflow. - // if itest::MAX == 2^j-1, then itest is a `j` bit int, - // so that `itest::MAX*itest::MAX == 2^(2*j)-2^(j+1)+1`, - // thussaturating_pow the overflowing result is exactly 1. - assert_eq!(r.wrapping_pow(2), 1 as $T); - assert_eq!(r.checked_pow(2), None); - assert_eq!(r.overflowing_pow(2), (1 as $T, true)); - assert_eq!(r.saturating_pow(2), MAX); - //test for negative exponent. - r = -2 as $T; - assert_eq!(r.pow(2), 4 as $T); - assert_eq!(r.pow(3), -8 as $T); - assert_eq!(r.pow(0), 1 as $T); - assert_eq!(r.wrapping_pow(2), 4 as $T); - assert_eq!(r.wrapping_pow(3), -8 as $T); - assert_eq!(r.wrapping_pow(0), 1 as $T); - assert_eq!(r.checked_pow(2), Some(4 as $T)); - assert_eq!(r.checked_pow(3), Some(-8 as $T)); - assert_eq!(r.checked_pow(0), Some(1 as $T)); - assert_eq!(r.overflowing_pow(2), (4 as $T, false)); - assert_eq!(r.overflowing_pow(3), (-8 as $T, false)); - assert_eq!(r.overflowing_pow(0), (1 as $T, false)); - assert_eq!(r.saturating_pow(2), 4 as $T); - assert_eq!(r.saturating_pow(3), -8 as $T); - assert_eq!(r.saturating_pow(0), 1 as $T); - } + fn test_pow() { + { + const R: $T = 2; + assert_eq_const_safe!(R.pow(2), 4 as $T); + assert_eq_const_safe!(R.pow(0), 1 as $T); + assert_eq_const_safe!(R.wrapping_pow(2), 4 as $T); + assert_eq_const_safe!(R.wrapping_pow(0), 1 as $T); + assert_eq_const_safe!(R.checked_pow(2), Some(4 as $T)); + assert_eq_const_safe!(R.checked_pow(0), Some(1 as $T)); + assert_eq_const_safe!(R.overflowing_pow(2), (4 as $T, false)); + assert_eq_const_safe!(R.overflowing_pow(0), (1 as $T, false)); + assert_eq_const_safe!(R.saturating_pow(2), 4 as $T); + assert_eq_const_safe!(R.saturating_pow(0), 1 as $T); + } + + { + const R: $T = MAX; + // use `^` to represent .pow() with no overflow. + // if itest::MAX == 2^j-1, then itest is a `j` bit int, + // so that `itest::MAX*itest::MAX == 2^(2*j)-2^(j+1)+1`, + // thussaturating_pow the overflowing result is exactly 1. + assert_eq_const_safe!(R.wrapping_pow(2), 1 as $T); + assert_eq_const_safe!(R.checked_pow(2), None); + assert_eq_const_safe!(R.overflowing_pow(2), (1 as $T, true)); + assert_eq_const_safe!(R.saturating_pow(2), MAX); + } + + { + // test for negative exponent. + const R: $T = -2; + assert_eq_const_safe!(R.pow(2), 4 as $T); + assert_eq_const_safe!(R.pow(3), -8 as $T); + assert_eq_const_safe!(R.pow(0), 1 as $T); + assert_eq_const_safe!(R.wrapping_pow(2), 4 as $T); + assert_eq_const_safe!(R.wrapping_pow(3), -8 as $T); + assert_eq_const_safe!(R.wrapping_pow(0), 1 as $T); + assert_eq_const_safe!(R.checked_pow(2), Some(4 as $T)); + assert_eq_const_safe!(R.checked_pow(3), Some(-8 as $T)); + assert_eq_const_safe!(R.checked_pow(0), Some(1 as $T)); + assert_eq_const_safe!(R.overflowing_pow(2), (4 as $T, false)); + assert_eq_const_safe!(R.overflowing_pow(3), (-8 as $T, false)); + assert_eq_const_safe!(R.overflowing_pow(0), (1 as $T, false)); + assert_eq_const_safe!(R.saturating_pow(2), 4 as $T); + assert_eq_const_safe!(R.saturating_pow(3), -8 as $T); + assert_eq_const_safe!(R.saturating_pow(0), 1 as $T); + } + } - #[test] - fn test_div_floor() { - let a: $T = 8; - let b = 3; - assert_eq!(a.div_floor(b), 2); - assert_eq!(a.div_floor(-b), -3); - assert_eq!((-a).div_floor(b), -3); - assert_eq!((-a).div_floor(-b), 2); - } + fn test_div_floor() { + const A: $T = 8; + const B: $T = 3; + assert_eq_const_safe!(A.div_floor(B), 2); + assert_eq_const_safe!(A.div_floor(-B), -3); + assert_eq_const_safe!((-A).div_floor(B), -3); + assert_eq_const_safe!((-A).div_floor(-B), 2); + } - #[test] - fn test_div_ceil() { - let a: $T = 8; - let b = 3; - assert_eq!(a.div_ceil(b), 3); - assert_eq!(a.div_ceil(-b), -2); - assert_eq!((-a).div_ceil(b), -2); - assert_eq!((-a).div_ceil(-b), 3); - } + fn test_div_ceil() { + const A: $T = 8; + const B: $T = 3; + assert_eq_const_safe!(A.div_ceil(B), 3); + assert_eq_const_safe!(A.div_ceil(-B), -2); + assert_eq_const_safe!((-A).div_ceil(B), -2); + assert_eq_const_safe!((-A).div_ceil(-B), 3); + } - #[test] - fn test_next_multiple_of() { - assert_eq!((16 as $T).next_multiple_of(8), 16); - assert_eq!((23 as $T).next_multiple_of(8), 24); - assert_eq!((16 as $T).next_multiple_of(-8), 16); - assert_eq!((23 as $T).next_multiple_of(-8), 16); - assert_eq!((-16 as $T).next_multiple_of(8), -16); - assert_eq!((-23 as $T).next_multiple_of(8), -16); - assert_eq!((-16 as $T).next_multiple_of(-8), -16); - assert_eq!((-23 as $T).next_multiple_of(-8), -24); - assert_eq!(MIN.next_multiple_of(-1), MIN); - } + fn test_next_multiple_of() { + assert_eq_const_safe!((16 as $T).next_multiple_of(8), 16); + assert_eq_const_safe!((23 as $T).next_multiple_of(8), 24); + assert_eq_const_safe!((16 as $T).next_multiple_of(-8), 16); + assert_eq_const_safe!((23 as $T).next_multiple_of(-8), 16); + assert_eq_const_safe!((-16 as $T).next_multiple_of(8), -16); + assert_eq_const_safe!((-23 as $T).next_multiple_of(8), -16); + assert_eq_const_safe!((-16 as $T).next_multiple_of(-8), -16); + assert_eq_const_safe!((-23 as $T).next_multiple_of(-8), -24); + assert_eq_const_safe!(MIN.next_multiple_of(-1), MIN); + } - #[test] - fn test_checked_next_multiple_of() { - assert_eq!((16 as $T).checked_next_multiple_of(8), Some(16)); - assert_eq!((23 as $T).checked_next_multiple_of(8), Some(24)); - assert_eq!((16 as $T).checked_next_multiple_of(-8), Some(16)); - assert_eq!((23 as $T).checked_next_multiple_of(-8), Some(16)); - assert_eq!((-16 as $T).checked_next_multiple_of(8), Some(-16)); - assert_eq!((-23 as $T).checked_next_multiple_of(8), Some(-16)); - assert_eq!((-16 as $T).checked_next_multiple_of(-8), Some(-16)); - assert_eq!((-23 as $T).checked_next_multiple_of(-8), Some(-24)); - assert_eq!((1 as $T).checked_next_multiple_of(0), None); - assert_eq!(MAX.checked_next_multiple_of(2), None); - assert_eq!(MIN.checked_next_multiple_of(-3), None); - assert_eq!(MIN.checked_next_multiple_of(-1), Some(MIN)); - } + fn test_checked_next_multiple_of() { + assert_eq_const_safe!((16 as $T).checked_next_multiple_of(8), Some(16)); + assert_eq_const_safe!((23 as $T).checked_next_multiple_of(8), Some(24)); + assert_eq_const_safe!((16 as $T).checked_next_multiple_of(-8), Some(16)); + assert_eq_const_safe!((23 as $T).checked_next_multiple_of(-8), Some(16)); + assert_eq_const_safe!((-16 as $T).checked_next_multiple_of(8), Some(-16)); + assert_eq_const_safe!((-23 as $T).checked_next_multiple_of(8), Some(-16)); + assert_eq_const_safe!((-16 as $T).checked_next_multiple_of(-8), Some(-16)); + assert_eq_const_safe!((-23 as $T).checked_next_multiple_of(-8), Some(-24)); + assert_eq_const_safe!((1 as $T).checked_next_multiple_of(0), None); + assert_eq_const_safe!(MAX.checked_next_multiple_of(2), None); + assert_eq_const_safe!(MIN.checked_next_multiple_of(-3), None); + assert_eq_const_safe!(MIN.checked_next_multiple_of(-1), Some(MIN)); + } - #[test] - fn test_carrying_add() { - assert_eq!($T::MAX.carrying_add(1, false), ($T::MIN, true)); - assert_eq!($T::MAX.carrying_add(0, true), ($T::MIN, true)); - assert_eq!($T::MAX.carrying_add(1, true), ($T::MIN + 1, true)); - assert_eq!($T::MAX.carrying_add(-1, false), ($T::MAX - 1, false)); - assert_eq!($T::MAX.carrying_add(-1, true), ($T::MAX, false)); // no intermediate overflow - assert_eq!($T::MIN.carrying_add(-1, false), ($T::MAX, true)); - assert_eq!($T::MIN.carrying_add(-1, true), ($T::MIN, false)); // no intermediate overflow - assert_eq!((0 as $T).carrying_add($T::MAX, true), ($T::MIN, true)); - assert_eq!((0 as $T).carrying_add($T::MIN, true), ($T::MIN + 1, false)); - } + fn test_carrying_add() { + assert_eq_const_safe!(MAX.carrying_add(1, false), (MIN, true)); + assert_eq_const_safe!(MAX.carrying_add(0, true), (MIN, true)); + assert_eq_const_safe!(MAX.carrying_add(1, true), (MIN + 1, true)); + assert_eq_const_safe!(MAX.carrying_add(-1, false), (MAX - 1, false)); + assert_eq_const_safe!(MAX.carrying_add(-1, true), (MAX, false)); // no intermediate overflow + assert_eq_const_safe!(MIN.carrying_add(-1, false), (MAX, true)); + assert_eq_const_safe!(MIN.carrying_add(-1, true), (MIN, false)); // no intermediate overflow + assert_eq_const_safe!((0 as $T).carrying_add(MAX, true), (MIN, true)); + assert_eq_const_safe!((0 as $T).carrying_add(MIN, true), (MIN + 1, false)); + } - #[test] - fn test_borrowing_sub() { - assert_eq!($T::MIN.borrowing_sub(1, false), ($T::MAX, true)); - assert_eq!($T::MIN.borrowing_sub(0, true), ($T::MAX, true)); - assert_eq!($T::MIN.borrowing_sub(1, true), ($T::MAX - 1, true)); - assert_eq!($T::MIN.borrowing_sub(-1, false), ($T::MIN + 1, false)); - assert_eq!($T::MIN.borrowing_sub(-1, true), ($T::MIN, false)); // no intermediate overflow - assert_eq!($T::MAX.borrowing_sub(-1, false), ($T::MIN, true)); - assert_eq!($T::MAX.borrowing_sub(-1, true), ($T::MAX, false)); // no intermediate overflow - assert_eq!((0 as $T).borrowing_sub($T::MIN, false), ($T::MIN, true)); - assert_eq!((0 as $T).borrowing_sub($T::MIN, true), ($T::MAX, false)); - } + fn test_borrowing_sub() { + assert_eq_const_safe!(MIN.borrowing_sub(1, false), (MAX, true)); + assert_eq_const_safe!(MIN.borrowing_sub(0, true), (MAX, true)); + assert_eq_const_safe!(MIN.borrowing_sub(1, true), (MAX - 1, true)); + assert_eq_const_safe!(MIN.borrowing_sub(-1, false), (MIN + 1, false)); + assert_eq_const_safe!(MIN.borrowing_sub(-1, true), (MIN, false)); // no intermediate overflow + assert_eq_const_safe!(MAX.borrowing_sub(-1, false), (MIN, true)); + assert_eq_const_safe!(MAX.borrowing_sub(-1, true), (MAX, false)); // no intermediate overflow + assert_eq_const_safe!((0 as $T).borrowing_sub(MIN, false), (MIN, true)); + assert_eq_const_safe!((0 as $T).borrowing_sub(MIN, true), (MAX, false)); + } - #[test] - fn test_midpoint() { - assert_eq!(<$T>::midpoint(1, 3), 2); - assert_eq!(<$T>::midpoint(3, 1), 2); - - assert_eq!(<$T>::midpoint(0, 0), 0); - assert_eq!(<$T>::midpoint(0, 2), 1); - assert_eq!(<$T>::midpoint(2, 0), 1); - assert_eq!(<$T>::midpoint(2, 2), 2); - - assert_eq!(<$T>::midpoint(1, 4), 2); - assert_eq!(<$T>::midpoint(4, 1), 2); - assert_eq!(<$T>::midpoint(3, 4), 3); - assert_eq!(<$T>::midpoint(4, 3), 3); - - assert_eq!(<$T>::midpoint(<$T>::MIN, <$T>::MAX), -1); - assert_eq!(<$T>::midpoint(<$T>::MAX, <$T>::MIN), -1); - assert_eq!(<$T>::midpoint(<$T>::MIN, <$T>::MIN), <$T>::MIN); - assert_eq!(<$T>::midpoint(<$T>::MAX, <$T>::MAX), <$T>::MAX); - - assert_eq!(<$T>::midpoint(<$T>::MIN, 6), <$T>::MIN / 2 + 3); - assert_eq!(<$T>::midpoint(6, <$T>::MIN), <$T>::MIN / 2 + 3); - assert_eq!(<$T>::midpoint(<$T>::MAX, 6), <$T>::MAX / 2 + 3); - assert_eq!(<$T>::midpoint(6, <$T>::MAX), <$T>::MAX / 2 + 3); + fn test_midpoint() { + assert_eq_const_safe!(<$T>::midpoint(1, 3), 2); + assert_eq_const_safe!(<$T>::midpoint(3, 1), 2); + + assert_eq_const_safe!(<$T>::midpoint(0, 0), 0); + assert_eq_const_safe!(<$T>::midpoint(0, 2), 1); + assert_eq_const_safe!(<$T>::midpoint(2, 0), 1); + assert_eq_const_safe!(<$T>::midpoint(2, 2), 2); + + assert_eq_const_safe!(<$T>::midpoint(1, 4), 2); + assert_eq_const_safe!(<$T>::midpoint(4, 1), 2); + assert_eq_const_safe!(<$T>::midpoint(3, 4), 3); + assert_eq_const_safe!(<$T>::midpoint(4, 3), 3); + + assert_eq_const_safe!(<$T>::midpoint(<$T>::MIN, <$T>::MAX), 0); + assert_eq_const_safe!(<$T>::midpoint(<$T>::MAX, <$T>::MIN), 0); + assert_eq_const_safe!(<$T>::midpoint(<$T>::MIN, <$T>::MIN), <$T>::MIN); + assert_eq_const_safe!(<$T>::midpoint(<$T>::MAX, <$T>::MAX), <$T>::MAX); + + assert_eq_const_safe!(<$T>::midpoint(<$T>::MIN, 6), <$T>::MIN / 2 + 3); + assert_eq_const_safe!(<$T>::midpoint(6, <$T>::MIN), <$T>::MIN / 2 + 3); + assert_eq_const_safe!(<$T>::midpoint(<$T>::MAX, 6), <$T>::MAX / 2 + 3); + assert_eq_const_safe!(<$T>::midpoint(6, <$T>::MAX), <$T>::MAX / 2 + 3); + } } }; } diff --git a/library/core/tests/num/midpoint.rs b/library/core/tests/num/midpoint.rs new file mode 100644 index 0000000000000..71e980067842a --- /dev/null +++ b/library/core/tests/num/midpoint.rs @@ -0,0 +1,54 @@ +//! Test the following expectations: +//! - midpoint(a, b) == (a + b) / 2 +//! - midpoint(a, b) == midpoint(b, a) +//! - midpoint(-a, -b) == -midpoint(a, b) + +#[test] +#[cfg(not(miri))] +fn midpoint_obvious_impl_i8() { + for a in i8::MIN..=i8::MAX { + for b in i8::MIN..=i8::MAX { + assert_eq!(i8::midpoint(a, b), ((a as i16 + b as i16) / 2) as i8); + } + } +} + +#[test] +#[cfg(not(miri))] +fn midpoint_obvious_impl_u8() { + for a in u8::MIN..=u8::MAX { + for b in u8::MIN..=u8::MAX { + assert_eq!(u8::midpoint(a, b), ((a as u16 + b as u16) / 2) as u8); + } + } +} + +#[test] +#[cfg(not(miri))] +fn midpoint_order_expectation_i8() { + for a in i8::MIN..=i8::MAX { + for b in i8::MIN..=i8::MAX { + assert_eq!(i8::midpoint(a, b), i8::midpoint(b, a)); + } + } +} + +#[test] +#[cfg(not(miri))] +fn midpoint_order_expectation_u8() { + for a in u8::MIN..=u8::MAX { + for b in u8::MIN..=u8::MAX { + assert_eq!(u8::midpoint(a, b), u8::midpoint(b, a)); + } + } +} + +#[test] +#[cfg(not(miri))] +fn midpoint_negative_expectation() { + for a in 0..=i8::MAX { + for b in 0..=i8::MAX { + assert_eq!(i8::midpoint(-a, -b), -i8::midpoint(a, b)); + } + } +} diff --git a/library/core/tests/num/mod.rs b/library/core/tests/num/mod.rs index b14fe0b22c311..0add9a01e682d 100644 --- a/library/core/tests/num/mod.rs +++ b/library/core/tests/num/mod.rs @@ -1,5 +1,5 @@ use core::fmt::Debug; -use core::num::{can_not_overflow, IntErrorKind, ParseIntError, TryFromIntError}; +use core::num::{IntErrorKind, ParseIntError, TryFromIntError, can_not_overflow}; use core::ops::{Add, Div, Mul, Rem, Sub}; use core::str::FromStr; @@ -28,6 +28,7 @@ mod dec2flt; mod flt2dec; mod int_log; mod int_sqrt; +mod midpoint; mod ops; mod wrapping; diff --git a/library/core/tests/num/uint_macros.rs b/library/core/tests/num/uint_macros.rs index f4fa789461eb8..ad8e48491e829 100644 --- a/library/core/tests/num/uint_macros.rs +++ b/library/core/tests/num/uint_macros.rs @@ -2,7 +2,6 @@ macro_rules! uint_module { ($T:ident) => { use core::ops::{BitAnd, BitOr, BitXor, Not, Shl, Shr}; use core::$T::*; - use std::str::FromStr; use crate::num; @@ -35,122 +34,115 @@ macro_rules! uint_module { const _0: $T = 0; const _1: $T = !0; - #[test] - fn test_count_ones() { - assert!(A.count_ones() == 3); - assert!(B.count_ones() == 2); - assert!(C.count_ones() == 5); - } + test_runtime_and_compiletime! { + fn test_count_ones() { + assert!(A.count_ones() == 3); + assert!(B.count_ones() == 2); + assert!(C.count_ones() == 5); + } - #[test] - fn test_count_zeros() { - assert!(A.count_zeros() == $T::BITS - 3); - assert!(B.count_zeros() == $T::BITS - 2); - assert!(C.count_zeros() == $T::BITS - 5); - } + fn test_count_zeros() { + assert!(A.count_zeros() == $T::BITS - 3); + assert!(B.count_zeros() == $T::BITS - 2); + assert!(C.count_zeros() == $T::BITS - 5); + } - #[test] - fn test_leading_trailing_ones() { - let a: $T = 0b0101_1111; - assert_eq!(a.trailing_ones(), 5); - assert_eq!((!a).leading_ones(), $T::BITS - 7); + fn test_leading_trailing_ones() { + const A: $T = 0b0101_1111; + assert_eq_const_safe!(A.trailing_ones(), 5); + assert_eq_const_safe!((!A).leading_ones(), $T::BITS - 7); - assert_eq!(a.reverse_bits().leading_ones(), 5); + assert_eq_const_safe!(A.reverse_bits().leading_ones(), 5); - assert_eq!(_1.leading_ones(), $T::BITS); - assert_eq!(_1.trailing_ones(), $T::BITS); + assert_eq_const_safe!(_1.leading_ones(), $T::BITS); + assert_eq_const_safe!(_1.trailing_ones(), $T::BITS); - assert_eq!((_1 << 1).trailing_ones(), 0); - assert_eq!((_1 >> 1).leading_ones(), 0); + assert_eq_const_safe!((_1 << 1).trailing_ones(), 0); + assert_eq_const_safe!((_1 >> 1).leading_ones(), 0); - assert_eq!((_1 << 1).leading_ones(), $T::BITS - 1); - assert_eq!((_1 >> 1).trailing_ones(), $T::BITS - 1); + assert_eq_const_safe!((_1 << 1).leading_ones(), $T::BITS - 1); + assert_eq_const_safe!((_1 >> 1).trailing_ones(), $T::BITS - 1); - assert_eq!(_0.leading_ones(), 0); - assert_eq!(_0.trailing_ones(), 0); + assert_eq_const_safe!(_0.leading_ones(), 0); + assert_eq_const_safe!(_0.trailing_ones(), 0); - let x: $T = 0b0010_1100; - assert_eq!(x.leading_ones(), 0); - assert_eq!(x.trailing_ones(), 0); - } + const X: $T = 0b0010_1100; + assert_eq_const_safe!(X.leading_ones(), 0); + assert_eq_const_safe!(X.trailing_ones(), 0); + } - #[test] - fn test_rotate() { - assert_eq!(A.rotate_left(6).rotate_right(2).rotate_right(4), A); - assert_eq!(B.rotate_left(3).rotate_left(2).rotate_right(5), B); - assert_eq!(C.rotate_left(6).rotate_right(2).rotate_right(4), C); - - // Rotating these should make no difference - // - // We test using 124 bits because to ensure that overlong bit shifts do - // not cause undefined behaviour. See #10183. - assert_eq!(_0.rotate_left(124), _0); - assert_eq!(_1.rotate_left(124), _1); - assert_eq!(_0.rotate_right(124), _0); - assert_eq!(_1.rotate_right(124), _1); - - // Rotating by 0 should have no effect - assert_eq!(A.rotate_left(0), A); - assert_eq!(B.rotate_left(0), B); - assert_eq!(C.rotate_left(0), C); - // Rotating by a multiple of word size should also have no effect - assert_eq!(A.rotate_left(128), A); - assert_eq!(B.rotate_left(128), B); - assert_eq!(C.rotate_left(128), C); - } + fn test_rotate() { + assert_eq_const_safe!(A.rotate_left(6).rotate_right(2).rotate_right(4), A); + assert_eq_const_safe!(B.rotate_left(3).rotate_left(2).rotate_right(5), B); + assert_eq_const_safe!(C.rotate_left(6).rotate_right(2).rotate_right(4), C); + + // Rotating these should make no difference + // + // We test using 124 bits because to ensure that overlong bit shifts do + // not cause undefined behavior. See #10183. + assert_eq_const_safe!(_0.rotate_left(124), _0); + assert_eq_const_safe!(_1.rotate_left(124), _1); + assert_eq_const_safe!(_0.rotate_right(124), _0); + assert_eq_const_safe!(_1.rotate_right(124), _1); + + // Rotating by 0 should have no effect + assert_eq_const_safe!(A.rotate_left(0), A); + assert_eq_const_safe!(B.rotate_left(0), B); + assert_eq_const_safe!(C.rotate_left(0), C); + // Rotating by a multiple of word size should also have no effect + assert_eq_const_safe!(A.rotate_left(128), A); + assert_eq_const_safe!(B.rotate_left(128), B); + assert_eq_const_safe!(C.rotate_left(128), C); + } - #[test] - fn test_swap_bytes() { - assert_eq!(A.swap_bytes().swap_bytes(), A); - assert_eq!(B.swap_bytes().swap_bytes(), B); - assert_eq!(C.swap_bytes().swap_bytes(), C); - - // Swapping these should make no difference - assert_eq!(_0.swap_bytes(), _0); - assert_eq!(_1.swap_bytes(), _1); - } + fn test_swap_bytes() { + assert_eq_const_safe!(A.swap_bytes().swap_bytes(), A); + assert_eq_const_safe!(B.swap_bytes().swap_bytes(), B); + assert_eq_const_safe!(C.swap_bytes().swap_bytes(), C); - #[test] - fn test_reverse_bits() { - assert_eq!(A.reverse_bits().reverse_bits(), A); - assert_eq!(B.reverse_bits().reverse_bits(), B); - assert_eq!(C.reverse_bits().reverse_bits(), C); - - // Swapping these should make no difference - assert_eq!(_0.reverse_bits(), _0); - assert_eq!(_1.reverse_bits(), _1); - } + // Swapping these should make no difference + assert_eq_const_safe!(_0.swap_bytes(), _0); + assert_eq_const_safe!(_1.swap_bytes(), _1); + } - #[test] - fn test_le() { - assert_eq!($T::from_le(A.to_le()), A); - assert_eq!($T::from_le(B.to_le()), B); - assert_eq!($T::from_le(C.to_le()), C); - assert_eq!($T::from_le(_0), _0); - assert_eq!($T::from_le(_1), _1); - assert_eq!(_0.to_le(), _0); - assert_eq!(_1.to_le(), _1); - } + fn test_reverse_bits() { + assert_eq_const_safe!(A.reverse_bits().reverse_bits(), A); + assert_eq_const_safe!(B.reverse_bits().reverse_bits(), B); + assert_eq_const_safe!(C.reverse_bits().reverse_bits(), C); - #[test] - fn test_be() { - assert_eq!($T::from_be(A.to_be()), A); - assert_eq!($T::from_be(B.to_be()), B); - assert_eq!($T::from_be(C.to_be()), C); - assert_eq!($T::from_be(_0), _0); - assert_eq!($T::from_be(_1), _1); - assert_eq!(_0.to_be(), _0); - assert_eq!(_1.to_be(), _1); - } + // Swapping these should make no difference + assert_eq_const_safe!(_0.reverse_bits(), _0); + assert_eq_const_safe!(_1.reverse_bits(), _1); + } - #[test] - fn test_unsigned_checked_div() { - assert!((10 as $T).checked_div(2) == Some(5)); - assert!((5 as $T).checked_div(0) == None); + fn test_le() { + assert_eq_const_safe!($T::from_le(A.to_le()), A); + assert_eq_const_safe!($T::from_le(B.to_le()), B); + assert_eq_const_safe!($T::from_le(C.to_le()), C); + assert_eq_const_safe!($T::from_le(_0), _0); + assert_eq_const_safe!($T::from_le(_1), _1); + assert_eq_const_safe!(_0.to_le(), _0); + assert_eq_const_safe!(_1.to_le(), _1); + } + + fn test_be() { + assert_eq_const_safe!($T::from_be(A.to_be()), A); + assert_eq_const_safe!($T::from_be(B.to_be()), B); + assert_eq_const_safe!($T::from_be(C.to_be()), C); + assert_eq_const_safe!($T::from_be(_0), _0); + assert_eq_const_safe!($T::from_be(_1), _1); + assert_eq_const_safe!(_0.to_be(), _0); + assert_eq_const_safe!(_1.to_be(), _1); + } + + fn test_unsigned_checked_div() { + assert_eq_const_safe!((10 as $T).checked_div(2), Some(5)); + assert_eq_const_safe!((5 as $T).checked_div(0), None); + } } - fn from_str(t: &str) -> Option { - FromStr::from_str(t).ok() + fn from_str(t: &str) -> Option { + core::str::FromStr::from_str(t).ok() } #[test] @@ -166,52 +158,55 @@ macro_rules! uint_module { assert_eq!(from_str::<$T>("x"), None); } - #[test] - pub fn test_parse_bytes() { - assert_eq!($T::from_str_radix("123", 10), Ok(123 as $T)); - assert_eq!($T::from_str_radix("1001", 2), Ok(9 as $T)); - assert_eq!($T::from_str_radix("123", 8), Ok(83 as $T)); - assert_eq!(u16::from_str_radix("123", 16), Ok(291 as u16)); - assert_eq!(u16::from_str_radix("ffff", 16), Ok(65535 as u16)); - assert_eq!($T::from_str_radix("z", 36), Ok(35 as $T)); - - assert_eq!($T::from_str_radix("Z", 10).ok(), None::<$T>); - assert_eq!($T::from_str_radix("_", 2).ok(), None::<$T>); - } + test_runtime_and_compiletime! { + fn test_parse_bytes() { + assert_eq_const_safe!($T::from_str_radix("123", 10), Ok(123 as $T)); + assert_eq_const_safe!($T::from_str_radix("1001", 2), Ok(9 as $T)); + assert_eq_const_safe!($T::from_str_radix("123", 8), Ok(83 as $T)); + assert_eq_const_safe!(u16::from_str_radix("123", 16), Ok(291 as u16)); + assert_eq_const_safe!(u16::from_str_radix("ffff", 16), Ok(65535 as u16)); + assert_eq_const_safe!($T::from_str_radix("z", 36), Ok(35 as $T)); + + assert!($T::from_str_radix("Z", 10).is_err()); + assert!($T::from_str_radix("_", 2).is_err()); + } - #[test] - fn test_pow() { - let mut r = 2 as $T; - assert_eq!(r.pow(2), 4 as $T); - assert_eq!(r.pow(0), 1 as $T); - assert_eq!(r.wrapping_pow(2), 4 as $T); - assert_eq!(r.wrapping_pow(0), 1 as $T); - assert_eq!(r.checked_pow(2), Some(4 as $T)); - assert_eq!(r.checked_pow(0), Some(1 as $T)); - assert_eq!(r.overflowing_pow(2), (4 as $T, false)); - assert_eq!(r.overflowing_pow(0), (1 as $T, false)); - assert_eq!(r.saturating_pow(2), 4 as $T); - assert_eq!(r.saturating_pow(0), 1 as $T); - - r = MAX; - // use `^` to represent .pow() with no overflow. - // if itest::MAX == 2^j-1, then itest is a `j` bit int, - // so that `itest::MAX*itest::MAX == 2^(2*j)-2^(j+1)+1`, - // thussaturating_pow the overflowing result is exactly 1. - assert_eq!(r.wrapping_pow(2), 1 as $T); - assert_eq!(r.checked_pow(2), None); - assert_eq!(r.overflowing_pow(2), (1 as $T, true)); - assert_eq!(r.saturating_pow(2), MAX); - } + fn test_pow() { + { + const R: $T = 2; + assert_eq_const_safe!(R.pow(2), 4 as $T); + assert_eq_const_safe!(R.pow(0), 1 as $T); + assert_eq_const_safe!(R.wrapping_pow(2), 4 as $T); + assert_eq_const_safe!(R.wrapping_pow(0), 1 as $T); + assert_eq_const_safe!(R.checked_pow(2), Some(4 as $T)); + assert_eq_const_safe!(R.checked_pow(0), Some(1 as $T)); + assert_eq_const_safe!(R.overflowing_pow(2), (4 as $T, false)); + assert_eq_const_safe!(R.overflowing_pow(0), (1 as $T, false)); + assert_eq_const_safe!(R.saturating_pow(2), 4 as $T); + assert_eq_const_safe!(R.saturating_pow(0), 1 as $T); + } + + { + const R: $T = $T::MAX; + // use `^` to represent .pow() with no overflow. + // if itest::MAX == 2^j-1, then itest is a `j` bit int, + // so that `itest::MAX*itest::MAX == 2^(2*j)-2^(j+1)+1`, + // thussaturating_pow the overflowing result is exactly 1. + assert_eq_const_safe!(R.wrapping_pow(2), 1 as $T); + assert_eq_const_safe!(R.checked_pow(2), None); + assert_eq_const_safe!(R.overflowing_pow(2), (1 as $T, true)); + assert_eq_const_safe!(R.saturating_pow(2), MAX); + } + } - #[test] - fn test_isqrt() { - assert_eq!((0 as $T).isqrt(), 0 as $T); - assert_eq!((1 as $T).isqrt(), 1 as $T); - assert_eq!((2 as $T).isqrt(), 1 as $T); - assert_eq!((99 as $T).isqrt(), 9 as $T); - assert_eq!((100 as $T).isqrt(), 10 as $T); - assert_eq!($T::MAX.isqrt(), (1 << ($T::BITS / 2)) - 1); + fn test_isqrt() { + assert_eq_const_safe!((0 as $T).isqrt(), 0 as $T); + assert_eq_const_safe!((1 as $T).isqrt(), 1 as $T); + assert_eq_const_safe!((2 as $T).isqrt(), 1 as $T); + assert_eq_const_safe!((99 as $T).isqrt(), 9 as $T); + assert_eq_const_safe!((100 as $T).isqrt(), 10 as $T); + assert_eq_const_safe!($T::MAX.isqrt(), (1 << ($T::BITS / 2)) - 1); + } } #[cfg(not(miri))] // Miri is too slow @@ -233,85 +228,79 @@ macro_rules! uint_module { } } - #[test] - fn test_div_floor() { - assert_eq!((8 as $T).div_floor(3), 2); - } + test_runtime_and_compiletime! { + fn test_div_floor() { + assert_eq_const_safe!((8 as $T).div_floor(3), 2); + } - #[test] - fn test_div_ceil() { - assert_eq!((8 as $T).div_ceil(3), 3); - } + fn test_div_ceil() { + assert_eq_const_safe!((8 as $T).div_ceil(3), 3); + } - #[test] - fn test_next_multiple_of() { - assert_eq!((16 as $T).next_multiple_of(8), 16); - assert_eq!((23 as $T).next_multiple_of(8), 24); - assert_eq!(MAX.next_multiple_of(1), MAX); - } + fn test_next_multiple_of() { + assert_eq_const_safe!((16 as $T).next_multiple_of(8), 16); + assert_eq_const_safe!((23 as $T).next_multiple_of(8), 24); + assert_eq_const_safe!(MAX.next_multiple_of(1), MAX); + } - #[test] - fn test_checked_next_multiple_of() { - assert_eq!((16 as $T).checked_next_multiple_of(8), Some(16)); - assert_eq!((23 as $T).checked_next_multiple_of(8), Some(24)); - assert_eq!((1 as $T).checked_next_multiple_of(0), None); - assert_eq!(MAX.checked_next_multiple_of(2), None); - } + fn test_checked_next_multiple_of() { + assert_eq_const_safe!((16 as $T).checked_next_multiple_of(8), Some(16)); + assert_eq_const_safe!((23 as $T).checked_next_multiple_of(8), Some(24)); + assert_eq_const_safe!((1 as $T).checked_next_multiple_of(0), None); + assert_eq_const_safe!(MAX.checked_next_multiple_of(2), None); + } - #[test] - fn test_is_next_multiple_of() { - assert!((12 as $T).is_multiple_of(4)); - assert!(!(12 as $T).is_multiple_of(5)); - assert!((0 as $T).is_multiple_of(0)); - assert!(!(12 as $T).is_multiple_of(0)); - } + fn test_is_next_multiple_of() { + assert!((12 as $T).is_multiple_of(4)); + assert!(!(12 as $T).is_multiple_of(5)); + assert!((0 as $T).is_multiple_of(0)); + assert!(!(12 as $T).is_multiple_of(0)); + } - #[test] - fn test_carrying_add() { - assert_eq!($T::MAX.carrying_add(1, false), (0, true)); - assert_eq!($T::MAX.carrying_add(0, true), (0, true)); - assert_eq!($T::MAX.carrying_add(1, true), (1, true)); - - assert_eq!($T::MIN.carrying_add($T::MAX, false), ($T::MAX, false)); - assert_eq!($T::MIN.carrying_add(0, true), (1, false)); - assert_eq!($T::MIN.carrying_add($T::MAX, true), (0, true)); - } + fn test_carrying_add() { + assert_eq_const_safe!($T::MAX.carrying_add(1, false), (0, true)); + assert_eq_const_safe!($T::MAX.carrying_add(0, true), (0, true)); + assert_eq_const_safe!($T::MAX.carrying_add(1, true), (1, true)); - #[test] - fn test_borrowing_sub() { - assert_eq!($T::MIN.borrowing_sub(1, false), ($T::MAX, true)); - assert_eq!($T::MIN.borrowing_sub(0, true), ($T::MAX, true)); - assert_eq!($T::MIN.borrowing_sub(1, true), ($T::MAX - 1, true)); - - assert_eq!($T::MAX.borrowing_sub($T::MAX, false), (0, false)); - assert_eq!($T::MAX.borrowing_sub(0, true), ($T::MAX - 1, false)); - assert_eq!($T::MAX.borrowing_sub($T::MAX, true), ($T::MAX, true)); - } + assert_eq_const_safe!($T::MIN.carrying_add($T::MAX, false), ($T::MAX, false)); + assert_eq_const_safe!($T::MIN.carrying_add(0, true), (1, false)); + assert_eq_const_safe!($T::MIN.carrying_add($T::MAX, true), (0, true)); + } - #[test] - fn test_midpoint() { - assert_eq!(<$T>::midpoint(1, 3), 2); - assert_eq!(<$T>::midpoint(3, 1), 2); - - assert_eq!(<$T>::midpoint(0, 0), 0); - assert_eq!(<$T>::midpoint(0, 2), 1); - assert_eq!(<$T>::midpoint(2, 0), 1); - assert_eq!(<$T>::midpoint(2, 2), 2); - - assert_eq!(<$T>::midpoint(1, 4), 2); - assert_eq!(<$T>::midpoint(4, 1), 2); - assert_eq!(<$T>::midpoint(3, 4), 3); - assert_eq!(<$T>::midpoint(4, 3), 3); - - assert_eq!(<$T>::midpoint(<$T>::MIN, <$T>::MAX), (<$T>::MAX - <$T>::MIN) / 2); - assert_eq!(<$T>::midpoint(<$T>::MAX, <$T>::MIN), (<$T>::MAX - <$T>::MIN) / 2); - assert_eq!(<$T>::midpoint(<$T>::MIN, <$T>::MIN), <$T>::MIN); - assert_eq!(<$T>::midpoint(<$T>::MAX, <$T>::MAX), <$T>::MAX); - - assert_eq!(<$T>::midpoint(<$T>::MIN, 6), <$T>::MIN / 2 + 3); - assert_eq!(<$T>::midpoint(6, <$T>::MIN), <$T>::MIN / 2 + 3); - assert_eq!(<$T>::midpoint(<$T>::MAX, 6), (<$T>::MAX - <$T>::MIN) / 2 + 3); - assert_eq!(<$T>::midpoint(6, <$T>::MAX), (<$T>::MAX - <$T>::MIN) / 2 + 3); + fn test_borrowing_sub() { + assert_eq_const_safe!($T::MIN.borrowing_sub(1, false), ($T::MAX, true)); + assert_eq_const_safe!($T::MIN.borrowing_sub(0, true), ($T::MAX, true)); + assert_eq_const_safe!($T::MIN.borrowing_sub(1, true), ($T::MAX - 1, true)); + + assert_eq_const_safe!($T::MAX.borrowing_sub($T::MAX, false), (0, false)); + assert_eq_const_safe!($T::MAX.borrowing_sub(0, true), ($T::MAX - 1, false)); + assert_eq_const_safe!($T::MAX.borrowing_sub($T::MAX, true), ($T::MAX, true)); + } + + fn test_midpoint() { + assert_eq_const_safe!(<$T>::midpoint(1, 3), 2); + assert_eq_const_safe!(<$T>::midpoint(3, 1), 2); + + assert_eq_const_safe!(<$T>::midpoint(0, 0), 0); + assert_eq_const_safe!(<$T>::midpoint(0, 2), 1); + assert_eq_const_safe!(<$T>::midpoint(2, 0), 1); + assert_eq_const_safe!(<$T>::midpoint(2, 2), 2); + + assert_eq_const_safe!(<$T>::midpoint(1, 4), 2); + assert_eq_const_safe!(<$T>::midpoint(4, 1), 2); + assert_eq_const_safe!(<$T>::midpoint(3, 4), 3); + assert_eq_const_safe!(<$T>::midpoint(4, 3), 3); + + assert_eq_const_safe!(<$T>::midpoint(<$T>::MIN, <$T>::MAX), (<$T>::MAX - <$T>::MIN) / 2); + assert_eq_const_safe!(<$T>::midpoint(<$T>::MAX, <$T>::MIN), (<$T>::MAX - <$T>::MIN) / 2); + assert_eq_const_safe!(<$T>::midpoint(<$T>::MIN, <$T>::MIN), <$T>::MIN); + assert_eq_const_safe!(<$T>::midpoint(<$T>::MAX, <$T>::MAX), <$T>::MAX); + + assert_eq_const_safe!(<$T>::midpoint(<$T>::MIN, 6), <$T>::MIN / 2 + 3); + assert_eq_const_safe!(<$T>::midpoint(6, <$T>::MIN), <$T>::MIN / 2 + 3); + assert_eq_const_safe!(<$T>::midpoint(<$T>::MAX, 6), (<$T>::MAX - <$T>::MIN) / 2 + 3); + assert_eq_const_safe!(<$T>::midpoint(6, <$T>::MAX), (<$T>::MAX - <$T>::MIN) / 2 + 3); + } } }; } diff --git a/library/core/tests/pin.rs b/library/core/tests/pin.rs index 7a6af46a74323..026d2ca8de26a 100644 --- a/library/core/tests/pin.rs +++ b/library/core/tests/pin.rs @@ -19,6 +19,10 @@ fn pin_const() { const REF: &'static usize = PINNED.get_ref(); assert_eq!(REF, POINTER); + const INT: u8 = 42; + const STATIC_REF: Pin<&'static u8> = Pin::static_ref(&INT); + assert_eq!(*STATIC_REF, INT); + // Note: `pin_mut_const` tests that the methods of `Pin<&mut T>` are usable in a const context. // A const fn is used because `&mut` is not (yet) usable in constants. const fn pin_mut_const() { diff --git a/library/core/tests/pin_macro.rs b/library/core/tests/pin_macro.rs index 36c6972515a96..43542397a6136 100644 --- a/library/core/tests/pin_macro.rs +++ b/library/core/tests/pin_macro.rs @@ -2,7 +2,7 @@ use core::marker::PhantomPinned; use core::mem::{drop as stuff, transmute}; -use core::pin::{pin, Pin}; +use core::pin::{Pin, pin}; #[test] fn basic() { diff --git a/library/core/tests/slice.rs b/library/core/tests/slice.rs index cdefb5d3eb201..9ae2bcc852649 100644 --- a/library/core/tests/slice.rs +++ b/library/core/tests/slice.rs @@ -1800,65 +1800,14 @@ fn brute_force_rotate_test_1() { } } -#[test] -#[cfg(not(target_arch = "wasm32"))] -fn sort_unstable() { - use rand::Rng; - - // Miri is too slow (but still need to `chain` to make the types match) - let lens = if cfg!(miri) { (2..20).chain(0..0) } else { (2..25).chain(500..510) }; - let rounds = if cfg!(miri) { 1 } else { 100 }; - - let mut v = [0; 600]; - let mut tmp = [0; 600]; - let mut rng = crate::test_rng(); - - for len in lens { - let v = &mut v[0..len]; - let tmp = &mut tmp[0..len]; - - for &modulus in &[5, 10, 100, 1000] { - for _ in 0..rounds { - for i in 0..len { - v[i] = rng.gen::() % modulus; - } - - // Sort in default order. - tmp.copy_from_slice(v); - tmp.sort_unstable(); - assert!(tmp.windows(2).all(|w| w[0] <= w[1])); - - // Sort in ascending order. - tmp.copy_from_slice(v); - tmp.sort_unstable_by(|a, b| a.cmp(b)); - assert!(tmp.windows(2).all(|w| w[0] <= w[1])); - - // Sort in descending order. - tmp.copy_from_slice(v); - tmp.sort_unstable_by(|a, b| b.cmp(a)); - assert!(tmp.windows(2).all(|w| w[0] >= w[1])); - } - } - } - - // Should not panic. - [0i32; 0].sort_unstable(); - [(); 10].sort_unstable(); - [(); 100].sort_unstable(); - - let mut v = [0xDEADBEEFu64]; - v.sort_unstable(); - assert!(v == [0xDEADBEEF]); -} - #[test] #[cfg(not(target_arch = "wasm32"))] #[cfg_attr(miri, ignore)] // Miri is too slow fn select_nth_unstable() { use core::cmp::Ordering::{Equal, Greater, Less}; - use rand::seq::SliceRandom; use rand::Rng; + use rand::seq::SliceRandom; let mut rng = crate::test_rng(); diff --git a/library/core/tests/waker.rs b/library/core/tests/waker.rs index 8f6bf0565fc35..4889b8959ece4 100644 --- a/library/core/tests/waker.rs +++ b/library/core/tests/waker.rs @@ -22,7 +22,7 @@ static WAKER_VTABLE: RawWakerVTable = RawWakerVTable::new( // https://github.com/rust-lang/rust/issues/102012#issuecomment-1915282956 mod nop_waker { - use core::future::{ready, Future}; + use core::future::{Future, ready}; use core::pin::Pin; use core::task::{Context, Poll, RawWaker, RawWakerVTable, Waker}; diff --git a/library/panic_unwind/src/emcc.rs b/library/panic_unwind/src/emcc.rs index 86a43184fb529..b986fc1c2a829 100644 --- a/library/panic_unwind/src/emcc.rs +++ b/library/panic_unwind/src/emcc.rs @@ -78,7 +78,7 @@ pub unsafe fn cleanup(ptr: *mut u8) -> Box { super::__rust_foreign_exception(); } - let canary = ptr::addr_of!((*adjusted_ptr).canary).read(); + let canary = (&raw const (*adjusted_ptr).canary).read(); if !ptr::eq(canary, &EXCEPTION_TYPE_INFO) { super::__rust_foreign_exception(); } @@ -98,14 +98,11 @@ pub unsafe fn panic(data: Box) -> u32 { if exception.is_null() { return uw::_URC_FATAL_PHASE1_ERROR as u32; } - ptr::write( - exception, - Exception { - canary: &EXCEPTION_TYPE_INFO, - caught: AtomicBool::new(false), - data: Some(data), - }, - ); + ptr::write(exception, Exception { + canary: &EXCEPTION_TYPE_INFO, + caught: AtomicBool::new(false), + data: Some(data), + }); __cxa_throw(exception as *mut _, &EXCEPTION_TYPE_INFO, exception_cleanup); } diff --git a/library/panic_unwind/src/gcc.rs b/library/panic_unwind/src/gcc.rs index 589d3c1b4d2d0..925af6c08322e 100644 --- a/library/panic_unwind/src/gcc.rs +++ b/library/panic_unwind/src/gcc.rs @@ -61,7 +61,7 @@ struct Exception { pub unsafe fn panic(data: Box) -> u32 { let exception = Box::new(Exception { _uwe: uw::_Unwind_Exception { - exception_class: rust_exception_class(), + exception_class: RUST_EXCEPTION_CLASS, exception_cleanup: Some(exception_cleanup), private: [core::ptr::null(); uw::unwinder_private_data_size], }, @@ -84,7 +84,7 @@ pub unsafe fn panic(data: Box) -> u32 { pub unsafe fn cleanup(ptr: *mut u8) -> Box { let exception = ptr as *mut uw::_Unwind_Exception; - if (*exception).exception_class != rust_exception_class() { + if (*exception).exception_class != RUST_EXCEPTION_CLASS { uw::_Unwind_DeleteException(exception); super::__rust_foreign_exception(); } @@ -92,7 +92,7 @@ pub unsafe fn cleanup(ptr: *mut u8) -> Box { let exception = exception.cast::(); // Just access the canary field, avoid accessing the entire `Exception` as // it can be a foreign Rust exception. - let canary = ptr::addr_of!((*exception).canary).read(); + let canary = (&raw const (*exception).canary).read(); if !ptr::eq(canary, &CANARY) { // A foreign Rust exception, treat it slightly differently from other // foreign exceptions, because call into `_Unwind_DeleteException` will @@ -107,7 +107,4 @@ pub unsafe fn cleanup(ptr: *mut u8) -> Box { // Rust's exception class identifier. This is used by personality routines to // determine whether the exception was thrown by their own runtime. -fn rust_exception_class() -> uw::_Unwind_Exception_Class { - // M O Z \0 R U S T -- vendor, language - 0x4d4f5a_00_52555354 -} +const RUST_EXCEPTION_CLASS: uw::_Unwind_Exception_Class = u64::from_ne_bytes(*b"MOZ\0RUST"); diff --git a/library/panic_unwind/src/lib.rs b/library/panic_unwind/src/lib.rs index 4552fb68d26d5..1981675f40922 100644 --- a/library/panic_unwind/src/lib.rs +++ b/library/panic_unwind/src/lib.rs @@ -19,8 +19,6 @@ #![feature(panic_unwind)] #![feature(staged_api)] #![feature(std_internals)] -#![feature(strict_provenance)] -#![feature(exposed_provenance)] #![feature(rustc_attrs)] #![panic_runtime] #![feature(panic_runtime)] @@ -48,7 +46,7 @@ cfg_if::cfg_if! { target_os = "psp", target_os = "xous", target_os = "solid_asp3", - all(target_family = "unix", not(any(target_os = "espidf", target_os = "rtems"))), + all(target_family = "unix", not(any(target_os = "espidf", target_os = "rtems", target_os = "nuttx"))), all(target_vendor = "fortanix", target_env = "sgx"), target_family = "wasm", ))] { diff --git a/library/panic_unwind/src/seh.rs b/library/panic_unwind/src/seh.rs index 070c11926f6e0..565a2b8c573b4 100644 --- a/library/panic_unwind/src/seh.rs +++ b/library/panic_unwind/src/seh.rs @@ -50,7 +50,6 @@ use alloc::boxed::Box; use core::any::Any; use core::ffi::{c_int, c_uint, c_void}; use core::mem::{self, ManuallyDrop}; -use core::ptr::{addr_of, addr_of_mut}; // NOTE(nbdd0121): The `canary` field is part of stable ABI. #[repr(C)] @@ -131,8 +130,6 @@ mod imp { #[cfg(not(target_arch = "x86"))] mod imp { - use core::ptr::addr_of; - // On 64-bit systems, SEH represents pointers as 32-bit offsets from `__ImageBase`. #[repr(transparent)] #[derive(Copy, Clone)] @@ -157,7 +154,7 @@ mod imp { // going to be cross-lang LTOed anyway. However, using expose is shorter and // requires less unsafe. let addr: usize = ptr.expose_provenance(); - let image_base = addr_of!(__ImageBase).addr(); + let image_base = (&raw const __ImageBase).addr(); let offset: usize = addr - image_base; Self(offset as u32) } @@ -250,7 +247,7 @@ extern "C" { // This is fine since the MSVC runtime uses string comparison on the type name // to match TypeDescriptors rather than pointer equality. static mut TYPE_DESCRIPTOR: _TypeDescriptor = _TypeDescriptor { - pVFTable: addr_of!(TYPE_INFO_VTABLE) as *const _, + pVFTable: (&raw const TYPE_INFO_VTABLE) as *const _, spare: core::ptr::null_mut(), name: TYPE_NAME, }; @@ -291,6 +288,8 @@ cfg_if::cfg_if! { } } +// FIXME(static_mut_refs): Do not allow `static_mut_refs` lint +#[allow(static_mut_refs)] pub unsafe fn panic(data: Box) -> u32 { use core::intrinsics::atomic_store_seqcst; @@ -302,8 +301,8 @@ pub unsafe fn panic(data: Box) -> u32 { // dropped when unwinding. Instead it will be dropped by exception_cleanup // which is invoked by the C++ runtime. let mut exception = - ManuallyDrop::new(Exception { canary: addr_of!(TYPE_DESCRIPTOR), data: Some(data) }); - let throw_ptr = addr_of_mut!(exception) as *mut _; + ManuallyDrop::new(Exception { canary: (&raw const TYPE_DESCRIPTOR), data: Some(data) }); + let throw_ptr = (&raw mut exception) as *mut _; // This... may seems surprising, and justifiably so. On 32-bit MSVC the // pointers between these structure are just that, pointers. On 64-bit MSVC, @@ -326,23 +325,23 @@ pub unsafe fn panic(data: Box) -> u32 { // In any case, we basically need to do something like this until we can // express more operations in statics (and we may never be able to). atomic_store_seqcst( - addr_of_mut!(THROW_INFO.pmfnUnwind).cast(), + (&raw mut THROW_INFO.pmfnUnwind).cast(), ptr_t::new(exception_cleanup as *mut u8).raw(), ); atomic_store_seqcst( - addr_of_mut!(THROW_INFO.pCatchableTypeArray).cast(), - ptr_t::new(addr_of_mut!(CATCHABLE_TYPE_ARRAY).cast()).raw(), + (&raw mut THROW_INFO.pCatchableTypeArray).cast(), + ptr_t::new((&raw mut CATCHABLE_TYPE_ARRAY).cast()).raw(), ); atomic_store_seqcst( - addr_of_mut!(CATCHABLE_TYPE_ARRAY.arrayOfCatchableTypes[0]).cast(), - ptr_t::new(addr_of_mut!(CATCHABLE_TYPE).cast()).raw(), + (&raw mut CATCHABLE_TYPE_ARRAY.arrayOfCatchableTypes[0]).cast(), + ptr_t::new((&raw mut CATCHABLE_TYPE).cast()).raw(), ); atomic_store_seqcst( - addr_of_mut!(CATCHABLE_TYPE.pType).cast(), - ptr_t::new(addr_of_mut!(TYPE_DESCRIPTOR).cast()).raw(), + (&raw mut CATCHABLE_TYPE.pType).cast(), + ptr_t::new((&raw mut TYPE_DESCRIPTOR).cast()).raw(), ); atomic_store_seqcst( - addr_of_mut!(CATCHABLE_TYPE.copyFunction).cast(), + (&raw mut CATCHABLE_TYPE.copyFunction).cast(), ptr_t::new(exception_copy as *mut u8).raw(), ); @@ -350,7 +349,7 @@ pub unsafe fn panic(data: Box) -> u32 { fn _CxxThrowException(pExceptionObject: *mut c_void, pThrowInfo: *mut u8) -> !; } - _CxxThrowException(throw_ptr, addr_of_mut!(THROW_INFO) as *mut _); + _CxxThrowException(throw_ptr, (&raw mut THROW_INFO) as *mut _); } pub unsafe fn cleanup(payload: *mut u8) -> Box { @@ -360,8 +359,8 @@ pub unsafe fn cleanup(payload: *mut u8) -> Box { super::__rust_foreign_exception(); } let exception = payload as *mut Exception; - let canary = addr_of!((*exception).canary).read(); - if !core::ptr::eq(canary, addr_of!(TYPE_DESCRIPTOR)) { + let canary = (&raw const (*exception).canary).read(); + if !core::ptr::eq(canary, &raw const TYPE_DESCRIPTOR) { // A foreign Rust exception. super::__rust_foreign_exception(); } diff --git a/library/portable-simd/crates/core_simd/src/lib.rs b/library/portable-simd/crates/core_simd/src/lib.rs index 331b66262490c..992a7705e3c52 100644 --- a/library/portable-simd/crates/core_simd/src/lib.rs +++ b/library/portable-simd/crates/core_simd/src/lib.rs @@ -1,8 +1,6 @@ #![no_std] #![feature( - const_intrinsic_copy, const_refs_to_cell, - const_maybe_uninit_as_mut_ptr, const_mut_refs, convert_float_to_int, core_intrinsics, @@ -11,7 +9,6 @@ repr_simd, simd_ffi, staged_api, - strict_provenance, prelude_import, ptr_metadata )] diff --git a/library/portable-simd/crates/core_simd/src/swizzle.rs b/library/portable-simd/crates/core_simd/src/swizzle.rs index 2f4f777b20e29..d62642fb9061b 100644 --- a/library/portable-simd/crates/core_simd/src/swizzle.rs +++ b/library/portable-simd/crates/core_simd/src/swizzle.rs @@ -85,7 +85,7 @@ pub trait Swizzle { LaneCount: SupportedLaneCount, LaneCount: SupportedLaneCount, { - // Safety: `vector` is a vector, and the index is a const array of u32. + // Safety: `vector` is a vector, and the index is a const vector of u32. unsafe { core::intrinsics::simd::simd_shuffle( vector, @@ -103,7 +103,11 @@ pub trait Swizzle { output[i] = index as u32; i += 1; } - output + + // The index list needs to be returned as a vector. + #[repr(simd)] + struct SimdShuffleIdx([u32; LEN]); + SimdShuffleIdx(output) }, ) } @@ -121,7 +125,7 @@ pub trait Swizzle { LaneCount: SupportedLaneCount, LaneCount: SupportedLaneCount, { - // Safety: `first` and `second` are vectors, and the index is a const array of u32. + // Safety: `first` and `second` are vectors, and the index is a const vector of u32. unsafe { core::intrinsics::simd::simd_shuffle( first, @@ -139,7 +143,11 @@ pub trait Swizzle { output[i] = index as u32; i += 1; } - output + + // The index list needs to be returned as a vector. + #[repr(simd)] + struct SimdShuffleIdx([u32; LEN]); + SimdShuffleIdx(output) }, ) } diff --git a/library/portable-simd/crates/core_simd/tests/pointers.rs b/library/portable-simd/crates/core_simd/tests/pointers.rs index 90bfc5d5fd6a5..d7db4e82b3ca2 100644 --- a/library/portable-simd/crates/core_simd/tests/pointers.rs +++ b/library/portable-simd/crates/core_simd/tests/pointers.rs @@ -1,4 +1,4 @@ -#![feature(portable_simd, strict_provenance, exposed_provenance)] +#![feature(portable_simd)] use core_simd::simd::{ ptr::{SimdConstPtr, SimdMutPtr}, diff --git a/library/proc_macro/src/bridge/buffer.rs b/library/proc_macro/src/bridge/buffer.rs index 78fcd1999b2f3..3760749d83a54 100644 --- a/library/proc_macro/src/bridge/buffer.rs +++ b/library/proc_macro/src/bridge/buffer.rs @@ -119,9 +119,7 @@ impl Write for Buffer { } impl Drop for Buffer { - // HACK(nbdd0121): Hack to prevent LLVM < 17.0.4 from misoptimising, - // change to `#[inline]` if fixed. - #[inline(never)] + #[inline] fn drop(&mut self) { let b = self.take(); (b.drop)(b); diff --git a/library/proc_macro/src/bridge/client.rs b/library/proc_macro/src/bridge/client.rs index 5a1086527a127..f6d4825c67b24 100644 --- a/library/proc_macro/src/bridge/client.rs +++ b/library/proc_macro/src/bridge/client.rs @@ -18,17 +18,10 @@ macro_rules! define_client_handles { $(pub(super) $ity: AtomicU32,)* } - impl HandleCounters { - // FIXME(eddyb) use a reference to the `static COUNTERS`, instead of - // a wrapper `fn` pointer, once `const fn` can reference `static`s. - extern "C" fn get() -> &'static Self { - static COUNTERS: HandleCounters = HandleCounters { - $($oty: AtomicU32::new(1),)* - $($ity: AtomicU32::new(1),)* - }; - &COUNTERS - } - } + static COUNTERS: HandleCounters = HandleCounters { + $($oty: AtomicU32::new(1),)* + $($ity: AtomicU32::new(1),)* + }; $( pub(crate) struct $oty { @@ -259,9 +252,7 @@ pub(crate) fn is_available() -> bool { /// and forcing the use of APIs that take/return `S::TokenStream`, server-side. #[repr(C)] pub struct Client { - // FIXME(eddyb) use a reference to the `static COUNTERS`, instead of - // a wrapper `fn` pointer, once `const fn` can reference `static`s. - pub(super) get_handle_counters: extern "C" fn() -> &'static HandleCounters, + pub(super) handle_counters: &'static HandleCounters, pub(super) run: extern "C" fn(BridgeConfig<'_>) -> Buffer, @@ -346,7 +337,7 @@ fn run_client DecodeMut<'a, 's, ()>, R: Encode<()>>( impl Client { pub const fn expand1(f: impl Fn(crate::TokenStream) -> crate::TokenStream + Copy) -> Self { Client { - get_handle_counters: HandleCounters::get, + handle_counters: &COUNTERS, run: super::selfless_reify::reify_to_extern_c_fn_hrt_bridge(move |bridge| { run_client(bridge, |input| f(crate::TokenStream(Some(input))).0) }), @@ -360,7 +351,7 @@ impl Client<(crate::TokenStream, crate::TokenStream), crate::TokenStream> { f: impl Fn(crate::TokenStream, crate::TokenStream) -> crate::TokenStream + Copy, ) -> Self { Client { - get_handle_counters: HandleCounters::get, + handle_counters: &COUNTERS, run: super::selfless_reify::reify_to_extern_c_fn_hrt_bridge(move |bridge| { run_client(bridge, |(input, input2)| { f(crate::TokenStream(Some(input)), crate::TokenStream(Some(input2))).0 diff --git a/library/proc_macro/src/bridge/server.rs b/library/proc_macro/src/bridge/server.rs index 692b6038a3872..97e5a603c3ac9 100644 --- a/library/proc_macro/src/bridge/server.rs +++ b/library/proc_macro/src/bridge/server.rs @@ -400,10 +400,10 @@ impl client::Client { S: Server, S::TokenStream: Default, { - let client::Client { get_handle_counters, run, _marker } = *self; + let client::Client { handle_counters, run, _marker } = *self; run_server( strategy, - get_handle_counters(), + handle_counters, server, as Types>::TokenStream::mark(input), run, @@ -426,10 +426,10 @@ impl client::Client<(crate::TokenStream, crate::TokenStream), crate::TokenStream S: Server, S::TokenStream: Default, { - let client::Client { get_handle_counters, run, _marker } = *self; + let client::Client { handle_counters, run, _marker } = *self; run_server( strategy, - get_handle_counters(), + handle_counters, server, ( as Types>::TokenStream::mark(input), diff --git a/library/proc_macro/src/bridge/symbol.rs b/library/proc_macro/src/bridge/symbol.rs index 37aaee6b21553..edad6e7ac393f 100644 --- a/library/proc_macro/src/bridge/symbol.rs +++ b/library/proc_macro/src/bridge/symbol.rs @@ -76,7 +76,7 @@ impl Symbol { .all(|b| matches!(b, b'_' | b'a'..=b'z' | b'A'..=b'Z' | b'0'..=b'9')) } - // Mimics the behaviour of `Symbol::can_be_raw` from `rustc_span` + // Mimics the behavior of `Symbol::can_be_raw` from `rustc_span` fn can_be_raw(string: &str) -> bool { match string { "_" | "super" | "self" | "Self" | "crate" => false, diff --git a/library/proc_macro/src/lib.rs b/library/proc_macro/src/lib.rs index 72b53c60f7439..15770248b3106 100644 --- a/library/proc_macro/src/lib.rs +++ b/library/proc_macro/src/lib.rs @@ -32,7 +32,7 @@ #![feature(restricted_std)] #![feature(rustc_attrs)] #![feature(min_specialization)] -#![feature(strict_provenance)] +#![feature(extend_one)] #![recursion_limit = "256"] #![allow(internal_features)] #![deny(ffi_unwind_calls)] @@ -44,6 +44,7 @@ pub mod bridge; mod diagnostic; mod escape; +mod to_tokens; use std::ffi::CStr; use std::ops::{Range, RangeBounds}; @@ -53,8 +54,10 @@ use std::{error, fmt}; #[unstable(feature = "proc_macro_diagnostic", issue = "54140")] pub use diagnostic::{Diagnostic, Level, MultiSpan}; +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +pub use to_tokens::ToTokens; -use crate::escape::{escape_bytes, EscapeOptions}; +use crate::escape::{EscapeOptions, escape_bytes}; /// Determines whether proc_macro has been made accessible to the currently /// running program. @@ -371,7 +374,7 @@ impl Extend for TokenStream { /// Public implementation details for the `TokenStream` type, such as iterators. #[stable(feature = "proc_macro_lib2", since = "1.29.0")] pub mod token_stream { - use crate::{bridge, Group, Ident, Literal, Punct, TokenStream, TokenTree}; + use crate::{Group, Ident, Literal, Punct, TokenStream, TokenTree, bridge}; /// An iterator over `TokenStream`'s `TokenTree`s. /// The iteration is "shallow", e.g., the iterator doesn't recurse into delimited groups, @@ -1166,7 +1169,7 @@ impl fmt::Debug for Ident { } } -/// A literal string (`"hello"`), byte string (`b"hello"`), +/// A literal string (`"hello"`), byte string (`b"hello"`), C string (`c"hello"`), /// character (`'a'`), byte character (`b'a'`), an integer or floating point number /// with or without a suffix (`1`, `1u8`, `2.3`, `2.3f32`). /// Boolean literals like `true` and `false` do not belong here, they are `Ident`s. diff --git a/library/proc_macro/src/to_tokens.rs b/library/proc_macro/src/to_tokens.rs new file mode 100644 index 0000000000000..8f697b416d5c6 --- /dev/null +++ b/library/proc_macro/src/to_tokens.rs @@ -0,0 +1,310 @@ +use std::borrow::Cow; +use std::ffi::{CStr, CString}; +use std::rc::Rc; + +use crate::{ConcatTreesHelper, Group, Ident, Literal, Punct, Span, TokenStream, TokenTree}; + +/// Types that can be interpolated inside a [`quote!`] invocation. +/// +/// [`quote!`]: crate::quote! +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +pub trait ToTokens { + /// Write `self` to the given `TokenStream`. + /// + /// # Example + /// + /// Example implementation for a struct representing Rust paths like + /// `std::cmp::PartialEq`: + /// + /// ``` + /// #![feature(proc_macro_totokens)] + /// + /// use std::iter; + /// use proc_macro::{Spacing, Punct, TokenStream, TokenTree, ToTokens}; + /// + /// pub struct Path { + /// pub global: bool, + /// pub segments: Vec, + /// } + /// + /// impl ToTokens for Path { + /// fn to_tokens(&self, tokens: &mut TokenStream) { + /// for (i, segment) in self.segments.iter().enumerate() { + /// if i > 0 || self.global { + /// // Double colon `::` + /// tokens.extend(iter::once(TokenTree::from(Punct::new(':', Spacing::Joint)))); + /// tokens.extend(iter::once(TokenTree::from(Punct::new(':', Spacing::Alone)))); + /// } + /// segment.to_tokens(tokens); + /// } + /// } + /// } + /// # + /// # pub struct PathSegment; + /// # + /// # impl ToTokens for PathSegment { + /// # fn to_tokens(&self, tokens: &mut TokenStream) { + /// # unimplemented!() + /// # } + /// # } + /// ``` + fn to_tokens(&self, tokens: &mut TokenStream); + + /// Convert `self` directly into a `TokenStream` object. + /// + /// This method is implicitly implemented using `to_tokens`, and acts as a + /// convenience method for consumers of the `ToTokens` trait. + fn to_token_stream(&self) -> TokenStream { + let mut tokens = TokenStream::new(); + self.to_tokens(&mut tokens); + tokens + } + + /// Convert `self` directly into a `TokenStream` object. + /// + /// This method is implicitly implemented using `to_tokens`, and acts as a + /// convenience method for consumers of the `ToTokens` trait. + fn into_token_stream(self) -> TokenStream + where + Self: Sized, + { + self.to_token_stream() + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for TokenTree { + fn to_tokens(&self, tokens: &mut TokenStream) { + tokens.extend_one(self.clone()); + } + + fn into_token_stream(self) -> TokenStream { + let mut builder = ConcatTreesHelper::new(1); + builder.push(self); + builder.build() + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for TokenStream { + fn to_tokens(&self, tokens: &mut TokenStream) { + tokens.extend(self.clone()); + } + + fn into_token_stream(self) -> TokenStream { + self + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for Literal { + fn to_tokens(&self, tokens: &mut TokenStream) { + tokens.extend_one(TokenTree::from(self.clone())); + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for Ident { + fn to_tokens(&self, tokens: &mut TokenStream) { + tokens.extend_one(TokenTree::from(self.clone())); + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for Punct { + fn to_tokens(&self, tokens: &mut TokenStream) { + tokens.extend_one(TokenTree::from(self.clone())); + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for Group { + fn to_tokens(&self, tokens: &mut TokenStream) { + tokens.extend_one(TokenTree::from(self.clone())); + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for &T { + fn to_tokens(&self, tokens: &mut TokenStream) { + (**self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for &mut T { + fn to_tokens(&self, tokens: &mut TokenStream) { + (**self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for Box { + fn to_tokens(&self, tokens: &mut TokenStream) { + (**self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for Rc { + fn to_tokens(&self, tokens: &mut TokenStream) { + (**self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for Cow<'_, T> { + fn to_tokens(&self, tokens: &mut TokenStream) { + (**self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for Option { + fn to_tokens(&self, tokens: &mut TokenStream) { + if let Some(t) = self { + t.to_tokens(tokens); + } + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for u8 { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::u8_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for u16 { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::u16_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for u32 { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::u32_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for u64 { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::u64_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for u128 { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::u128_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for i8 { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::i8_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for i16 { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::i16_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for i32 { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::i32_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for i64 { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::i64_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for i128 { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::i128_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for f32 { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::f32_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for f64 { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::f64_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for usize { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::usize_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for isize { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::isize_suffixed(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for bool { + fn to_tokens(&self, tokens: &mut TokenStream) { + let word = if *self { "true" } else { "false" }; + Ident::new(word, Span::call_site()).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for char { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::character(*self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for str { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::string(self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for String { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::string(self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for CStr { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::c_string(self).to_tokens(tokens) + } +} + +#[unstable(feature = "proc_macro_totokens", issue = "130977")] +impl ToTokens for CString { + fn to_tokens(&self, tokens: &mut TokenStream) { + Literal::c_string(self).to_tokens(tokens) + } +} diff --git a/library/std/Cargo.toml b/library/std/Cargo.toml index 2d75324261318..19cb57f24b848 100644 --- a/library/std/Cargo.toml +++ b/library/std/Cargo.toml @@ -17,12 +17,15 @@ cfg-if = { version = "1.0", features = ['rustc-dep-of-std'] } panic_unwind = { path = "../panic_unwind", optional = true } panic_abort = { path = "../panic_abort" } core = { path = "../core", public = true } -compiler_builtins = { version = "0.1.123" } -profiler_builtins = { path = "../profiler_builtins", optional = true } +compiler_builtins = { version = "0.1.136" } unwind = { path = "../unwind" } -hashbrown = { version = "0.14", default-features = false, features = [ +hashbrown = { version = "0.15", default-features = false, features = [ 'rustc-dep-of-std', ] } +# FIXME(#127890): `object` depends on `memchr`, but `memchr` > v2.5 causes +# issues with LTO. This dependency is not used directly, but pin it here so +# it resolves to 2.5. To be removed once rust-lang/rust#127890 is fixed. +memchr = { version = "=2.5.0", default-features = false, features = ["rustc-dep-of-std"] } std_detect = { path = "../stdarch/crates/std_detect", default-features = false, features = [ 'rustc-dep-of-std', ] } @@ -36,7 +39,7 @@ miniz_oxide = { version = "0.7.0", optional = true, default-features = false } addr2line = { version = "0.22.0", optional = true, default-features = false } [target.'cfg(not(all(windows, target_env = "msvc")))'.dependencies] -libc = { version = "0.2.156", default-features = false, features = [ +libc = { version = "0.2.161", default-features = false, features = [ 'rustc-dep-of-std', ], public = true } @@ -95,7 +98,6 @@ backtrace = [ ] panic-unwind = ["panic_unwind"] -profiler = ["profiler_builtins"] compiler-builtins-c = ["alloc/compiler-builtins-c"] compiler-builtins-mem = ["alloc/compiler-builtins-mem"] compiler-builtins-no-asm = ["alloc/compiler-builtins-no-asm"] diff --git a/library/std/build.rs b/library/std/build.rs index ba1eece46f3ce..032326556bd5b 100644 --- a/library/std/build.rs +++ b/library/std/build.rs @@ -7,6 +7,7 @@ fn main() { let target_vendor = env::var("CARGO_CFG_TARGET_VENDOR").expect("CARGO_CFG_TARGET_VENDOR was not set"); let target_env = env::var("CARGO_CFG_TARGET_ENV").expect("CARGO_CFG_TARGET_ENV was not set"); + let target_abi = env::var("CARGO_CFG_TARGET_ABI").expect("CARGO_CFG_TARGET_ABI was not set"); let target_pointer_width: u32 = env::var("CARGO_CFG_TARGET_POINTER_WIDTH") .expect("CARGO_CFG_TARGET_POINTER_WIDTH was not set") .parse() @@ -54,6 +55,7 @@ fn main() { || target_os == "teeos" || target_os == "zkvm" || target_os == "rtems" + || target_os == "nuttx" // See src/bootstrap/src/core/build_steps/synthetic_targets.rs || env::var("RUSTC_BOOTSTRAP_SYNTHETIC_TARGET").is_ok() @@ -95,30 +97,24 @@ fn main() { let has_reliable_f16 = match (target_arch.as_str(), target_os.as_str()) { // We can always enable these in Miri as that is not affected by codegen bugs. _ if is_miri => true, - // Selection failure until recent LLVM - // FIXME(llvm19): can probably be removed at the version bump - ("loongarch64", _) => false, // Selection failure ("s390x", _) => false, // Unsupported ("arm64ec", _) => false, // MinGW ABI bugs - ("x86_64", "windows") => false, - // Apple has a special ABI for `f16` that we do not yet support - // FIXME(builtins): fixed by - ("x86" | "x86_64", _) if target_vendor == "apple" => false, - // Missing `__gnu_h2f_ieee` and `__gnu_f2h_ieee` + ("x86_64", "windows") if target_env == "gnu" && target_abi != "llvm" => false, + // Infinite recursion + ("csky", _) => false, + ("hexagon", _) => false, + ("loongarch64", _) => false, + ("mips" | "mips64" | "mips32r6" | "mips64r6", _) => false, ("powerpc" | "powerpc64", _) => false, - // Missing `__gnu_h2f_ieee` and `__gnu_f2h_ieee` - ("mips" | "mips32r6" | "mips64" | "mips64r6", _) => false, - // Missing `__extendhfsf` and `__truncsfhf` - ("riscv32" | "riscv64", _) => false, - // Most OSs are missing `__extendhfsf` and `__truncsfhf` - (_, "linux" | "macos") => true, - // Almost all OSs besides Linux and MacOS are missing symbols until compiler-builtins can - // be updated. will get some of these, the - // next CB update should get the rest. - _ => false, + ("sparc" | "sparc64", _) => false, + ("wasm32" | "wasm64", _) => false, + // `f16` support only requires that symbols converting to and from `f32` are available. We + // provide these in `compiler-builtins`, so `f16` should be available on all platforms that + // do not have other ABI issues or LLVM crashes. + _ => true, }; let has_reliable_f128 = match (target_arch.as_str(), target_os.as_str()) { @@ -134,10 +130,10 @@ fn main() { // ABI unsupported ("sparc", _) => false, // MinGW ABI bugs - ("x86_64", "windows") => false, + ("x86_64", "windows") if target_env == "gnu" && target_abi != "llvm" => false, // 64-bit Linux is about the only platform to have f128 symbols by default (_, "linux") if target_pointer_width == 64 => true, - // Same as for f16, except MacOS is also missing f128 symbols. + // Almost all OSs are missing symbol. compiler-builtins will have to add them. _ => false, }; diff --git a/library/std/src/ascii.rs b/library/std/src/ascii.rs index 3a2880fd50904..3813f3237fb34 100644 --- a/library/std/src/ascii.rs +++ b/library/std/src/ascii.rs @@ -16,7 +16,7 @@ #[unstable(feature = "ascii_char", issue = "110998")] pub use core::ascii::Char; #[stable(feature = "rust1", since = "1.0.0")] -pub use core::ascii::{escape_default, EscapeDefault}; +pub use core::ascii::{EscapeDefault, escape_default}; /// Extension methods for ASCII-subset only operations. /// diff --git a/library/std/src/backtrace.rs b/library/std/src/backtrace.rs index 7df9a8a14b00c..fc333d7ff3f95 100644 --- a/library/std/src/backtrace.rs +++ b/library/std/src/backtrace.rs @@ -91,9 +91,9 @@ mod tests; use crate::backtrace_rs::{self, BytesOrWideString}; use crate::ffi::c_void; use crate::panic::UnwindSafe; +use crate::sync::LazyLock; use crate::sync::atomic::AtomicU8; use crate::sync::atomic::Ordering::Relaxed; -use crate::sync::LazyLock; use crate::sys::backtrace::{lock, output_filename, set_image_base}; use crate::{env, fmt}; diff --git a/library/std/src/collections/hash/map.rs b/library/std/src/collections/hash/map.rs index 822fa5791e300..ded4f404d781e 100644 --- a/library/std/src/collections/hash/map.rs +++ b/library/std/src/collections/hash/map.rs @@ -909,8 +909,11 @@ where /// Attempts to get mutable references to `N` values in the map at once. /// /// Returns an array of length `N` with the results of each query. For soundness, at most one - /// mutable reference will be returned to any value. `None` will be returned if any of the - /// keys are duplicates or missing. + /// mutable reference will be returned to any value. `None` will be used if the key is missing. + /// + /// # Panics + /// + /// Panics if any keys are overlapping. /// /// # Examples /// @@ -924,16 +927,23 @@ where /// libraries.insert("Herzogin-Anna-Amalia-Bibliothek".to_string(), 1691); /// libraries.insert("Library of Congress".to_string(), 1800); /// + /// // Get Athenæum and Bodleian Library + /// let [Some(a), Some(b)] = libraries.get_many_mut([ + /// "Athenæum", + /// "Bodleian Library", + /// ]) else { panic!() }; + /// + /// // Assert values of Athenæum and Library of Congress /// let got = libraries.get_many_mut([ /// "Athenæum", /// "Library of Congress", /// ]); /// assert_eq!( /// got, - /// Some([ - /// &mut 1807, - /// &mut 1800, - /// ]), + /// [ + /// Some(&mut 1807), + /// Some(&mut 1800), + /// ], /// ); /// /// // Missing keys result in None @@ -941,18 +951,31 @@ where /// "Athenæum", /// "New York Public Library", /// ]); - /// assert_eq!(got, None); + /// assert_eq!( + /// got, + /// [ + /// Some(&mut 1807), + /// None + /// ] + /// ); + /// ``` /// - /// // Duplicate keys result in None + /// ```should_panic + /// #![feature(map_many_mut)] + /// use std::collections::HashMap; + /// + /// let mut libraries = HashMap::new(); + /// libraries.insert("Athenæum".to_string(), 1807); + /// + /// // Duplicate keys panic! /// let got = libraries.get_many_mut([ /// "Athenæum", /// "Athenæum", /// ]); - /// assert_eq!(got, None); /// ``` #[inline] #[unstable(feature = "map_many_mut", issue = "97601")] - pub fn get_many_mut(&mut self, ks: [&Q; N]) -> Option<[&'_ mut V; N]> + pub fn get_many_mut(&mut self, ks: [&Q; N]) -> [Option<&'_ mut V>; N] where K: Borrow, Q: Hash + Eq, @@ -963,10 +986,10 @@ where /// Attempts to get mutable references to `N` values in the map at once, without validating that /// the values are unique. /// - /// Returns an array of length `N` with the results of each query. `None` will be returned if - /// any of the keys are missing. + /// Returns an array of length `N` with the results of each query. `None` will be used if + /// the key is missing. /// - /// For a safe alternative see [`get_many_mut`](Self::get_many_mut). + /// For a safe alternative see [`get_many_mut`](`HashMap::get_many_mut`). /// /// # Safety /// @@ -987,31 +1010,39 @@ where /// libraries.insert("Herzogin-Anna-Amalia-Bibliothek".to_string(), 1691); /// libraries.insert("Library of Congress".to_string(), 1800); /// - /// let got = libraries.get_many_mut([ + /// // SAFETY: The keys do not overlap. + /// let [Some(a), Some(b)] = (unsafe { libraries.get_many_unchecked_mut([ + /// "Athenæum", + /// "Bodleian Library", + /// ]) }) else { panic!() }; + /// + /// // SAFETY: The keys do not overlap. + /// let got = unsafe { libraries.get_many_unchecked_mut([ /// "Athenæum", /// "Library of Congress", - /// ]); + /// ]) }; /// assert_eq!( /// got, - /// Some([ - /// &mut 1807, - /// &mut 1800, - /// ]), + /// [ + /// Some(&mut 1807), + /// Some(&mut 1800), + /// ], /// ); /// - /// // Missing keys result in None - /// let got = libraries.get_many_mut([ + /// // SAFETY: The keys do not overlap. + /// let got = unsafe { libraries.get_many_unchecked_mut([ /// "Athenæum", /// "New York Public Library", - /// ]); - /// assert_eq!(got, None); + /// ]) }; + /// // Missing keys result in None + /// assert_eq!(got, [Some(&mut 1807), None]); /// ``` #[inline] #[unstable(feature = "map_many_mut", issue = "97601")] pub unsafe fn get_many_unchecked_mut( &mut self, ks: [&Q; N], - ) -> Option<[&'_ mut V; N]> + ) -> [Option<&'_ mut V>; N] where K: Borrow, Q: Hash + Eq, @@ -1037,6 +1068,7 @@ where /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "hashmap_contains_key")] pub fn contains_key(&self, k: &Q) -> bool where K: Borrow, @@ -1100,6 +1132,7 @@ where #[inline] #[stable(feature = "rust1", since = "1.0.0")] #[rustc_confusables("push", "append", "put")] + #[cfg_attr(not(test), rustc_diagnostic_item = "hashmap_insert")] pub fn insert(&mut self, k: K, v: V) -> Option { self.base.insert(k, v) } @@ -1391,6 +1424,7 @@ where /// let iter = map.iter(); /// ``` #[stable(feature = "rust1", since = "1.0.0")] +#[cfg_attr(not(test), rustc_diagnostic_item = "hashmap_iter_ty")] pub struct Iter<'a, K: 'a, V: 'a> { base: base::Iter<'a, K, V>, } @@ -1404,6 +1438,14 @@ impl Clone for Iter<'_, K, V> { } } +#[stable(feature = "default_iters_hash", since = "1.83.0")] +impl Default for Iter<'_, K, V> { + #[inline] + fn default() -> Self { + Iter { base: Default::default() } + } +} + #[stable(feature = "std_debug", since = "1.16.0")] impl fmt::Debug for Iter<'_, K, V> { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { @@ -1429,6 +1471,7 @@ impl fmt::Debug for Iter<'_, K, V> { /// let iter = map.iter_mut(); /// ``` #[stable(feature = "rust1", since = "1.0.0")] +#[cfg_attr(not(test), rustc_diagnostic_item = "hashmap_iter_mut_ty")] pub struct IterMut<'a, K: 'a, V: 'a> { base: base::IterMut<'a, K, V>, } @@ -1441,6 +1484,14 @@ impl<'a, K, V> IterMut<'a, K, V> { } } +#[stable(feature = "default_iters_hash", since = "1.83.0")] +impl Default for IterMut<'_, K, V> { + #[inline] + fn default() -> Self { + IterMut { base: Default::default() } + } +} + /// An owning iterator over the entries of a `HashMap`. /// /// This `struct` is created by the [`into_iter`] method on [`HashMap`] @@ -1471,6 +1522,14 @@ impl IntoIter { } } +#[stable(feature = "default_iters_hash", since = "1.83.0")] +impl Default for IntoIter { + #[inline] + fn default() -> Self { + IntoIter { base: Default::default() } + } +} + /// An iterator over the keys of a `HashMap`. /// /// This `struct` is created by the [`keys`] method on [`HashMap`]. See its @@ -1489,6 +1548,7 @@ impl IntoIter { /// let iter_keys = map.keys(); /// ``` #[stable(feature = "rust1", since = "1.0.0")] +#[cfg_attr(not(test), rustc_diagnostic_item = "hashmap_keys_ty")] pub struct Keys<'a, K: 'a, V: 'a> { inner: Iter<'a, K, V>, } @@ -1502,6 +1562,14 @@ impl Clone for Keys<'_, K, V> { } } +#[stable(feature = "default_iters_hash", since = "1.83.0")] +impl Default for Keys<'_, K, V> { + #[inline] + fn default() -> Self { + Keys { inner: Default::default() } + } +} + #[stable(feature = "std_debug", since = "1.16.0")] impl fmt::Debug for Keys<'_, K, V> { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { @@ -1527,6 +1595,7 @@ impl fmt::Debug for Keys<'_, K, V> { /// let iter_values = map.values(); /// ``` #[stable(feature = "rust1", since = "1.0.0")] +#[cfg_attr(not(test), rustc_diagnostic_item = "hashmap_values_ty")] pub struct Values<'a, K: 'a, V: 'a> { inner: Iter<'a, K, V>, } @@ -1540,6 +1609,14 @@ impl Clone for Values<'_, K, V> { } } +#[stable(feature = "default_iters_hash", since = "1.83.0")] +impl Default for Values<'_, K, V> { + #[inline] + fn default() -> Self { + Values { inner: Default::default() } + } +} + #[stable(feature = "std_debug", since = "1.16.0")] impl fmt::Debug for Values<'_, K, V> { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { @@ -1565,6 +1642,7 @@ impl fmt::Debug for Values<'_, K, V> { /// let iter = map.drain(); /// ``` #[stable(feature = "drain", since = "1.6.0")] +#[cfg_attr(not(test), rustc_diagnostic_item = "hashmap_drain_ty")] pub struct Drain<'a, K: 'a, V: 'a> { base: base::Drain<'a, K, V>, } @@ -1622,10 +1700,19 @@ where /// let iter_values = map.values_mut(); /// ``` #[stable(feature = "map_values_mut", since = "1.10.0")] +#[cfg_attr(not(test), rustc_diagnostic_item = "hashmap_values_mut_ty")] pub struct ValuesMut<'a, K: 'a, V: 'a> { inner: IterMut<'a, K, V>, } +#[stable(feature = "default_iters_hash", since = "1.83.0")] +impl Default for ValuesMut<'_, K, V> { + #[inline] + fn default() -> Self { + ValuesMut { inner: Default::default() } + } +} + /// An owning iterator over the keys of a `HashMap`. /// /// This `struct` is created by the [`into_keys`] method on [`HashMap`]. @@ -1648,6 +1735,14 @@ pub struct IntoKeys { inner: IntoIter, } +#[stable(feature = "default_iters_hash", since = "1.83.0")] +impl Default for IntoKeys { + #[inline] + fn default() -> Self { + IntoKeys { inner: Default::default() } + } +} + /// An owning iterator over the values of a `HashMap`. /// /// This `struct` is created by the [`into_values`] method on [`HashMap`]. @@ -1670,6 +1765,14 @@ pub struct IntoValues { inner: IntoIter, } +#[stable(feature = "default_iters_hash", since = "1.83.0")] +impl Default for IntoValues { + #[inline] + fn default() -> Self { + IntoValues { inner: Default::default() } + } +} + /// A builder for computing where in a HashMap a key-value pair would be stored. /// /// See the [`HashMap::raw_entry_mut`] docs for usage examples. @@ -2754,7 +2857,6 @@ impl<'a, K, V> Entry<'a, K, V> { /// # Examples /// /// ``` - /// #![feature(entry_insert)] /// use std::collections::HashMap; /// /// let mut map: HashMap<&str, String> = HashMap::new(); @@ -2763,7 +2865,7 @@ impl<'a, K, V> Entry<'a, K, V> { /// assert_eq!(entry.key(), &"poneyland"); /// ``` #[inline] - #[unstable(feature = "entry_insert", issue = "65225")] + #[stable(feature = "entry_insert", since = "1.83.0")] pub fn insert_entry(self, value: V) -> OccupiedEntry<'a, K, V> { match self { Occupied(mut entry) => { @@ -2971,64 +3073,6 @@ impl<'a, K, V> OccupiedEntry<'a, K, V> { pub fn remove(self) -> V { self.base.remove() } - - /// Replaces the entry, returning the old key and value. The new key in the hash map will be - /// the key used to create this entry. - /// - /// # Examples - /// - /// ``` - /// #![feature(map_entry_replace)] - /// use std::collections::hash_map::{Entry, HashMap}; - /// use std::rc::Rc; - /// - /// let mut map: HashMap, u32> = HashMap::new(); - /// map.insert(Rc::new("Stringthing".to_string()), 15); - /// - /// let my_key = Rc::new("Stringthing".to_string()); - /// - /// if let Entry::Occupied(entry) = map.entry(my_key) { - /// // Also replace the key with a handle to our other key. - /// let (old_key, old_value): (Rc, u32) = entry.replace_entry(16); - /// } - /// - /// ``` - #[inline] - #[unstable(feature = "map_entry_replace", issue = "44286")] - pub fn replace_entry(self, value: V) -> (K, V) { - self.base.replace_entry(value) - } - - /// Replaces the key in the hash map with the key used to create this entry. - /// - /// # Examples - /// - /// ``` - /// #![feature(map_entry_replace)] - /// use std::collections::hash_map::{Entry, HashMap}; - /// use std::rc::Rc; - /// - /// let mut map: HashMap, u32> = HashMap::new(); - /// let known_strings: Vec> = Vec::new(); - /// - /// // Initialise known strings, run program, etc. - /// - /// reclaim_memory(&mut map, &known_strings); - /// - /// fn reclaim_memory(map: &mut HashMap, u32>, known_strings: &[Rc] ) { - /// for s in known_strings { - /// if let Entry::Occupied(entry) = map.entry(Rc::clone(s)) { - /// // Replaces the entry's key with our version of it in `known_strings`. - /// entry.replace_key(); - /// } - /// } - /// } - /// ``` - #[inline] - #[unstable(feature = "map_entry_replace", issue = "44286")] - pub fn replace_key(self) -> K { - self.base.replace_key() - } } impl<'a, K: 'a, V: 'a> VacantEntry<'a, K, V> { @@ -3097,7 +3141,6 @@ impl<'a, K: 'a, V: 'a> VacantEntry<'a, K, V> { /// # Examples /// /// ``` - /// #![feature(entry_insert)] /// use std::collections::HashMap; /// use std::collections::hash_map::Entry; /// @@ -3109,7 +3152,7 @@ impl<'a, K: 'a, V: 'a> VacantEntry<'a, K, V> { /// assert_eq!(map["poneyland"], 37); /// ``` #[inline] - #[unstable(feature = "entry_insert", issue = "65225")] + #[stable(feature = "entry_insert", since = "1.83.0")] pub fn insert_entry(self, value: V) -> OccupiedEntry<'a, K, V> { let base = self.base.insert_entry(value); OccupiedEntry { base } diff --git a/library/std/src/collections/hash/map/tests.rs b/library/std/src/collections/hash/map/tests.rs index 6641197c3724a..b79ad1c3119ff 100644 --- a/library/std/src/collections/hash/map/tests.rs +++ b/library/std/src/collections/hash/map/tests.rs @@ -274,7 +274,7 @@ fn test_lots_of_insertions() { for _ in 0..loops { assert!(m.is_empty()); - let count = if cfg!(miri) { 101 } else { 1001 }; + let count = if cfg!(miri) { 66 } else { 1001 }; for i in 1..count { assert!(m.insert(i, i).is_none()); @@ -947,7 +947,7 @@ fn test_raw_entry() { mod test_extract_if { use super::*; - use crate::panic::{catch_unwind, AssertUnwindSafe}; + use crate::panic::{AssertUnwindSafe, catch_unwind}; use crate::sync::atomic::{AtomicUsize, Ordering}; trait EqSorted: Iterator { @@ -1018,6 +1018,7 @@ mod test_extract_if { } #[test] + #[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] fn drop_panic_leak() { static PREDS: AtomicUsize = AtomicUsize::new(0); static DROPS: AtomicUsize = AtomicUsize::new(0); @@ -1047,6 +1048,7 @@ mod test_extract_if { } #[test] + #[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] fn pred_panic_leak() { static PREDS: AtomicUsize = AtomicUsize::new(0); static DROPS: AtomicUsize = AtomicUsize::new(0); @@ -1076,6 +1078,7 @@ mod test_extract_if { // Same as above, but attempt to use the iterator again after the panic in the predicate #[test] + #[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] fn pred_panic_reuse() { static PREDS: AtomicUsize = AtomicUsize::new(0); static DROPS: AtomicUsize = AtomicUsize::new(0); @@ -1095,7 +1098,7 @@ mod test_extract_if { _ => panic!(), }); catch_unwind(AssertUnwindSafe(|| while it.next().is_some() {})).unwrap_err(); - // Iterator behaviour after a panic is explicitly unspecified, + // Iterator behavior after a panic is explicitly unspecified, // so this is just the current implementation: let result = catch_unwind(AssertUnwindSafe(|| it.next())); assert!(result.is_err()); diff --git a/library/std/src/collections/hash/set.rs b/library/std/src/collections/hash/set.rs index d611353b0d3f2..e1e0eb36d23f0 100644 --- a/library/std/src/collections/hash/set.rs +++ b/library/std/src/collections/hash/set.rs @@ -187,6 +187,7 @@ impl HashSet { #[inline] #[rustc_lint_query_instability] #[stable(feature = "rust1", since = "1.0.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "hashset_iter")] pub fn iter(&self) -> Iter<'_, T> { Iter { base: self.base.iter() } } @@ -723,38 +724,6 @@ where self.base.get_or_insert(value) } - /// Inserts an owned copy of the given `value` into the set if it is not - /// present, then returns a reference to the value in the set. - /// - /// # Examples - /// - /// ``` - /// #![feature(hash_set_entry)] - /// - /// use std::collections::HashSet; - /// - /// let mut set: HashSet = ["cat", "dog", "horse"] - /// .iter().map(|&pet| pet.to_owned()).collect(); - /// - /// assert_eq!(set.len(), 3); - /// for &pet in &["cat", "dog", "fish"] { - /// let value = set.get_or_insert_owned(pet); - /// assert_eq!(value, pet); - /// } - /// assert_eq!(set.len(), 4); // a new "fish" was inserted - /// ``` - #[inline] - #[unstable(feature = "hash_set_entry", issue = "60896")] - pub fn get_or_insert_owned(&mut self, value: &Q) -> &T - where - T: Borrow, - Q: Hash + Eq + ToOwned, - { - // Although the raw entry gives us `&mut T`, we only return `&T` to be consistent with - // `get`. Key mutation is "raw" because you're not supposed to affect `Eq` or `Hash`. - self.base.get_or_insert_owned(value) - } - /// Inserts a value computed from `f` into the set if the given `value` is /// not present, then returns a reference to the value in the set. /// @@ -1270,10 +1239,19 @@ where /// let mut iter = a.iter(); /// ``` #[stable(feature = "rust1", since = "1.0.0")] +#[cfg_attr(not(test), rustc_diagnostic_item = "hashset_iter_ty")] pub struct Iter<'a, K: 'a> { base: base::Iter<'a, K>, } +#[stable(feature = "default_iters_hash", since = "1.83.0")] +impl Default for Iter<'_, K> { + #[inline] + fn default() -> Self { + Iter { base: Default::default() } + } +} + /// An owning iterator over the items of a `HashSet`. /// /// This `struct` is created by the [`into_iter`] method on [`HashSet`] @@ -1295,6 +1273,14 @@ pub struct IntoIter { base: base::IntoIter, } +#[stable(feature = "default_iters_hash", since = "1.83.0")] +impl Default for IntoIter { + #[inline] + fn default() -> Self { + IntoIter { base: Default::default() } + } +} + /// A draining iterator over the items of a `HashSet`. /// /// This `struct` is created by the [`drain`] method on [`HashSet`]. @@ -1312,6 +1298,7 @@ pub struct IntoIter { /// let mut drain = a.drain(); /// ``` #[stable(feature = "rust1", since = "1.0.0")] +#[cfg_attr(not(test), rustc_diagnostic_item = "hashset_drain_ty")] pub struct Drain<'a, K: 'a> { base: base::Drain<'a, K>, } diff --git a/library/std/src/collections/hash/set/tests.rs b/library/std/src/collections/hash/set/tests.rs index 4e6351652721f..8ee8a3e8bf6ae 100644 --- a/library/std/src/collections/hash/set/tests.rs +++ b/library/std/src/collections/hash/set/tests.rs @@ -1,8 +1,8 @@ use super::HashSet; use crate::hash::RandomState; -use crate::panic::{catch_unwind, AssertUnwindSafe}; -use crate::sync::atomic::{AtomicU32, Ordering}; +use crate::panic::{AssertUnwindSafe, catch_unwind}; use crate::sync::Arc; +use crate::sync::atomic::{AtomicU32, Ordering}; #[test] fn test_zero_capacities() { @@ -429,6 +429,7 @@ fn test_extract_if() { } #[test] +#[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] fn test_extract_if_drop_panic_leak() { static PREDS: AtomicU32 = AtomicU32::new(0); static DROPS: AtomicU32 = AtomicU32::new(0); @@ -459,6 +460,7 @@ fn test_extract_if_drop_panic_leak() { } #[test] +#[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] fn test_extract_if_pred_panic_leak() { static PREDS: AtomicU32 = AtomicU32::new(0); static DROPS: AtomicU32 = AtomicU32::new(0); diff --git a/library/std/src/collections/mod.rs b/library/std/src/collections/mod.rs index 3b04412e76630..889ed3c538035 100644 --- a/library/std/src/collections/mod.rs +++ b/library/std/src/collections/mod.rs @@ -79,41 +79,49 @@ //! see each type's documentation, and note that the names of actual methods may //! differ from the tables below on certain collections. //! -//! Throughout the documentation, we will follow a few conventions. For all -//! operations, the collection's size is denoted by n. If another collection is -//! involved in the operation, it contains m elements. Operations which have an -//! *amortized* cost are suffixed with a `*`. Operations with an *expected* -//! cost are suffixed with a `~`. +//! Throughout the documentation, we will adhere to the following conventions +//! for operation notation: //! -//! All amortized costs are for the potential need to resize when capacity is -//! exhausted. If a resize occurs it will take *O*(*n*) time. Our collections never -//! automatically shrink, so removal operations aren't amortized. Over a -//! sufficiently large series of operations, the average cost per operation will -//! deterministically equal the given cost. +//! * The collection's size is denoted by `n`. +//! * If a second collection is involved, its size is denoted by `m`. +//! * Item indices are denoted by `i`. +//! * Operations which have an *amortized* cost are suffixed with a `*`. +//! * Operations with an *expected* cost are suffixed with a `~`. //! -//! Only [`HashMap`] has expected costs, due to the probabilistic nature of hashing. -//! It is theoretically possible, though very unlikely, for [`HashMap`] to -//! experience worse performance. +//! Calling operations that add to a collection will occasionally require a +//! collection to be resized - an extra operation that takes *O*(*n*) time. //! -//! ## Sequences +//! *Amortized* costs are calculated to account for the time cost of such resize +//! operations *over a sufficiently large series of operations*. An individual +//! operation may be slower or faster due to the sporadic nature of collection +//! resizing, however the average cost per operation will approach the amortized +//! cost. //! -//! | | get(i) | insert(i) | remove(i) | append | split_off(i) | -//! |----------------|------------------------|-------------------------|------------------------|-----------|------------------------| -//! | [`Vec`] | *O*(1) | *O*(*n*-*i*)* | *O*(*n*-*i*) | *O*(*m*)* | *O*(*n*-*i*) | -//! | [`VecDeque`] | *O*(1) | *O*(min(*i*, *n*-*i*))* | *O*(min(*i*, *n*-*i*)) | *O*(*m*)* | *O*(min(*i*, *n*-*i*)) | -//! | [`LinkedList`] | *O*(min(*i*, *n*-*i*)) | *O*(min(*i*, *n*-*i*)) | *O*(min(*i*, *n*-*i*)) | *O*(1) | *O*(min(*i*, *n*-*i*)) | +//! Rust's collections never automatically shrink, so removal operations aren't +//! amortized. //! -//! Note that where ties occur, [`Vec`] is generally going to be faster than [`VecDeque`], and -//! [`VecDeque`] is generally going to be faster than [`LinkedList`]. +//! [`HashMap`] uses *expected* costs. It is theoretically possible, though very +//! unlikely, for [`HashMap`] to experience significantly worse performance than +//! the expected cost. This is due to the probabilistic nature of hashing - i.e. +//! it is possible to generate a duplicate hash given some input key that will +//! requires extra computation to correct. //! -//! ## Maps +//! ## Cost of Collection Operations //! -//! For Sets, all operations have the cost of the equivalent Map operation. //! -//! | | get | insert | remove | range | append | -//! |--------------|---------------|---------------|---------------|---------------|--------------| -//! | [`HashMap`] | *O*(1)~ | *O*(1)~* | *O*(1)~ | N/A | N/A | -//! | [`BTreeMap`] | *O*(log(*n*)) | *O*(log(*n*)) | *O*(log(*n*)) | *O*(log(*n*)) | *O*(*n*+*m*) | +//! | | get(i) | insert(i) | remove(i) | append(Vec(m)) | split_off(i) | range | append | +//! |----------------|------------------------|-------------------------|------------------------|-------------------|------------------------|-----------------|--------------| +//! | [`Vec`] | *O*(1) | *O*(*n*-*i*)* | *O*(*n*-*i*) | *O*(*m*)* | *O*(*n*-*i*) | N/A | N/A | +//! | [`VecDeque`] | *O*(1) | *O*(min(*i*, *n*-*i*))* | *O*(min(*i*, *n*-*i*)) | *O*(*m*)* | *O*(min(*i*, *n*-*i*)) | N/A | N/A | +//! | [`LinkedList`] | *O*(min(*i*, *n*-*i*)) | *O*(min(*i*, *n*-*i*)) | *O*(min(*i*, *n*-*i*)) | *O*(1) | *O*(min(*i*, *n*-*i*)) | N/A | N/A | +//! | [`HashMap`] | *O*(1)~ | *O*(1)~* | *O*(1)~ | N/A | N/A | N/A | N/A | +//! | [`BTreeMap`] | *O*(log(*n*)) | *O*(log(*n*)) | *O*(log(*n*)) | N/A | N/A | *O*(log(*n*)) | *O*(*n*+*m*) | +//! +//! Note that where ties occur, [`Vec`] is generally going to be faster than +//! [`VecDeque`], and [`VecDeque`] is generally going to be faster than +//! [`LinkedList`]. +//! +//! For Sets, all operations have the cost of the equivalent Map operation. //! //! # Correct and Efficient Usage of Collections //! @@ -410,13 +418,13 @@ pub use alloc_crate::collections::TryReserveError; )] pub use alloc_crate::collections::TryReserveErrorKind; #[stable(feature = "rust1", since = "1.0.0")] -pub use alloc_crate::collections::{binary_heap, btree_map, btree_set}; -#[stable(feature = "rust1", since = "1.0.0")] -pub use alloc_crate::collections::{linked_list, vec_deque}; -#[stable(feature = "rust1", since = "1.0.0")] pub use alloc_crate::collections::{BTreeMap, BTreeSet, BinaryHeap}; #[stable(feature = "rust1", since = "1.0.0")] pub use alloc_crate::collections::{LinkedList, VecDeque}; +#[stable(feature = "rust1", since = "1.0.0")] +pub use alloc_crate::collections::{binary_heap, btree_map, btree_set}; +#[stable(feature = "rust1", since = "1.0.0")] +pub use alloc_crate::collections::{linked_list, vec_deque}; #[stable(feature = "rust1", since = "1.0.0")] #[doc(inline)] diff --git a/library/std/src/env.rs b/library/std/src/env.rs index 28916130b1900..d732a15117e9e 100644 --- a/library/std/src/env.rs +++ b/library/std/src/env.rs @@ -618,7 +618,7 @@ impl Error for JoinPathsError { /// /// # Deprecation /// -/// This function is deprecated because the behaviour on Windows is not correct. +/// This function is deprecated because the behavior on Windows is not correct. /// The 'HOME' environment variable is not standard on Windows, and may not produce /// desired results; for instance, under Cygwin or Mingw it will return `/home/you` /// when it should return `C:\Users\you`. @@ -935,106 +935,147 @@ impl fmt::Debug for ArgsOs { pub mod consts { use crate::sys::env::os; - /// A string describing the architecture of the CPU that is currently - /// in use. + /// A string describing the architecture of the CPU that is currently in use. + /// An example value may be: `"x86"`, `"arm"` or `"riscv64"`. /// - /// Some possible values: + ///

Full list of possible values /// - /// - x86 - /// - x86_64 - /// - arm - /// - aarch64 - /// - loongarch64 - /// - m68k - /// - csky - /// - mips - /// - mips64 - /// - powerpc - /// - powerpc64 - /// - riscv64 - /// - s390x - /// - sparc64 + /// * `"x86"` + /// * `"x86_64"` + /// * `"arm"` + /// * `"aarch64"` + /// * `"m68k"` + /// * `"mips"` + /// * `"mips32r6"` + /// * `"mips64"` + /// * `"mips64r6"` + /// * `"csky"` + /// * `"powerpc"` + /// * `"powerpc64"` + /// * `"riscv32"` + /// * `"riscv64"` + /// * `"s390x"` + /// * `"sparc"` + /// * `"sparc64"` + /// * `"hexagon"` + /// * `"loongarch64"` + /// + ///
#[stable(feature = "env", since = "1.0.0")] pub const ARCH: &str = env!("STD_ENV_ARCH"); - /// The family of the operating system. Example value is `unix`. + /// A string describing the family of the operating system. + /// An example value may be: `"unix"`, or `"windows"`. + /// + /// This value may be an empty string if the family is unknown. + /// + ///
Full list of possible values /// - /// Some possible values: + /// * `"unix"` + /// * `"windows"` + /// * `"itron"` + /// * `"wasm"` + /// * `""` /// - /// - unix - /// - windows + ///
#[stable(feature = "env", since = "1.0.0")] pub const FAMILY: &str = os::FAMILY; /// A string describing the specific operating system in use. - /// Example value is `linux`. + /// An example value may be: `"linux"`, or `"freebsd"`. /// - /// Some possible values: + ///
Full list of possible values /// - /// - linux - /// - macos - /// - ios - /// - freebsd - /// - dragonfly - /// - netbsd - /// - openbsd - /// - solaris - /// - android - /// - windows + /// * `"linux"` + /// * `"windows"` + /// * `"macos"` + /// * `"android"` + /// * `"ios"` + /// * `"openbsd"` + /// * `"freebsd"` + /// * `"netbsd"` + /// * `"wasi"` + /// * `"hermit"` + /// * `"aix"` + /// * `"apple"` + /// * `"dragonfly"` + /// * `"emscripten"` + /// * `"espidf"` + /// * `"fortanix"` + /// * `"uefi"` + /// * `"fuchsia"` + /// * `"haiku"` + /// * `"hermit"` + /// * `"watchos"` + /// * `"visionos"` + /// * `"tvos"` + /// * `"horizon"` + /// * `"hurd"` + /// * `"illumos"` + /// * `"l4re"` + /// * `"nto"` + /// * `"redox"` + /// * `"solaris"` + /// * `"solid_asp3` + /// * `"vita"` + /// * `"vxworks"` + /// * `"xous"` + /// + ///
#[stable(feature = "env", since = "1.0.0")] pub const OS: &str = os::OS; - /// Specifies the filename prefix used for shared libraries on this - /// platform. Example value is `lib`. - /// - /// Some possible values: - /// - /// - lib - /// - `""` (an empty string) + /// Specifies the filename prefix, if any, used for shared libraries on this platform. + /// This is either `"lib"` or an empty string. (`""`). #[stable(feature = "env", since = "1.0.0")] pub const DLL_PREFIX: &str = os::DLL_PREFIX; - /// Specifies the filename suffix used for shared libraries on this - /// platform. Example value is `.so`. - /// - /// Some possible values: + /// Specifies the filename suffix, if any, used for shared libraries on this platform. + /// An example value may be: `".so"`, `".elf"`, or `".dll"`. /// - /// - .so - /// - .dylib - /// - .dll + /// The possible values are identical to those of [`DLL_EXTENSION`], but with the leading period included. #[stable(feature = "env", since = "1.0.0")] pub const DLL_SUFFIX: &str = os::DLL_SUFFIX; - /// Specifies the file extension used for shared libraries on this - /// platform that goes after the dot. Example value is `so`. + /// Specifies the file extension, if any, used for shared libraries on this platform that goes after the dot. + /// An example value may be: `"so"`, `"elf"`, or `"dll"`. + /// + ///
Full list of possible values /// - /// Some possible values: + /// * `"so"` + /// * `"dylib"` + /// * `"dll"` + /// * `"sgxs"` + /// * `"a"` + /// * `"elf"` + /// * `"wasm"` + /// * `""` (an empty string) /// - /// - so - /// - dylib - /// - dll + ///
#[stable(feature = "env", since = "1.0.0")] pub const DLL_EXTENSION: &str = os::DLL_EXTENSION; - /// Specifies the filename suffix used for executable binaries on this - /// platform. Example value is `.exe`. + /// Specifies the filename suffix, if any, used for executable binaries on this platform. + /// An example value may be: `".exe"`, or `".efi"`. /// - /// Some possible values: - /// - /// - .exe - /// - .nexe - /// - .pexe - /// - `""` (an empty string) + /// The possible values are identical to those of [`EXE_EXTENSION`], but with the leading period included. #[stable(feature = "env", since = "1.0.0")] pub const EXE_SUFFIX: &str = os::EXE_SUFFIX; - /// Specifies the file extension, if any, used for executable binaries - /// on this platform. Example value is `exe`. + /// Specifies the file extension, if any, used for executable binaries on this platform. + /// An example value may be: `"exe"`, or an empty string (`""`). + /// + ///
Full list of possible values /// - /// Some possible values: + /// * `"exe"` + /// * `"efi"` + /// * `"js"` + /// * `"sgxs"` + /// * `"elf"` + /// * `"wasm"` + /// * `""` (an empty string) /// - /// - exe - /// - `""` (an empty string) + ///
#[stable(feature = "env", since = "1.0.0")] pub const EXE_EXTENSION: &str = os::EXE_EXTENSION; } diff --git a/library/std/src/env/tests.rs b/library/std/src/env/tests.rs index fc7aee2973329..d021726106872 100644 --- a/library/std/src/env/tests.rs +++ b/library/std/src/env/tests.rs @@ -1,7 +1,7 @@ use super::*; #[test] -#[cfg_attr(any(target_os = "emscripten", target_env = "sgx"), ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi", target_env = "sgx"), ignore)] fn test_self_exe_path() { let path = current_exe(); assert!(path.is_ok()); diff --git a/library/std/src/error.rs b/library/std/src/error.rs index 3e17431af45b0..b3e63aaf1c567 100644 --- a/library/std/src/error.rs +++ b/library/std/src/error.rs @@ -7,7 +7,7 @@ mod tests; #[stable(feature = "rust1", since = "1.0.0")] pub use core::error::Error; #[unstable(feature = "error_generic_member_access", issue = "99301")] -pub use core::error::{request_ref, request_value, Request}; +pub use core::error::{Request, request_ref, request_value}; use crate::backtrace::Backtrace; use crate::fmt::{self, Write}; diff --git a/library/std/src/f128.rs b/library/std/src/f128.rs index b436fe9929c36..229f979b5b10b 100644 --- a/library/std/src/f128.rs +++ b/library/std/src/f128.rs @@ -210,8 +210,9 @@ impl f128 { #[inline] #[rustc_allow_incoherent_impl] #[unstable(feature = "f128", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "method returns a new number and does not mutate the original value"] - pub fn abs(self) -> Self { + pub const fn abs(self) -> Self { // FIXME(f16_f128): replace with `intrinsics::fabsf128` when available // We don't do this now because LLVM has lowering bugs for f128 math. Self::from_bits(self.to_bits() & !(1 << 127)) @@ -240,8 +241,9 @@ impl f128 { #[inline] #[rustc_allow_incoherent_impl] #[unstable(feature = "f128", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "method returns a new number and does not mutate the original value"] - pub fn signum(self) -> f128 { + pub const fn signum(self) -> f128 { if self.is_nan() { Self::NAN } else { 1.0_f128.copysign(self) } } @@ -278,8 +280,9 @@ impl f128 { #[inline] #[rustc_allow_incoherent_impl] #[unstable(feature = "f128", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "method returns a new number and does not mutate the original value"] - pub fn copysign(self, sign: f128) -> f128 { + pub const fn copysign(self, sign: f128) -> f128 { unsafe { intrinsics::copysignf128(self, sign) } } diff --git a/library/std/src/f16.rs b/library/std/src/f16.rs index b2cd5fae9d04a..bed21cda1cd91 100644 --- a/library/std/src/f16.rs +++ b/library/std/src/f16.rs @@ -210,8 +210,9 @@ impl f16 { #[inline] #[rustc_allow_incoherent_impl] #[unstable(feature = "f16", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "method returns a new number and does not mutate the original value"] - pub fn abs(self) -> Self { + pub const fn abs(self) -> Self { // FIXME(f16_f128): replace with `intrinsics::fabsf16` when available Self::from_bits(self.to_bits() & !(1 << 15)) } @@ -239,8 +240,9 @@ impl f16 { #[inline] #[rustc_allow_incoherent_impl] #[unstable(feature = "f16", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "method returns a new number and does not mutate the original value"] - pub fn signum(self) -> f16 { + pub const fn signum(self) -> f16 { if self.is_nan() { Self::NAN } else { 1.0_f16.copysign(self) } } @@ -277,8 +279,9 @@ impl f16 { #[inline] #[rustc_allow_incoherent_impl] #[unstable(feature = "f16", issue = "116909")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[must_use = "method returns a new number and does not mutate the original value"] - pub fn copysign(self, sign: f16) -> f16 { + pub const fn copysign(self, sign: f16) -> f16 { unsafe { intrinsics::copysignf16(self, sign) } } diff --git a/library/std/src/f32.rs b/library/std/src/f32.rs index cafbe9761da19..30cf4e1f756e0 100644 --- a/library/std/src/f32.rs +++ b/library/std/src/f32.rs @@ -18,8 +18,8 @@ mod tests; #[stable(feature = "rust1", since = "1.0.0")] #[allow(deprecated, deprecated_in_future)] pub use core::f32::{ - consts, DIGITS, EPSILON, INFINITY, MANTISSA_DIGITS, MAX, MAX_10_EXP, MAX_EXP, MIN, MIN_10_EXP, - MIN_EXP, MIN_POSITIVE, NAN, NEG_INFINITY, RADIX, + DIGITS, EPSILON, INFINITY, MANTISSA_DIGITS, MAX, MAX_10_EXP, MAX_EXP, MIN, MIN_10_EXP, MIN_EXP, + MIN_POSITIVE, NAN, NEG_INFINITY, RADIX, consts, }; #[cfg(not(test))] @@ -194,8 +194,9 @@ impl f32 { #[rustc_allow_incoherent_impl] #[must_use = "method returns a new number and does not mutate the original value"] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn abs(self) -> f32 { + pub const fn abs(self) -> f32 { unsafe { intrinsics::fabsf32(self) } } @@ -218,8 +219,9 @@ impl f32 { #[rustc_allow_incoherent_impl] #[must_use = "method returns a new number and does not mutate the original value"] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn signum(self) -> f32 { + pub const fn signum(self) -> f32 { if self.is_nan() { Self::NAN } else { 1.0_f32.copysign(self) } } @@ -253,7 +255,8 @@ impl f32 { #[must_use = "method returns a new number and does not mutate the original value"] #[inline] #[stable(feature = "copysign", since = "1.35.0")] - pub fn copysign(self, sign: f32) -> f32 { + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] + pub const fn copysign(self, sign: f32) -> f32 { unsafe { intrinsics::copysignf32(self, sign) } } diff --git a/library/std/src/f32/tests.rs b/library/std/src/f32/tests.rs index 3a4c1c120a495..99cfcfb231dad 100644 --- a/library/std/src/f32/tests.rs +++ b/library/std/src/f32/tests.rs @@ -2,31 +2,24 @@ use crate::f32::consts; use crate::num::{FpCategory as Fp, *}; /// Smallest number -#[allow(dead_code)] // unused on x86 const TINY_BITS: u32 = 0x1; /// Next smallest number -#[allow(dead_code)] // unused on x86 const TINY_UP_BITS: u32 = 0x2; /// Exponent = 0b11...10, Sifnificand 0b1111..10. Min val > 0 -#[allow(dead_code)] // unused on x86 const MAX_DOWN_BITS: u32 = 0x7f7f_fffe; /// Zeroed exponent, full significant -#[allow(dead_code)] // unused on x86 const LARGEST_SUBNORMAL_BITS: u32 = 0x007f_ffff; /// Exponent = 0b1, zeroed significand -#[allow(dead_code)] // unused on x86 const SMALLEST_NORMAL_BITS: u32 = 0x0080_0000; /// First pattern over the mantissa -#[allow(dead_code)] // unused on x86 const NAN_MASK1: u32 = 0x002a_aaaa; /// Second pattern over the mantissa -#[allow(dead_code)] // unused on x86 const NAN_MASK2: u32 = 0x0055_5555; #[allow(unused_macros)] @@ -353,9 +346,6 @@ fn test_is_sign_negative() { assert!((-f32::NAN).is_sign_negative()); } -// Ignore test on x87 floating point, these platforms do not guarantee NaN -// payloads are preserved and flush denormals to zero, failing the tests. -#[cfg(not(target_arch = "x86"))] #[test] fn test_next_up() { let tiny = f32::from_bits(TINY_BITS); @@ -386,9 +376,6 @@ fn test_next_up() { assert_f32_biteq!(nan2.next_up(), nan2); } -// Ignore test on x87 floating point, these platforms do not guarantee NaN -// payloads are preserved and flush denormals to zero, failing the tests. -#[cfg(not(target_arch = "x86"))] #[test] fn test_next_down() { let tiny = f32::from_bits(TINY_BITS); diff --git a/library/std/src/f64.rs b/library/std/src/f64.rs index fba283e3a44bc..51d5476b372d2 100644 --- a/library/std/src/f64.rs +++ b/library/std/src/f64.rs @@ -18,8 +18,8 @@ mod tests; #[stable(feature = "rust1", since = "1.0.0")] #[allow(deprecated, deprecated_in_future)] pub use core::f64::{ - consts, DIGITS, EPSILON, INFINITY, MANTISSA_DIGITS, MAX, MAX_10_EXP, MAX_EXP, MIN, MIN_10_EXP, - MIN_EXP, MIN_POSITIVE, NAN, NEG_INFINITY, RADIX, + DIGITS, EPSILON, INFINITY, MANTISSA_DIGITS, MAX, MAX_10_EXP, MAX_EXP, MIN, MIN_10_EXP, MIN_EXP, + MIN_POSITIVE, NAN, NEG_INFINITY, RADIX, consts, }; #[cfg(not(test))] @@ -194,8 +194,9 @@ impl f64 { #[rustc_allow_incoherent_impl] #[must_use = "method returns a new number and does not mutate the original value"] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn abs(self) -> f64 { + pub const fn abs(self) -> f64 { unsafe { intrinsics::fabsf64(self) } } @@ -218,8 +219,9 @@ impl f64 { #[rustc_allow_incoherent_impl] #[must_use = "method returns a new number and does not mutate the original value"] #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn signum(self) -> f64 { + pub const fn signum(self) -> f64 { if self.is_nan() { Self::NAN } else { 1.0_f64.copysign(self) } } @@ -252,8 +254,9 @@ impl f64 { #[rustc_allow_incoherent_impl] #[must_use = "method returns a new number and does not mutate the original value"] #[stable(feature = "copysign", since = "1.35.0")] + #[rustc_const_unstable(feature = "const_float_methods", issue = "130843")] #[inline] - pub fn copysign(self, sign: f64) -> f64 { + pub const fn copysign(self, sign: f64) -> f64 { unsafe { intrinsics::copysignf64(self, sign) } } diff --git a/library/std/src/f64/tests.rs b/library/std/src/f64/tests.rs index bac8405f97361..3fac2efe0d76c 100644 --- a/library/std/src/f64/tests.rs +++ b/library/std/src/f64/tests.rs @@ -2,31 +2,24 @@ use crate::f64::consts; use crate::num::{FpCategory as Fp, *}; /// Smallest number -#[allow(dead_code)] // unused on x86 const TINY_BITS: u64 = 0x1; /// Next smallest number -#[allow(dead_code)] // unused on x86 const TINY_UP_BITS: u64 = 0x2; /// Exponent = 0b11...10, Sifnificand 0b1111..10. Min val > 0 -#[allow(dead_code)] // unused on x86 const MAX_DOWN_BITS: u64 = 0x7fef_ffff_ffff_fffe; /// Zeroed exponent, full significant -#[allow(dead_code)] // unused on x86 const LARGEST_SUBNORMAL_BITS: u64 = 0x000f_ffff_ffff_ffff; /// Exponent = 0b1, zeroed significand -#[allow(dead_code)] // unused on x86 const SMALLEST_NORMAL_BITS: u64 = 0x0010_0000_0000_0000; /// First pattern over the mantissa -#[allow(dead_code)] // unused on x86 const NAN_MASK1: u64 = 0x000a_aaaa_aaaa_aaaa; /// Second pattern over the mantissa -#[allow(dead_code)] // unused on x86 const NAN_MASK2: u64 = 0x0005_5555_5555_5555; #[allow(unused_macros)] @@ -343,9 +336,6 @@ fn test_is_sign_negative() { assert!((-f64::NAN).is_sign_negative()); } -// Ignore test on x87 floating point, these platforms do not guarantee NaN -// payloads are preserved and flush denormals to zero, failing the tests. -#[cfg(not(target_arch = "x86"))] #[test] fn test_next_up() { let tiny = f64::from_bits(TINY_BITS); @@ -375,9 +365,6 @@ fn test_next_up() { assert_f64_biteq!(nan2.next_up(), nan2); } -// Ignore test on x87 floating point, these platforms do not guarantee NaN -// payloads are preserved and flush denormals to zero, failing the tests. -#[cfg(not(target_arch = "x86"))] #[test] fn test_next_down() { let tiny = f64::from_bits(TINY_BITS); diff --git a/library/std/src/ffi/mod.rs b/library/std/src/ffi/mod.rs index 2b67750c2f0a9..469136be8838a 100644 --- a/library/std/src/ffi/mod.rs +++ b/library/std/src/ffi/mod.rs @@ -166,11 +166,6 @@ pub mod c_str; #[stable(feature = "core_c_void", since = "1.30.0")] pub use core::ffi::c_void; -#[stable(feature = "core_ffi_c", since = "1.64.0")] -pub use core::ffi::{ - c_char, c_double, c_float, c_int, c_long, c_longlong, c_schar, c_short, c_uchar, c_uint, - c_ulong, c_ulonglong, c_ushort, -}; #[unstable( feature = "c_variadic", reason = "the `c_variadic` feature has not been properly tested on \ @@ -178,6 +173,11 @@ pub use core::ffi::{ issue = "44930" )] pub use core::ffi::{VaList, VaListImpl}; +#[stable(feature = "core_ffi_c", since = "1.64.0")] +pub use core::ffi::{ + c_char, c_double, c_float, c_int, c_long, c_longlong, c_schar, c_short, c_uchar, c_uint, + c_ulong, c_ulonglong, c_ushort, +}; #[doc(no_inline)] #[stable(feature = "cstr_from_bytes_until_nul", since = "1.69.0")] diff --git a/library/std/src/ffi/os_str.rs b/library/std/src/ffi/os_str.rs index 99bea676e1224..2243f100643df 100644 --- a/library/std/src/ffi/os_str.rs +++ b/library/std/src/ffi/os_str.rs @@ -9,7 +9,6 @@ use crate::borrow::{Borrow, Cow}; use crate::collections::TryReserveError; use crate::hash::{Hash, Hasher}; use crate::ops::{self, Range}; -use crate::ptr::addr_of_mut; use crate::rc::Rc; use crate::str::FromStr; use crate::sync::Arc; @@ -196,6 +195,7 @@ impl OsString { /// let os_str = OsStr::new("foo"); /// assert_eq!(os_string.as_os_str(), os_str); /// ``` + #[cfg_attr(not(test), rustc_diagnostic_item = "os_string_as_os_str")] #[stable(feature = "rust1", since = "1.0.0")] #[must_use] #[inline] @@ -918,6 +918,7 @@ impl OsStr { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[inline] + #[cfg_attr(not(test), rustc_diagnostic_item = "os_str_to_os_string")] pub fn to_os_string(&self) -> OsString { OsString { inner: self.inner.to_owned() } } @@ -1270,7 +1271,7 @@ unsafe impl CloneToUninit for OsStr { #[cfg_attr(debug_assertions, track_caller)] unsafe fn clone_to_uninit(&self, dst: *mut Self) { // SAFETY: we're just a wrapper around a platform-specific Slice - unsafe { self.inner.clone_to_uninit(addr_of_mut!((*dst).inner)) } + unsafe { self.inner.clone_to_uninit(&raw mut (*dst).inner) } } } diff --git a/library/std/src/fs.rs b/library/std/src/fs.rs index 6a0d9f47960ec..3079c8b1d905a 100644 --- a/library/std/src/fs.rs +++ b/library/std/src/fs.rs @@ -8,7 +8,15 @@ #![stable(feature = "rust1", since = "1.0.0")] #![deny(unsafe_op_in_unsafe_fn)] -#[cfg(all(test, not(any(target_os = "emscripten", target_env = "sgx", target_os = "xous"))))] +#[cfg(all( + test, + not(any( + target_os = "emscripten", + target_os = "wasi", + target_env = "sgx", + target_os = "xous" + )) +))] mod tests; use crate::ffi::OsString; @@ -375,6 +383,45 @@ impl File { OpenOptions::new().read(true).open(path.as_ref()) } + /// Attempts to open a file in read-only mode with buffering. + /// + /// See the [`OpenOptions::open`] method, the [`BufReader`][io::BufReader] type, + /// and the [`BufRead`][io::BufRead] trait for more details. + /// + /// If you only need to read the entire file contents, + /// consider [`std::fs::read()`][self::read] or + /// [`std::fs::read_to_string()`][self::read_to_string] instead. + /// + /// # Errors + /// + /// This function will return an error if `path` does not already exist, + /// or if memory allocation fails for the new buffer. + /// Other errors may also be returned according to [`OpenOptions::open`]. + /// + /// # Examples + /// + /// ```no_run + /// #![feature(file_buffered)] + /// use std::fs::File; + /// use std::io::BufRead; + /// + /// fn main() -> std::io::Result<()> { + /// let mut f = File::open_buffered("foo.txt")?; + /// assert!(f.capacity() > 0); + /// for (line, i) in f.lines().zip(1..) { + /// println!("{i:6}: {}", line?); + /// } + /// Ok(()) + /// } + /// ``` + #[unstable(feature = "file_buffered", issue = "130804")] + pub fn open_buffered>(path: P) -> io::Result> { + // Allocate the buffer *first* so we don't affect the filesystem otherwise. + let buffer = io::BufReader::::try_new_buffer()?; + let file = File::open(path)?; + Ok(io::BufReader::with_buffer(file, buffer)) + } + /// Opens a file in write-only mode. /// /// This function will create a file if it does not exist, @@ -404,6 +451,45 @@ impl File { OpenOptions::new().write(true).create(true).truncate(true).open(path.as_ref()) } + /// Opens a file in write-only mode with buffering. + /// + /// This function will create a file if it does not exist, + /// and will truncate it if it does. + /// + /// Depending on the platform, this function may fail if the + /// full directory path does not exist. + /// + /// See the [`OpenOptions::open`] method and the + /// [`BufWriter`][io::BufWriter] type for more details. + /// + /// See also [`std::fs::write()`][self::write] for a simple function to + /// create a file with some given data. + /// + /// # Examples + /// + /// ```no_run + /// #![feature(file_buffered)] + /// use std::fs::File; + /// use std::io::Write; + /// + /// fn main() -> std::io::Result<()> { + /// let mut f = File::create_buffered("foo.txt")?; + /// assert!(f.capacity() > 0); + /// for i in 0..100 { + /// writeln!(&mut f, "{i}")?; + /// } + /// f.flush()?; + /// Ok(()) + /// } + /// ``` + #[unstable(feature = "file_buffered", issue = "130804")] + pub fn create_buffered>(path: P) -> io::Result> { + // Allocate the buffer *first* so we don't affect the filesystem otherwise. + let buffer = io::BufWriter::::try_new_buffer()?; + let file = File::create(path)?; + Ok(io::BufWriter::with_buffer(file, buffer)) + } + /// Creates a new file in read-write mode; error if the file exists. /// /// This function will create a file if it does not exist, or return an error if it does. This @@ -466,6 +552,7 @@ impl File { /// ``` #[must_use] #[stable(feature = "with_options", since = "1.58.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "file_options")] pub fn options() -> OpenOptions { OpenOptions::new() } @@ -835,7 +922,7 @@ impl Read for &File { } #[stable(feature = "rust1", since = "1.0.0")] impl Write for &File { - /// Writes some bytes from the file. + /// Writes some bytes to the file. /// /// See [`Write::write`] docs for more info. /// @@ -1009,6 +1096,7 @@ impl OpenOptions { /// let mut options = OpenOptions::new(); /// let file = options.read(true).open("foo.txt"); /// ``` + #[cfg_attr(not(test), rustc_diagnostic_item = "open_options_new")] #[stable(feature = "rust1", since = "1.0.0")] #[must_use] pub fn new() -> Self { @@ -1109,7 +1197,7 @@ impl OpenOptions { /// Sets the option for truncating a previous file. /// - /// If a file is successfully opened with this option set it will truncate + /// If a file is successfully opened with this option set to true, it will truncate /// the file to 0 length if it already exists. /// /// The file must be opened with write access for truncate to work. @@ -1989,6 +2077,11 @@ impl AsInner for DirEntry { /// * The file doesn't exist. /// * The user lacks permissions to remove the file. /// +/// This function will only ever return an error of kind `NotFound` if the given +/// path does not exist. Note that the inverse is not true, +/// ie. if a path does not exist, its removal may fail for a number of reasons, +/// such as insufficient permissions. +/// /// # Examples /// /// ```no_run @@ -2446,6 +2539,11 @@ pub fn create_dir_all>(path: P) -> io::Result<()> { /// * The user lacks permissions to remove the directory at the provided `path`. /// * The directory isn't empty. /// +/// This function will only ever return an error of kind `NotFound` if the given +/// path does not exist. Note that the inverse is not true, +/// ie. if a path does not exist, its removal may fail for a number of reasons, +/// such as insufficient permissions. +/// /// # Examples /// /// ```no_run @@ -2471,16 +2569,15 @@ pub fn remove_dir>(path: P) -> io::Result<()> { /// # Platform-specific behavior /// /// This function currently corresponds to `openat`, `fdopendir`, `unlinkat` and `lstat` functions -/// on Unix (except for macOS before version 10.10 and REDOX) and the `CreateFileW`, -/// `GetFileInformationByHandleEx`, `SetFileInformationByHandle`, and `NtCreateFile` functions on -/// Windows. Note that, this [may change in the future][changes]. +/// on Unix (except for REDOX) and the `CreateFileW`, `GetFileInformationByHandleEx`, +/// `SetFileInformationByHandle`, and `NtCreateFile` functions on Windows. Note that, this +/// [may change in the future][changes]. /// /// [changes]: io#platform-specific-behavior /// -/// On macOS before version 10.10 and REDOX, as well as when running in Miri for any target, this -/// function is not protected against time-of-check to time-of-use (TOCTOU) race conditions, and -/// should not be used in security-sensitive code on those platforms. All other platforms are -/// protected. +/// On REDOX, as well as when running in Miri for any target, this function is not protected against +/// time-of-check to time-of-use (TOCTOU) race conditions, and should not be used in +/// security-sensitive code on those platforms. All other platforms are protected. /// /// # Errors /// @@ -2521,7 +2618,7 @@ pub fn remove_dir_all>(path: P) -> io::Result<()> { /// # Platform-specific behavior /// /// This function currently corresponds to the `opendir` function on Unix -/// and the `FindFirstFile` function on Windows. Advancing the iterator +/// and the `FindFirstFileEx` function on Windows. Advancing the iterator /// currently corresponds to `readdir` on Unix and `FindNextFile` on Windows. /// Note that, this [may change in the future][changes]. /// diff --git a/library/std/src/fs/tests.rs b/library/std/src/fs/tests.rs index 13028c4c3b57e..0672fe6f7718a 100644 --- a/library/std/src/fs/tests.rs +++ b/library/std/src/fs/tests.rs @@ -1,7 +1,5 @@ use rand::RngCore; -#[cfg(target_os = "macos")] -use crate::ffi::{c_char, c_int}; use crate::fs::{self, File, FileTimes, OpenOptions}; use crate::io::prelude::*; use crate::io::{BorrowedBuf, ErrorKind, SeekFrom}; @@ -13,12 +11,10 @@ use crate::os::unix::fs::symlink as symlink_file; #[cfg(unix)] use crate::os::unix::fs::symlink as junction_point; #[cfg(windows)] -use crate::os::windows::fs::{junction_point, symlink_dir, symlink_file, OpenOptionsExt}; +use crate::os::windows::fs::{OpenOptionsExt, junction_point, symlink_dir, symlink_file}; use crate::path::Path; use crate::sync::Arc; -#[cfg(target_os = "macos")] -use crate::sys::weak::weak; -use crate::sys_common::io::test::{tmpdir, TempDir}; +use crate::sys_common::io::test::{TempDir, tmpdir}; use crate::time::{Duration, Instant, SystemTime}; use crate::{env, str, thread}; @@ -80,17 +76,6 @@ pub fn got_symlink_permission(tmpdir: &TempDir) -> bool { } } -#[cfg(target_os = "macos")] -fn able_to_not_follow_symlinks_while_hard_linking() -> bool { - weak!(fn linkat(c_int, *const c_char, c_int, *const c_char, c_int) -> c_int); - linkat.get().is_some() -} - -#[cfg(not(target_os = "macos"))] -fn able_to_not_follow_symlinks_while_hard_linking() -> bool { - return true; -} - #[test] fn file_test_io_smoke_test() { let message = "it's alright. have a good time"; @@ -1456,9 +1441,6 @@ fn symlink_hard_link() { if !got_symlink_permission(&tmpdir) { return; }; - if !able_to_not_follow_symlinks_while_hard_linking() { - return; - } // Create "file", a file. check!(fs::File::create(tmpdir.join("file"))); @@ -1750,7 +1732,7 @@ fn windows_unix_socket_exists() { let bytes = core::slice::from_raw_parts(bytes.as_ptr().cast::(), bytes.len()); addr.sun_path[..bytes.len()].copy_from_slice(bytes); let len = mem::size_of_val(&addr) as i32; - let result = c::bind(socket, ptr::addr_of!(addr).cast::(), len); + let result = c::bind(socket, (&raw const addr).cast::(), len); c::closesocket(socket); assert_eq!(result, 0); } diff --git a/library/std/src/hash/random.rs b/library/std/src/hash/random.rs index 8ef45172eac40..40f3a90f60c8a 100644 --- a/library/std/src/hash/random.rs +++ b/library/std/src/hash/random.rs @@ -10,7 +10,8 @@ #[allow(deprecated)] use super::{BuildHasher, Hasher, SipHasher13}; use crate::cell::Cell; -use crate::{fmt, sys}; +use crate::fmt; +use crate::sys::random::hashmap_random_keys; /// `RandomState` is the default state for [`HashMap`] types. /// @@ -65,7 +66,7 @@ impl RandomState { // increment one of the seeds on every RandomState creation, giving // every corresponding HashMap a different iteration order. thread_local!(static KEYS: Cell<(u64, u64)> = { - Cell::new(sys::hashmap_random_keys()) + Cell::new(hashmap_random_keys()) }); KEYS.with(|keys| { diff --git a/library/std/src/io/buffered/bufreader.rs b/library/std/src/io/buffered/bufreader.rs index cf226bd28d005..8b46738ab8aee 100644 --- a/library/std/src/io/buffered/bufreader.rs +++ b/library/std/src/io/buffered/bufreader.rs @@ -4,8 +4,8 @@ use buffer::Buffer; use crate::fmt; use crate::io::{ - self, uninlined_slow_read_byte, BorrowedCursor, BufRead, IoSliceMut, Read, Seek, SeekFrom, - SizeHint, SpecReadByte, DEFAULT_BUF_SIZE, + self, BorrowedCursor, BufRead, DEFAULT_BUF_SIZE, IoSliceMut, Read, Seek, SeekFrom, SizeHint, + SpecReadByte, uninlined_slow_read_byte, }; /// The `BufReader` struct adds buffering to any reader. @@ -74,6 +74,14 @@ impl BufReader { BufReader::with_capacity(DEFAULT_BUF_SIZE, inner) } + pub(crate) fn try_new_buffer() -> io::Result { + Buffer::try_with_capacity(DEFAULT_BUF_SIZE) + } + + pub(crate) fn with_buffer(inner: R, buf: Buffer) -> Self { + Self { inner, buf } + } + /// Creates a new `BufReader` with the specified buffer capacity. /// /// # Examples @@ -99,7 +107,10 @@ impl BufReader { impl BufReader { /// Attempt to look ahead `n` bytes. /// - /// `n` must be less than `capacity`. + /// `n` must be less than or equal to `capacity`. + /// + /// the returned slice may be less than `n` bytes long if + /// end of file is reached. /// /// ## Examples /// @@ -117,6 +128,7 @@ impl BufReader { /// let mut s = String::new(); /// rdr.read_to_string(&mut s).unwrap(); /// assert_eq!(&s, "hello"); + /// assert_eq!(rdr.peek(1).unwrap().len(), 0); /// ``` #[unstable(feature = "bufreader_peek", issue = "128405")] pub fn peek(&mut self, n: usize) -> io::Result<&[u8]> { @@ -125,7 +137,11 @@ impl BufReader { if self.buf.pos() > 0 { self.buf.backshift(); } - self.buf.read_more(&mut self.inner)?; + let new = self.buf.read_more(&mut self.inner)?; + if new == 0 { + // end of file, no more bytes to read + return Ok(&self.buf.buffer()[..]); + } debug_assert_eq!(self.buf.pos(), 0); } Ok(&self.buf.buffer()[..n]) @@ -267,6 +283,7 @@ impl BufReader { // This is only used by a test which asserts that the initialization-tracking is correct. #[cfg(test)] impl BufReader { + #[allow(missing_docs)] pub fn initialized(&self) -> usize { self.buf.initialized() } @@ -340,7 +357,7 @@ impl Read for BufReader { let prev = cursor.written(); let mut rem = self.fill_buf()?; - rem.read_buf(cursor.reborrow())?; + rem.read_buf(cursor.reborrow())?; // actually never fails self.consume(cursor.written() - prev); //slice impl of read_buf known to never unfill buf diff --git a/library/std/src/io/buffered/bufreader/buffer.rs b/library/std/src/io/buffered/bufreader/buffer.rs index ccd67fafb45b4..52fe49985c65a 100644 --- a/library/std/src/io/buffered/bufreader/buffer.rs +++ b/library/std/src/io/buffered/bufreader/buffer.rs @@ -10,7 +10,7 @@ //! without encountering any runtime bounds checks. use crate::cmp; -use crate::io::{self, BorrowedBuf, Read}; +use crate::io::{self, BorrowedBuf, ErrorKind, Read}; use crate::mem::MaybeUninit; pub struct Buffer { @@ -36,6 +36,16 @@ impl Buffer { Self { buf, pos: 0, filled: 0, initialized: 0 } } + #[inline] + pub fn try_with_capacity(capacity: usize) -> io::Result { + match Box::try_new_uninit_slice(capacity) { + Ok(buf) => Ok(Self { buf, pos: 0, filled: 0, initialized: 0 }), + Err(_) => { + Err(io::const_io_error!(ErrorKind::OutOfMemory, "failed to allocate read buffer")) + } + } + } + #[inline] pub fn buffer(&self) -> &[u8] { // SAFETY: self.pos and self.cap are valid, and self.cap => self.pos, and @@ -98,7 +108,7 @@ impl Buffer { } /// Read more bytes into the buffer without discarding any of its contents - pub fn read_more(&mut self, mut reader: impl Read) -> io::Result<()> { + pub fn read_more(&mut self, mut reader: impl Read) -> io::Result { let mut buf = BorrowedBuf::from(&mut self.buf[self.pos..]); let old_init = self.initialized - self.pos; unsafe { @@ -107,7 +117,7 @@ impl Buffer { reader.read_buf(buf.unfilled())?; self.filled += buf.len(); self.initialized += buf.init_len() - old_init; - Ok(()) + Ok(buf.len()) } /// Remove bytes that have already been read from the buffer. @@ -133,11 +143,13 @@ impl Buffer { buf.set_init(self.initialized); } - reader.read_buf(buf.unfilled())?; + let result = reader.read_buf(buf.unfilled()); self.pos = 0; self.filled = buf.len(); self.initialized = buf.init_len(); + + result?; } Ok(self.buffer()) } diff --git a/library/std/src/io/buffered/bufwriter.rs b/library/std/src/io/buffered/bufwriter.rs index 21650d467446e..c41bae2aa4e81 100644 --- a/library/std/src/io/buffered/bufwriter.rs +++ b/library/std/src/io/buffered/bufwriter.rs @@ -1,5 +1,5 @@ use crate::io::{ - self, ErrorKind, IntoInnerError, IoSlice, Seek, SeekFrom, Write, DEFAULT_BUF_SIZE, + self, DEFAULT_BUF_SIZE, ErrorKind, IntoInnerError, IoSlice, Seek, SeekFrom, Write, }; use crate::mem::{self, ManuallyDrop}; use crate::{error, fmt, ptr}; @@ -94,6 +94,16 @@ impl BufWriter { BufWriter::with_capacity(DEFAULT_BUF_SIZE, inner) } + pub(crate) fn try_new_buffer() -> io::Result> { + Vec::try_with_capacity(DEFAULT_BUF_SIZE).map_err(|_| { + io::const_io_error!(ErrorKind::OutOfMemory, "failed to allocate write buffer") + }) + } + + pub(crate) fn with_buffer(inner: W, buf: Vec) -> Self { + Self { inner, buf, panicked: false } + } + /// Creates a new `BufWriter` with at least the specified buffer capacity. /// /// # Examples diff --git a/library/std/src/io/buffered/tests.rs b/library/std/src/io/buffered/tests.rs index d89ecd317d6ee..bff0f823c4b5a 100644 --- a/library/std/src/io/buffered/tests.rs +++ b/library/std/src/io/buffered/tests.rs @@ -164,6 +164,7 @@ fn test_buffered_reader_stream_position() { } #[test] +#[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] fn test_buffered_reader_stream_position_panic() { let inner: &[u8] = &[5, 6, 7, 0, 1, 2, 3, 4]; let mut reader = BufReader::with_capacity(4, io::Cursor::new(inner)); @@ -487,7 +488,7 @@ fn dont_panic_in_drop_on_panicked_flush() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn panic_in_write_doesnt_flush_in_drop() { static WRITES: AtomicUsize = AtomicUsize::new(0); diff --git a/library/std/src/io/copy.rs b/library/std/src/io/copy.rs index d49866345cbf6..8d733325b3be7 100644 --- a/library/std/src/io/copy.rs +++ b/library/std/src/io/copy.rs @@ -1,4 +1,4 @@ -use super::{BorrowedBuf, BufReader, BufWriter, Read, Result, Write, DEFAULT_BUF_SIZE}; +use super::{BorrowedBuf, BufReader, BufWriter, DEFAULT_BUF_SIZE, Read, Result, Write}; use crate::alloc::Allocator; use crate::cmp; use crate::collections::VecDeque; diff --git a/library/std/src/io/copy/tests.rs b/library/std/src/io/copy/tests.rs index 7e08826a7e1d8..2e0eb6cdce666 100644 --- a/library/std/src/io/copy/tests.rs +++ b/library/std/src/io/copy/tests.rs @@ -122,8 +122,8 @@ mod io_benches { use test::Bencher; use crate::fs::{File, OpenOptions}; - use crate::io::prelude::*; use crate::io::BufReader; + use crate::io::prelude::*; #[bench] fn bench_copy_buf_reader(b: &mut Bencher) { diff --git a/library/std/src/io/error.rs b/library/std/src/io/error.rs index e8ae1d99fbf37..adf103e9430b8 100644 --- a/library/std/src/io/error.rs +++ b/library/std/src/io/error.rs @@ -223,10 +223,10 @@ pub enum ErrorKind { #[stable(feature = "rust1", since = "1.0.0")] ConnectionReset, /// The remote host is not reachable. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] HostUnreachable, /// The network containing the remote host is not reachable. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] NetworkUnreachable, /// The connection was aborted (terminated) by the remote server. #[stable(feature = "rust1", since = "1.0.0")] @@ -243,7 +243,7 @@ pub enum ErrorKind { #[stable(feature = "rust1", since = "1.0.0")] AddrNotAvailable, /// The system's networking is down. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] NetworkDown, /// The operation failed because a pipe was closed. #[stable(feature = "rust1", since = "1.0.0")] @@ -259,18 +259,18 @@ pub enum ErrorKind { /// /// For example, a filesystem path was specified where one of the intermediate directory /// components was, in fact, a plain file. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] NotADirectory, /// The filesystem object is, unexpectedly, a directory. /// /// A directory was specified when a non-directory was expected. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] IsADirectory, /// A non-empty directory was specified where an empty directory was expected. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] DirectoryNotEmpty, /// The filesystem or storage medium is read-only, but a write operation was attempted. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] ReadOnlyFilesystem, /// Loop in the filesystem or IO subsystem; often, too many levels of symbolic links. /// @@ -285,7 +285,7 @@ pub enum ErrorKind { /// /// With some network filesystems, notably NFS, an open file (or directory) can be invalidated /// by problems with the network or server. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] StaleNetworkFileHandle, /// A parameter was incorrect. #[stable(feature = "rust1", since = "1.0.0")] @@ -319,13 +319,13 @@ pub enum ErrorKind { /// The underlying storage (typically, a filesystem) is full. /// /// This does not include out of quota errors. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] StorageFull, /// Seek on unseekable file. /// /// Seeking was attempted on an open file handle which is not suitable for seeking - for /// example, on Unix, a named pipe opened with `File::open`. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] NotSeekable, /// Filesystem quota was exceeded. #[unstable(feature = "io_error_more", issue = "86442")] @@ -335,22 +335,22 @@ pub enum ErrorKind { /// This might arise from a hard limit of the underlying filesystem or file access API, or from /// an administratively imposed resource limitation. Simple disk full, and out of quota, have /// their own errors. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] FileTooLarge, /// Resource is busy. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] ResourceBusy, /// Executable file is busy. /// /// An attempt was made to write to a file which is also in use as a running program. (Not all /// operating systems detect this situation.) - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] ExecutableFileBusy, /// Deadlock (avoided). /// /// A file locking operation would result in deadlock. This situation is typically detected, if /// at all, on a best-effort basis. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] Deadlock, /// Cross-device or cross-filesystem (hard) link or rename. #[unstable(feature = "io_error_more", issue = "86442")] @@ -358,7 +358,7 @@ pub enum ErrorKind { /// Too many (hard) links to the same filesystem object. /// /// The filesystem does not support making so many hardlinks to the same file. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] TooManyLinks, /// A filename was invalid. /// @@ -369,7 +369,7 @@ pub enum ErrorKind { /// /// When trying to run an external program, a system or process limit on the size of the /// arguments would have been exceeded. - #[unstable(feature = "io_error_more", issue = "86442")] + #[stable(feature = "io_error_a_bit_more", since = "1.83.0")] ArgumentListTooLong, /// This operation was interrupted. /// @@ -400,6 +400,11 @@ pub enum ErrorKind { #[stable(feature = "out_of_memory_error", since = "1.54.0")] OutOfMemory, + /// The operation was partially successful and needs to be checked + /// later on due to not blocking. + #[unstable(feature = "io_error_inprogress", issue = "130840")] + InProgress, + // "Unusual" error kinds which do not correspond simply to (sets // of) OS error codes, should be added just above this comment. // `Other` and `Uncategorized` should remain at the end: @@ -449,6 +454,7 @@ impl ErrorKind { FilesystemQuotaExceeded => "filesystem quota exceeded", HostUnreachable => "host unreachable", Interrupted => "operation interrupted", + InProgress => "in progress", InvalidData => "invalid data", InvalidFilename => "invalid filename", InvalidInput => "invalid input parameter", diff --git a/library/std/src/io/error/repr_bitpacked.rs b/library/std/src/io/error/repr_bitpacked.rs index 9d3ade46bd929..a839a2fbac117 100644 --- a/library/std/src/io/error/repr_bitpacked.rs +++ b/library/std/src/io/error/repr_bitpacked.rs @@ -124,6 +124,7 @@ const TAG_SIMPLE: usize = 0b11; /// is_unwind_safe::(); /// ``` #[repr(transparent)] +#[rustc_insignificant_dtor] pub(super) struct Repr(NonNull<()>, PhantomData>>); // All the types `Repr` stores internally are Send + Sync, and so is it. @@ -348,6 +349,7 @@ fn kind_from_prim(ek: u32) -> Option { UnexpectedEof, Unsupported, OutOfMemory, + InProgress, Uncategorized, }) } diff --git a/library/std/src/io/error/tests.rs b/library/std/src/io/error/tests.rs index 064e2e36b7a1d..00d04984a3854 100644 --- a/library/std/src/io/error/tests.rs +++ b/library/std/src/io/error/tests.rs @@ -1,4 +1,4 @@ -use super::{const_io_error, Custom, Error, ErrorData, ErrorKind, Repr, SimpleMessage}; +use super::{Custom, Error, ErrorData, ErrorKind, Repr, SimpleMessage, const_io_error}; use crate::assert_matches::assert_matches; use crate::mem::size_of; use crate::sys::decode_error_kind; diff --git a/library/std/src/io/impls.rs b/library/std/src/io/impls.rs index 85023540a816f..b952c85addf65 100644 --- a/library/std/src/io/impls.rs +++ b/library/std/src/io/impls.rs @@ -453,6 +453,29 @@ impl Read for VecDeque { Ok(n) } + fn read_exact(&mut self, buf: &mut [u8]) -> io::Result<()> { + let (front, back) = self.as_slices(); + + // Use only the front buffer if it is big enough to fill `buf`, else use + // the back buffer too. + match buf.split_at_mut_checked(front.len()) { + None => buf.copy_from_slice(&front[..buf.len()]), + Some((buf_front, buf_back)) => match back.split_at_checked(buf_back.len()) { + Some((back, _)) => { + buf_front.copy_from_slice(front); + buf_back.copy_from_slice(back); + } + None => { + self.clear(); + return Err(io::Error::READ_EXACT_EOF); + } + }, + } + + self.drain(..buf.len()); + Ok(()) + } + #[inline] fn read_buf(&mut self, cursor: BorrowedCursor<'_>) -> io::Result<()> { let (ref mut front, _) = self.as_slices(); @@ -462,6 +485,29 @@ impl Read for VecDeque { Ok(()) } + fn read_buf_exact(&mut self, mut cursor: BorrowedCursor<'_>) -> io::Result<()> { + let len = cursor.capacity(); + let (front, back) = self.as_slices(); + + match front.split_at_checked(cursor.capacity()) { + Some((front, _)) => cursor.append(front), + None => { + cursor.append(front); + match back.split_at_checked(cursor.capacity()) { + Some((back, _)) => cursor.append(back), + None => { + cursor.append(back); + self.clear(); + return Err(io::Error::READ_EXACT_EOF); + } + } + } + } + + self.drain(..len); + Ok(()) + } + #[inline] fn read_to_end(&mut self, buf: &mut Vec) -> io::Result { // The total len is known upfront so we can reserve it in a single call. diff --git a/library/std/src/io/mod.rs b/library/std/src/io/mod.rs index 644b294db8da1..71dfd0676c942 100644 --- a/library/std/src/io/mod.rs +++ b/library/std/src/io/mod.rs @@ -307,9 +307,9 @@ pub(crate) use error::const_io_error; pub use self::buffered::WriterPanicked; #[unstable(feature = "raw_os_error_ty", issue = "107792")] pub use self::error::RawOsError; -pub(crate) use self::stdio::attempt_print_to_stderr; #[stable(feature = "is_terminal", since = "1.70.0")] pub use self::stdio::IsTerminal; +pub(crate) use self::stdio::attempt_print_to_stderr; #[unstable(feature = "print_internals", issue = "none")] #[doc(hidden)] pub use self::stdio::{_eprint, _print}; @@ -322,8 +322,8 @@ pub use self::{ copy::copy, cursor::Cursor, error::{Error, ErrorKind, Result}, - stdio::{stderr, stdin, stdout, Stderr, StderrLock, Stdin, StdinLock, Stdout, StdoutLock}, - util::{empty, repeat, sink, Empty, Repeat, Sink}, + stdio::{Stderr, StderrLock, Stdin, StdinLock, Stdout, StdoutLock, stderr, stdin, stdout}, + util::{Empty, Repeat, Sink, empty, repeat, sink}, }; use crate::mem::take; use crate::ops::{Deref, DerefMut}; @@ -398,8 +398,7 @@ where // - avoid passing large buffers to readers that always initialize the free capacity if they perform short reads (#23815, #23820) // - pass large buffers to readers that do not initialize the spare capacity. this can amortize per-call overheads // - and finally pass not-too-small and not-too-large buffers to Windows read APIs because they manage to suffer from both problems -// at the same time, i.e. small reads suffer from syscall overhead, all reads incur initialization cost -// proportional to buffer size (#110650) +// at the same time, i.e. small reads suffer from syscall overhead, all reads incur costs proportional to buffer size (#110650) // pub(crate) fn default_read_to_end( r: &mut R, @@ -444,6 +443,8 @@ pub(crate) fn default_read_to_end( } } + let mut consecutive_short_reads = 0; + loop { if buf.len() == buf.capacity() && buf.capacity() == start_cap { // The buffer might be an exact fit. Let's read into a probe buffer @@ -473,37 +474,50 @@ pub(crate) fn default_read_to_end( } let mut cursor = read_buf.unfilled(); - loop { + let result = loop { match r.read_buf(cursor.reborrow()) { - Ok(()) => break, Err(e) if e.is_interrupted() => continue, - Err(e) => return Err(e), + // Do not stop now in case of error: we might have received both data + // and an error + res => break res, } - } + }; let unfilled_but_initialized = cursor.init_ref().len(); let bytes_read = cursor.written(); let was_fully_initialized = read_buf.init_len() == buf_len; + // SAFETY: BorrowedBuf's invariants mean this much memory is initialized. + unsafe { + let new_len = bytes_read + buf.len(); + buf.set_len(new_len); + } + + // Now that all data is pushed to the vector, we can fail without data loss + result?; + if bytes_read == 0 { return Ok(buf.len() - start_len); } + if bytes_read < buf_len { + consecutive_short_reads += 1; + } else { + consecutive_short_reads = 0; + } + // store how much was initialized but not filled initialized = unfilled_but_initialized; - // SAFETY: BorrowedBuf's invariants mean this much memory is initialized. - unsafe { - let new_len = bytes_read + buf.len(); - buf.set_len(new_len); - } - // Use heuristics to determine the max read size if no initial size hint was provided if size_hint.is_none() { // The reader is returning short reads but it doesn't call ensure_init(). // In that case we no longer need to restrict read sizes to avoid // initialization costs. - if !was_fully_initialized { + // When reading from disk we usually don't get any short reads except at EOF. + // So we wait for at least 2 short reads before uncapping the read buffer; + // this helps with the Windows issue. + if !was_fully_initialized && consecutive_short_reads > 1 { max_read_size = usize::MAX; } @@ -964,6 +978,8 @@ pub trait Read { /// with uninitialized buffers. The new data will be appended to any existing contents of `buf`. /// /// The default implementation delegates to `read`. + /// + /// This method makes it possible to return both data and an error but it is advised against. #[unstable(feature = "read_buf", issue = "78485")] fn read_buf(&mut self, buf: BorrowedCursor<'_>) -> Result<()> { default_read_buf(|b| self.read(b), buf) @@ -2058,6 +2074,7 @@ pub trait Seek { /// It is used by the [`Seek`] trait. #[derive(Copy, PartialEq, Eq, Clone, Debug)] #[stable(feature = "rust1", since = "1.0.0")] +#[cfg_attr(not(test), rustc_diagnostic_item = "SeekFrom")] pub enum SeekFrom { /// Sets the offset to the provided number of bytes. #[stable(feature = "rust1", since = "1.0.0")] @@ -2365,8 +2382,6 @@ pub trait BufRead: Read { /// about Ferris from a binary string, skipping the fun fact: /// /// ``` - /// #![feature(bufread_skip_until)] - /// /// use std::io::{self, BufRead}; /// /// let mut cursor = io::Cursor::new(b"Ferris\0Likes long walks on the beach\0Crustacean\0"); @@ -2390,7 +2405,7 @@ pub trait BufRead: Read { /// assert_eq!(num_bytes, 11); /// assert_eq!(animal, b"Crustacean\0"); /// ``` - #[unstable(feature = "bufread_skip_until", issue = "111735")] + #[stable(feature = "bufread_skip_until", since = "1.83.0")] fn skip_until(&mut self, byte: u8) -> Result { skip_until(self, byte) } @@ -2930,7 +2945,7 @@ impl Read for Take { } let mut cursor = sliced_buf.unfilled(); - self.inner.read_buf(cursor.reborrow())?; + let result = self.inner.read_buf(cursor.reborrow()); let new_init = cursor.init_ref().len(); let filled = sliced_buf.len(); @@ -2945,13 +2960,14 @@ impl Read for Take { } self.limit -= filled as u64; + + result } else { let written = buf.written(); - self.inner.read_buf(buf.reborrow())?; + let result = self.inner.read_buf(buf.reborrow()); self.limit -= (buf.written() - written) as u64; + result } - - Ok(()) } } diff --git a/library/std/src/io/stdio.rs b/library/std/src/io/stdio.rs index 6de069a518e3d..35b38ed783ff2 100644 --- a/library/std/src/io/stdio.rs +++ b/library/std/src/io/stdio.rs @@ -370,7 +370,12 @@ impl Stdin { /// Locks this handle and reads a line of input, appending it to the specified buffer. /// /// For detailed semantics of this method, see the documentation on - /// [`BufRead::read_line`]. + /// [`BufRead::read_line`]. In particular: + /// * Previous content of the buffer will be preserved. To avoid appending + /// to the buffer, you need to [`clear`] it first. + /// * The trailing newline character, if any, is included in the buffer. + /// + /// [`clear`]: String::clear /// /// # Examples /// @@ -394,6 +399,7 @@ impl Stdin { /// in which case it will wait for the Enter key to be pressed before /// continuing #[stable(feature = "rust1", since = "1.0.0")] + #[rustc_confusables("get_line")] pub fn read_line(&self, buf: &mut String) -> io::Result { self.lock().read_line(buf) } diff --git a/library/std/src/io/stdio/tests.rs b/library/std/src/io/stdio/tests.rs index f89fd27ce6c23..bf8f3a5adfb6f 100644 --- a/library/std/src/io/stdio/tests.rs +++ b/library/std/src/io/stdio/tests.rs @@ -25,7 +25,7 @@ fn stderrlock_unwind_safe() { fn assert_unwind_safe() {} #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn panic_doesnt_poison() { thread::spawn(|| { let _a = stdin(); @@ -48,17 +48,17 @@ fn panic_doesnt_poison() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn test_lock_stderr() { test_lock(stderr, || stderr().lock()); } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn test_lock_stdin() { test_lock(stdin, || stdin().lock()); } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn test_lock_stdout() { test_lock(stdout, || stdout().lock()); } @@ -159,8 +159,7 @@ where assert_eq!(rx2.recv().unwrap(), Release2); // release th2 th2.join().unwrap(); th1.join().unwrap(); - assert_eq!( - *log.lock().unwrap(), - [Start1, Acquire1, Start2, Release1, Acquire2, Release2, Acquire1, Release1] - ); + assert_eq!(*log.lock().unwrap(), [ + Start1, Acquire1, Start2, Release1, Acquire2, Release2, Acquire1, Release1 + ]); } diff --git a/library/std/src/io/tests.rs b/library/std/src/io/tests.rs index 24e5a1dfd5c00..56b71c47dc73c 100644 --- a/library/std/src/io/tests.rs +++ b/library/std/src/io/tests.rs @@ -1,7 +1,7 @@ -use super::{repeat, BorrowedBuf, Cursor, SeekFrom}; +use super::{BorrowedBuf, Cursor, SeekFrom, repeat}; use crate::cmp::{self, min}; use crate::io::{ - self, BufRead, BufReader, IoSlice, IoSliceMut, Read, Seek, Write, DEFAULT_BUF_SIZE, + self, BufRead, BufReader, DEFAULT_BUF_SIZE, IoSlice, IoSliceMut, Read, Seek, Write, }; use crate::mem::MaybeUninit; use crate::ops::Deref; @@ -735,6 +735,69 @@ fn read_buf_full_read() { assert_eq!(BufReader::new(FullRead).fill_buf().unwrap().len(), DEFAULT_BUF_SIZE); } +struct DataAndErrorReader(&'static [u8]); + +impl Read for DataAndErrorReader { + fn read(&mut self, _buf: &mut [u8]) -> io::Result { + panic!("We want tests to use `read_buf`") + } + + fn read_buf(&mut self, buf: io::BorrowedCursor<'_>) -> io::Result<()> { + self.0.read_buf(buf).unwrap(); + Err(io::Error::other("error")) + } +} + +#[test] +fn read_buf_data_and_error_take() { + let mut buf = [0; 64]; + let mut buf = io::BorrowedBuf::from(buf.as_mut_slice()); + + let mut r = DataAndErrorReader(&[4, 5, 6]).take(1); + assert!(r.read_buf(buf.unfilled()).is_err()); + assert_eq!(buf.filled(), &[4]); + + assert!(r.read_buf(buf.unfilled()).is_ok()); + assert_eq!(buf.filled(), &[4]); + assert_eq!(r.get_ref().0, &[5, 6]); +} + +#[test] +fn read_buf_data_and_error_buf() { + let mut r = BufReader::new(DataAndErrorReader(&[4, 5, 6])); + + assert!(r.fill_buf().is_err()); + assert_eq!(r.fill_buf().unwrap(), &[4, 5, 6]); +} + +#[test] +fn read_buf_data_and_error_read_to_end() { + let mut r = DataAndErrorReader(&[4, 5, 6]); + + let mut v = Vec::with_capacity(200); + assert!(r.read_to_end(&mut v).is_err()); + + assert_eq!(v, &[4, 5, 6]); +} + +#[test] +fn read_to_end_error() { + struct ErrorReader; + + impl Read for ErrorReader { + fn read(&mut self, _buf: &mut [u8]) -> io::Result { + Err(io::Error::other("error")) + } + } + + let mut r = [4, 5, 6].chain(ErrorReader); + + let mut v = Vec::with_capacity(200); + assert!(r.read_to_end(&mut v).is_err()); + + assert_eq!(v, &[4, 5, 6]); +} + #[test] // Miri does not support signalling OOM #[cfg_attr(miri, ignore)] diff --git a/library/std/src/io/util/tests.rs b/library/std/src/io/util/tests.rs index 1dff3f3832bd7..0599a881af179 100644 --- a/library/std/src/io/util/tests.rs +++ b/library/std/src/io/util/tests.rs @@ -1,5 +1,5 @@ use crate::io::prelude::*; -use crate::io::{empty, repeat, sink, BorrowedBuf, Empty, Repeat, SeekFrom, Sink}; +use crate::io::{BorrowedBuf, Empty, Repeat, SeekFrom, Sink, empty, repeat, sink}; use crate::mem::MaybeUninit; #[test] diff --git a/library/std/src/keyword_docs.rs b/library/std/src/keyword_docs.rs index 9f4d244b5479e..30d43c8bbfd8c 100644 --- a/library/std/src/keyword_docs.rs +++ b/library/std/src/keyword_docs.rs @@ -2146,10 +2146,13 @@ mod unsafe_keyword {} #[doc(keyword = "use")] // -/// Import or rename items from other crates or modules. +/// Import or rename items from other crates or modules, or specify precise capturing +/// with `use<..>`. /// -/// Usually a `use` keyword is used to shorten the path required to refer to a module item. -/// The keyword may appear in modules, blocks and even functions, usually at the top. +/// ## Importing items +/// +/// The `use` keyword is employed to shorten the path required to refer to a module item. +/// The keyword may appear in modules, blocks, and even functions, typically at the top. /// /// The most basic usage of the keyword is `use path::to::item;`, /// though a number of convenient shortcuts are supported: @@ -2190,19 +2193,48 @@ mod unsafe_keyword {} /// // Compiles. /// let _ = VariantA; /// -/// // Does not compile ! +/// // Does not compile! /// let n = new(); /// ``` /// -/// For more information on `use` and paths in general, see the [Reference]. +/// For more information on `use` and paths in general, see the [Reference][ref-use-decls]. /// /// The differences about paths and the `use` keyword between the 2015 and 2018 editions -/// can also be found in the [Reference]. +/// can also be found in the [Reference][ref-use-decls]. +/// +/// ## Precise capturing +/// +/// The `use<..>` syntax is used within certain `impl Trait` bounds to control which generic +/// parameters are captured. This is important for return-position `impl Trait` (RPIT) types, +/// as it affects borrow checking by controlling which generic parameters can be used in the +/// hidden type. +/// +/// For example, the following function demonstrates an error without precise capturing in +/// Rust 2021 and earlier editions: +/// +/// ```rust,compile_fail,edition2021 +/// fn f(x: &()) -> impl Sized { x } +/// ``` +/// +/// By using `use<'_>` for precise capturing, it can be resolved: +/// +/// ```rust +/// fn f(x: &()) -> impl Sized + use<'_> { x } +/// ``` +/// +/// This syntax specifies that the elided lifetime be captured and therefore available for +/// use in the hidden type. +/// +/// In Rust 2024, opaque types automatically capture all lifetime parameters in scope. +/// `use<..>` syntax serves as an important way of opting-out of that default. +/// +/// For more details about precise capturing, see the [Reference][ref-impl-trait]. /// /// [`crate`]: keyword.crate.html /// [`self`]: keyword.self.html /// [`super`]: keyword.super.html -/// [Reference]: ../reference/items/use-declarations.html +/// [ref-use-decls]: ../reference/items/use-declarations.html +/// [ref-impl-trait]: ../reference/types/impl-trait.html mod use_keyword {} #[doc(keyword = "where")] @@ -2349,12 +2381,13 @@ mod async_keyword {} /// [`async`]: ../std/keyword.async.html mod await_keyword {} +// FIXME(dyn_compat_renaming): Update URL and link text. #[doc(keyword = "dyn")] // /// `dyn` is a prefix of a [trait object]'s type. /// /// The `dyn` keyword is used to highlight that calls to methods on the associated `Trait` -/// are [dynamically dispatched]. To use the trait this way, it must be 'object safe'. +/// are [dynamically dispatched]. To use the trait this way, it must be 'dyn-compatible'[^1]. /// /// Unlike generic parameters or `impl Trait`, the compiler does not know the concrete type that /// is being passed. That is, the type has been [erased]. @@ -2382,6 +2415,7 @@ mod await_keyword {} /// [ref-trait-obj]: ../reference/types/trait-object.html /// [ref-obj-safety]: ../reference/items/traits.html#object-safety /// [erased]: https://en.wikipedia.org/wiki/Type_erasure +/// [^1]: Formerly known as 'object safe'. mod dyn_keyword {} #[doc(keyword = "union")] diff --git a/library/std/src/lib.rs b/library/std/src/lib.rs index 3121ee8be8722..337540ff27d07 100644 --- a/library/std/src/lib.rs +++ b/library/std/src/lib.rs @@ -32,13 +32,17 @@ //! //! Once you are familiar with the contents of the standard library you may //! begin to find the verbosity of the prose distracting. At this stage in your -//! development you may want to press the `[-]` button near the top of the -//! page to collapse it into a more skimmable view. -//! -//! While you are looking at that `[-]` button also notice the `source` -//! link. Rust's API documentation comes with the source code and you are -//! encouraged to read it. The standard library source is generally high -//! quality and a peek behind the curtains is often enlightening. +//! development you may want to press the +//! +//! Summary button near the +//! top of the page to collapse it into a more skimmable view. +//! +//! While you are looking at the top of the page, also notice the +//! source link. Rust's API documentation comes with the source +//! code and you are encouraged to read it. The standard library source is +//! generally high quality and a peek behind the curtains is +//! often enlightening. //! //! # What is in the standard library documentation? //! @@ -149,7 +153,7 @@ //! the [`io`], [`fs`], and [`net`] modules. //! //! The [`thread`] module contains Rust's threading abstractions. [`sync`] -//! contains further primitive shared memory types, including [`atomic`] and +//! contains further primitive shared memory types, including [`atomic`], [`mpmc`] and //! [`mpsc`], which contains the channel types for message passing. //! //! # Use before and after `main()` @@ -173,6 +177,7 @@ //! - after-main use of thread-locals, which also affects additional features: //! - [`thread::current()`] //! - [`thread::scope()`] +//! - [`sync::mpmc`] //! - [`sync::mpsc`] //! - before-main stdio file descriptors are not guaranteed to be open on unix platforms //! @@ -198,6 +203,7 @@ //! [`atomic`]: sync::atomic //! [`for`]: ../book/ch03-05-control-flow.html#looping-through-a-collection-with-for //! [`str`]: prim@str +//! [`mpmc`]: sync::mpmc //! [`mpsc`]: sync::mpsc //! [`std::cmp`]: cmp //! [`std::slice`]: mod@slice @@ -261,6 +267,7 @@ #![allow(unused_features)] // // Features: +#![cfg_attr(not(bootstrap), feature(autodiff))] #![cfg_attr(test, feature(internal_output_capture, print_internals, update_panic_count, rt))] #![cfg_attr( all(target_vendor = "fortanix", target_env = "sgx"), @@ -273,6 +280,8 @@ // // Language features: // tidy-alphabetical-start +#![cfg_attr(bootstrap, feature(strict_provenance))] +#![cfg_attr(not(bootstrap), feature(strict_provenance_lints))] #![feature(alloc_error_handler)] #![feature(allocator_internals)] #![feature(allow_internal_unsafe)] @@ -282,7 +291,7 @@ #![feature(cfg_target_thread_local)] #![feature(cfi_encoding)] #![feature(concat_idents)] -#![feature(const_mut_refs)] +#![feature(const_float_methods)] #![feature(decl_macro)] #![feature(deprecated_suggestion)] #![feature(doc_cfg)] @@ -319,6 +328,7 @@ // // Library features (core): // tidy-alphabetical-start +#![feature(array_chunks)] #![feature(c_str_module)] #![feature(char_internals)] #![feature(clone_to_uninit)] @@ -329,7 +339,6 @@ #![feature(error_iter)] #![feature(exact_size_is_empty)] #![feature(exclusive_wrapper)] -#![feature(exposed_provenance)] #![feature(extend_one)] #![feature(float_gamma)] #![feature(float_minimum_maximum)] @@ -338,6 +347,7 @@ #![feature(hasher_prefixfree_extras)] #![feature(hashmap_internals)] #![feature(ip)] +#![feature(lazy_get)] #![feature(maybe_uninit_slice)] #![feature(maybe_uninit_write_slice)] #![feature(panic_can_unwind)] @@ -348,13 +358,14 @@ #![feature(prelude_2024)] #![feature(ptr_as_uninit)] #![feature(ptr_mask)] +#![feature(random)] #![feature(slice_internals)] #![feature(slice_ptr_get)] #![feature(slice_range)] #![feature(std_internals)] #![feature(str_internals)] -#![feature(strict_provenance)] #![feature(strict_provenance_atomic_ptr)] +#![feature(sync_unsafe_cell)] #![feature(ub_checks)] // tidy-alphabetical-end // @@ -368,6 +379,7 @@ #![feature(slice_concat_trait)] #![feature(thin_box)] #![feature(try_reserve_kind)] +#![feature(try_with_capacity)] #![feature(vec_into_raw_parts)] // tidy-alphabetical-end // @@ -405,9 +417,6 @@ // tidy-alphabetical-start #![feature(const_collections_with_hasher)] #![feature(const_hash)] -#![feature(const_ip)] -#![feature(const_ipv4)] -#![feature(const_ipv6)] #![feature(thread_local_internals)] // tidy-alphabetical-end // @@ -473,7 +482,7 @@ pub mod prelude; #[stable(feature = "rust1", since = "1.0.0")] pub use core::any; -#[stable(feature = "core_array", since = "1.36.0")] +#[stable(feature = "core_array", since = "1.35.0")] pub use core::array; #[unstable(feature = "async_iterator", issue = "79024")] pub use core::async_iter; @@ -493,9 +502,9 @@ pub use core::default; pub use core::future; #[stable(feature = "core_hint", since = "1.27.0")] pub use core::hint; -#[stable(feature = "i128", since = "1.26.0")] +#[stable(feature = "rust1", since = "1.0.0")] #[allow(deprecated, deprecated_in_future)] -pub use core::i128; +pub use core::i8; #[stable(feature = "rust1", since = "1.0.0")] #[allow(deprecated, deprecated_in_future)] pub use core::i16; @@ -505,9 +514,9 @@ pub use core::i32; #[stable(feature = "rust1", since = "1.0.0")] #[allow(deprecated, deprecated_in_future)] pub use core::i64; -#[stable(feature = "rust1", since = "1.0.0")] +#[stable(feature = "i128", since = "1.26.0")] #[allow(deprecated, deprecated_in_future)] -pub use core::i8; +pub use core::i128; #[stable(feature = "rust1", since = "1.0.0")] pub use core::intrinsics; #[stable(feature = "rust1", since = "1.0.0")] @@ -529,9 +538,9 @@ pub use core::pin; pub use core::ptr; #[stable(feature = "rust1", since = "1.0.0")] pub use core::result; -#[stable(feature = "i128", since = "1.26.0")] +#[stable(feature = "rust1", since = "1.0.0")] #[allow(deprecated, deprecated_in_future)] -pub use core::u128; +pub use core::u8; #[stable(feature = "rust1", since = "1.0.0")] #[allow(deprecated, deprecated_in_future)] pub use core::u16; @@ -541,9 +550,9 @@ pub use core::u32; #[stable(feature = "rust1", since = "1.0.0")] #[allow(deprecated, deprecated_in_future)] pub use core::u64; -#[stable(feature = "rust1", since = "1.0.0")] +#[stable(feature = "i128", since = "1.26.0")] #[allow(deprecated, deprecated_in_future)] -pub use core::u8; +pub use core::u128; #[stable(feature = "rust1", since = "1.0.0")] #[allow(deprecated, deprecated_in_future)] pub use core::usize; @@ -595,6 +604,8 @@ pub mod path; #[unstable(feature = "anonymous_pipe", issue = "127154")] pub mod pipe; pub mod process; +#[unstable(feature = "random", issue = "130703")] +pub mod random; pub mod sync; pub mod time; @@ -616,7 +627,13 @@ pub mod simd { #[doc(inline)] pub use crate::std_float::StdFloat; } - +#[cfg(not(bootstrap))] +#[unstable(feature = "autodiff", issue = "124509")] +/// This module provides support for automatic differentiation. +pub mod autodiff { + /// This macro handles automatic differentiation. + pub use core::autodiff::autodiff; +} #[stable(feature = "futures_api", since = "1.36.0")] pub mod task { //! Types and Traits for working with asynchronous tasks. @@ -649,9 +666,9 @@ pub mod arch { #[stable(feature = "simd_x86", since = "1.27.0")] pub use std_detect::is_x86_feature_detected; #[unstable(feature = "stdarch_mips_feature_detection", issue = "111188")] - pub use std_detect::{is_mips64_feature_detected, is_mips_feature_detected}; + pub use std_detect::{is_mips_feature_detected, is_mips64_feature_detected}; #[unstable(feature = "stdarch_powerpc_feature_detection", issue = "111191")] - pub use std_detect::{is_powerpc64_feature_detected, is_powerpc_feature_detected}; + pub use std_detect::{is_powerpc_feature_detected, is_powerpc64_feature_detected}; } // This was stabilized in the crate root so we have to keep it there. diff --git a/library/std/src/net/ip_addr.rs b/library/std/src/net/ip_addr.rs index 8a9426b61f999..4d673a1d66db6 100644 --- a/library/std/src/net/ip_addr.rs +++ b/library/std/src/net/ip_addr.rs @@ -1,5 +1,5 @@ // Tests for this module -#[cfg(all(test, not(target_os = "emscripten")))] +#[cfg(all(test, not(any(target_os = "emscripten", all(target_os = "wasi", target_env = "p1")))))] mod tests; #[stable(feature = "ip_addr", since = "1.7.0")] diff --git a/library/std/src/net/ip_addr/tests.rs b/library/std/src/net/ip_addr/tests.rs index ab99c0c2fcc16..7bed6f8a0f523 100644 --- a/library/std/src/net/ip_addr/tests.rs +++ b/library/std/src/net/ip_addr/tests.rs @@ -1,5 +1,5 @@ -use crate::net::test::{sa4, tsa}; use crate::net::Ipv4Addr; +use crate::net::test::{sa4, tsa}; #[test] fn to_socket_addr_socketaddr() { diff --git a/library/std/src/net/socket_addr.rs b/library/std/src/net/socket_addr.rs index 84922aabdb569..ba9c948a2e96f 100644 --- a/library/std/src/net/socket_addr.rs +++ b/library/std/src/net/socket_addr.rs @@ -1,5 +1,5 @@ // Tests for this module -#[cfg(all(test, not(target_os = "emscripten")))] +#[cfg(all(test, not(any(target_os = "emscripten", all(target_os = "wasi", target_env = "p1")))))] mod tests; #[stable(feature = "rust1", since = "1.0.0")] diff --git a/library/std/src/net/tcp.rs b/library/std/src/net/tcp.rs index 22d2dfe65a249..67a0f7e439d55 100644 --- a/library/std/src/net/tcp.rs +++ b/library/std/src/net/tcp.rs @@ -1,6 +1,13 @@ #![deny(unsafe_op_in_unsafe_fn)] -#[cfg(all(test, not(any(target_os = "emscripten", target_os = "xous"))))] +#[cfg(all( + test, + not(any( + target_os = "emscripten", + all(target_os = "wasi", target_env = "p1"), + target_os = "xous" + )) +))] mod tests; use crate::fmt; @@ -8,7 +15,7 @@ use crate::io::prelude::*; use crate::io::{self, BorrowedCursor, IoSlice, IoSliceMut}; use crate::iter::FusedIterator; use crate::net::{Shutdown, SocketAddr, ToSocketAddrs}; -use crate::sys_common::{net as net_imp, AsInner, FromInner, IntoInner}; +use crate::sys_common::{AsInner, FromInner, IntoInner, net as net_imp}; use crate::time::Duration; /// A TCP stream between a local and a remote socket. @@ -561,7 +568,7 @@ impl TcpStream { /// Moves this TCP stream into or out of nonblocking mode. /// - /// This will result in `read`, `write`, `recv` and `send` operations + /// This will result in `read`, `write`, `recv` and `send` system operations /// becoming nonblocking, i.e., immediately returning from their calls. /// If the IO operation is successful, `Ok` is returned and no further /// action is required. If the IO operation could not be completed and needs diff --git a/library/std/src/net/tcp/tests.rs b/library/std/src/net/tcp/tests.rs index d26517d74e492..a7b5cdf4ec061 100644 --- a/library/std/src/net/tcp/tests.rs +++ b/library/std/src/net/tcp/tests.rs @@ -57,6 +57,7 @@ fn connect_timeout_error() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn listen_localhost() { let socket_addr = next_test_ip4(); let listener = t!(TcpListener::bind(&socket_addr)); @@ -73,6 +74,7 @@ fn listen_localhost() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn connect_loopback() { each_ip(&mut |addr| { let acceptor = t!(TcpListener::bind(&addr)); @@ -94,6 +96,7 @@ fn connect_loopback() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn smoke_test() { each_ip(&mut |addr| { let acceptor = t!(TcpListener::bind(&addr)); @@ -114,6 +117,7 @@ fn smoke_test() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn read_eof() { each_ip(&mut |addr| { let acceptor = t!(TcpListener::bind(&addr)); @@ -133,6 +137,7 @@ fn read_eof() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn write_close() { each_ip(&mut |addr| { let acceptor = t!(TcpListener::bind(&addr)); @@ -161,6 +166,7 @@ fn write_close() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn multiple_connect_serial() { each_ip(&mut |addr| { let max = 10; @@ -183,6 +189,7 @@ fn multiple_connect_serial() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn multiple_connect_interleaved_greedy_schedule() { const MAX: usize = 10; each_ip(&mut |addr| { @@ -220,6 +227,7 @@ fn multiple_connect_interleaved_greedy_schedule() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn multiple_connect_interleaved_lazy_schedule() { const MAX: usize = 10; each_ip(&mut |addr| { @@ -255,6 +263,7 @@ fn multiple_connect_interleaved_lazy_schedule() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn socket_and_peer_name() { each_ip(&mut |addr| { let listener = t!(TcpListener::bind(&addr)); @@ -270,6 +279,7 @@ fn socket_and_peer_name() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn partial_read() { each_ip(&mut |addr| { let (tx, rx) = channel(); @@ -291,6 +301,7 @@ fn partial_read() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn read_buf() { each_ip(&mut |addr| { let srv = t!(TcpListener::bind(&addr)); @@ -389,6 +400,7 @@ fn double_bind() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn tcp_clone_smoke() { each_ip(&mut |addr| { let acceptor = t!(TcpListener::bind(&addr)); @@ -420,6 +432,7 @@ fn tcp_clone_smoke() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn tcp_clone_two_read() { each_ip(&mut |addr| { let acceptor = t!(TcpListener::bind(&addr)); @@ -454,6 +467,7 @@ fn tcp_clone_two_read() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn tcp_clone_two_write() { each_ip(&mut |addr| { let acceptor = t!(TcpListener::bind(&addr)); @@ -483,6 +497,7 @@ fn tcp_clone_two_write() { #[test] // FIXME: https://github.com/fortanix/rust-sgx/issues/110 #[cfg_attr(target_env = "sgx", ignore)] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn shutdown_smoke() { each_ip(&mut |addr| { let a = t!(TcpListener::bind(&addr)); @@ -505,6 +520,7 @@ fn shutdown_smoke() { #[test] // FIXME: https://github.com/fortanix/rust-sgx/issues/110 #[cfg_attr(target_env = "sgx", ignore)] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn close_readwrite_smoke() { each_ip(&mut |addr| { let a = t!(TcpListener::bind(&addr)); @@ -547,6 +563,7 @@ fn close_readwrite_smoke() { #[cfg_attr(target_env = "sgx", ignore)] // On windows, shutdown will not wake up blocking I/O operations. #[cfg_attr(windows, ignore)] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn close_read_wakes_up() { each_ip(&mut |addr| { let listener = t!(TcpListener::bind(&addr)); @@ -574,6 +591,7 @@ fn close_read_wakes_up() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn clone_while_reading() { each_ip(&mut |addr| { let accept = t!(TcpListener::bind(&addr)); @@ -614,6 +632,7 @@ fn clone_while_reading() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn clone_accept_smoke() { each_ip(&mut |addr| { let a = t!(TcpListener::bind(&addr)); @@ -632,6 +651,7 @@ fn clone_accept_smoke() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn clone_accept_concurrent() { each_ip(&mut |addr| { let a = t!(TcpListener::bind(&addr)); @@ -670,10 +690,10 @@ fn debug() { addr.to_string() } + #[cfg(any(unix, target_os = "wasi"))] + use crate::os::fd::AsRawFd; #[cfg(target_env = "sgx")] use crate::os::fortanix_sgx::io::AsRawFd; - #[cfg(unix)] - use crate::os::unix::io::AsRawFd; #[cfg(not(windows))] fn render_inner(addr: &dyn AsRawFd) -> impl fmt::Debug { addr.as_raw_fd() @@ -714,6 +734,7 @@ fn debug() { ignore )] #[cfg_attr(target_env = "sgx", ignore)] // FIXME: https://github.com/fortanix/rust-sgx/issues/31 +#[cfg_attr(target_os = "wasi", ignore)] // timeout not supported #[test] fn timeouts() { let addr = next_test_ip4(); @@ -742,6 +763,7 @@ fn timeouts() { #[test] #[cfg_attr(target_env = "sgx", ignore)] // FIXME: https://github.com/fortanix/rust-sgx/issues/31 +#[cfg_attr(target_os = "wasi", ignore)] // timeout not supported fn test_read_timeout() { let addr = next_test_ip4(); let listener = t!(TcpListener::bind(&addr)); @@ -763,6 +785,7 @@ fn test_read_timeout() { #[test] #[cfg_attr(target_env = "sgx", ignore)] // FIXME: https://github.com/fortanix/rust-sgx/issues/31 +#[cfg_attr(target_os = "wasi", ignore)] // timeout not supported fn test_read_with_timeout() { let addr = next_test_ip4(); let listener = t!(TcpListener::bind(&addr)); @@ -810,6 +833,7 @@ fn test_timeout_zero_duration() { #[test] #[cfg_attr(target_env = "sgx", ignore)] +#[cfg_attr(target_os = "wasi", ignore)] // linger not supported fn linger() { let addr = next_test_ip4(); let _listener = t!(TcpListener::bind(&addr)); @@ -879,6 +903,7 @@ fn set_nonblocking() { #[test] #[cfg_attr(target_env = "sgx", ignore)] // FIXME: https://github.com/fortanix/rust-sgx/issues/31 +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn peek() { each_ip(&mut |addr| { let (txdone, rxdone) = channel(); diff --git a/library/std/src/net/udp.rs b/library/std/src/net/udp.rs index 32e9086003d6b..6df47d7b0e0cd 100644 --- a/library/std/src/net/udp.rs +++ b/library/std/src/net/udp.rs @@ -1,10 +1,18 @@ -#[cfg(all(test, not(any(target_os = "emscripten", target_env = "sgx", target_os = "xous"))))] +#[cfg(all( + test, + not(any( + target_os = "emscripten", + all(target_os = "wasi", target_env = "p1"), + target_env = "sgx", + target_os = "xous" + )) +))] mod tests; use crate::fmt; use crate::io::{self, ErrorKind}; use crate::net::{Ipv4Addr, Ipv6Addr, SocketAddr, ToSocketAddrs}; -use crate::sys_common::{net as net_imp, AsInner, FromInner, IntoInner}; +use crate::sys_common::{AsInner, FromInner, IntoInner, net as net_imp}; use crate::time::Duration; /// A UDP socket. @@ -579,8 +587,8 @@ impl UdpSocket { /// This function specifies a new multicast group for this socket to join. /// The address must be a valid multicast address, and `interface` is the /// address of the local interface with which the system should join the - /// multicast group. If it's equal to `INADDR_ANY` then an appropriate - /// interface is chosen by the system. + /// multicast group. If it's equal to [`UNSPECIFIED`](Ipv4Addr::UNSPECIFIED) + /// then an appropriate interface is chosen by the system. #[stable(feature = "net2_mutators", since = "1.9.0")] pub fn join_multicast_v4(&self, multiaddr: &Ipv4Addr, interface: &Ipv4Addr) -> io::Result<()> { self.0.join_multicast_v4(multiaddr, interface) @@ -764,7 +772,7 @@ impl UdpSocket { /// Moves this UDP socket into or out of nonblocking mode. /// - /// This will result in `recv`, `recv_from`, `send`, and `send_to` + /// This will result in `recv`, `recv_from`, `send`, and `send_to` system /// operations becoming nonblocking, i.e., immediately returning from their /// calls. If the IO operation is successful, `Ok` is returned and no /// further action is required. If the IO operation could not be completed diff --git a/library/std/src/net/udp/tests.rs b/library/std/src/net/udp/tests.rs index 0cf9936645290..1c8c58d187957 100644 --- a/library/std/src/net/udp/tests.rs +++ b/library/std/src/net/udp/tests.rs @@ -27,6 +27,7 @@ fn bind_error() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn socket_smoke_test_ip4() { each_ip(&mut |server_ip, client_ip| { let (tx1, rx1) = channel(); @@ -69,6 +70,7 @@ fn socket_peer() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn udp_clone_smoke() { each_ip(&mut |addr1, addr2| { let sock1 = t!(UdpSocket::bind(&addr1)); @@ -98,6 +100,7 @@ fn udp_clone_smoke() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn udp_clone_two_read() { each_ip(&mut |addr1, addr2| { let sock1 = t!(UdpSocket::bind(&addr1)); @@ -130,6 +133,7 @@ fn udp_clone_two_read() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // no threads fn udp_clone_two_write() { each_ip(&mut |addr1, addr2| { let sock1 = t!(UdpSocket::bind(&addr1)); @@ -183,6 +187,7 @@ fn debug() { any(target_os = "netbsd", target_os = "openbsd", target_os = "vxworks", target_os = "nto"), ignore )] +#[cfg_attr(target_os = "wasi", ignore)] // timeout not supported #[test] fn timeouts() { let addr = next_test_ip4(); @@ -208,6 +213,7 @@ fn timeouts() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // timeout not supported fn test_read_timeout() { let addr = next_test_ip4(); @@ -232,6 +238,7 @@ fn test_read_timeout() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // timeout not supported fn test_read_with_timeout() { let addr = next_test_ip4(); @@ -291,6 +298,7 @@ fn connect_send_recv() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // peek not supported fn connect_send_peek_recv() { each_ip(&mut |addr, _| { let socket = t!(UdpSocket::bind(&addr)); @@ -313,6 +321,7 @@ fn connect_send_peek_recv() { } #[test] +#[cfg_attr(target_os = "wasi", ignore)] // peek_from not supported fn peek_from() { each_ip(&mut |addr, _| { let socket = t!(UdpSocket::bind(&addr)); diff --git a/library/std/src/num.rs b/library/std/src/num.rs index c1e6e7e628c83..d2f679e7dde54 100644 --- a/library/std/src/num.rs +++ b/library/std/src/num.rs @@ -26,9 +26,9 @@ pub use core::num::ZeroablePrimitive; #[stable(feature = "rust1", since = "1.0.0")] pub use core::num::{FpCategory, ParseFloatError, ParseIntError, TryFromIntError}; #[stable(feature = "signed_nonzero", since = "1.34.0")] -pub use core::num::{NonZeroI128, NonZeroI16, NonZeroI32, NonZeroI64, NonZeroI8, NonZeroIsize}; +pub use core::num::{NonZeroI8, NonZeroI16, NonZeroI32, NonZeroI64, NonZeroI128, NonZeroIsize}; #[stable(feature = "nonzero", since = "1.28.0")] -pub use core::num::{NonZeroU128, NonZeroU16, NonZeroU32, NonZeroU64, NonZeroU8, NonZeroUsize}; +pub use core::num::{NonZeroU8, NonZeroU16, NonZeroU32, NonZeroU64, NonZeroU128, NonZeroUsize}; #[cfg(test)] use crate::fmt; diff --git a/library/std/src/os/fortanix_sgx/mod.rs b/library/std/src/os/fortanix_sgx/mod.rs index 64f4d97ca95e2..2b5ccbe98f1b3 100644 --- a/library/std/src/os/fortanix_sgx/mod.rs +++ b/library/std/src/os/fortanix_sgx/mod.rs @@ -22,12 +22,12 @@ pub mod usercalls { /// Lowest-level interfaces to usercalls and usercall ABI type definitions. pub mod raw { pub use crate::sys::abi::usercalls::raw::{ - accept_stream, alloc, async_queues, bind_stream, close, connect_stream, do_usercall, - exit, flush, free, insecure_time, launch_thread, read, read_alloc, send, wait, write, - ByteBuffer, Cancel, Error, Fd, FifoDescriptor, Register, RegisterArgument, Result, - Return, ReturnValue, Tcs, Usercall, Usercalls as UsercallNrs, EV_RETURNQ_NOT_EMPTY, - EV_UNPARK, EV_USERCALLQ_NOT_FULL, FD_STDERR, FD_STDIN, FD_STDOUT, RESULT_SUCCESS, - USERCALL_USER_DEFINED, WAIT_INDEFINITE, WAIT_NO, + ByteBuffer, Cancel, EV_RETURNQ_NOT_EMPTY, EV_UNPARK, EV_USERCALLQ_NOT_FULL, Error, + FD_STDERR, FD_STDIN, FD_STDOUT, Fd, FifoDescriptor, RESULT_SUCCESS, Register, + RegisterArgument, Result, Return, ReturnValue, Tcs, USERCALL_USER_DEFINED, Usercall, + Usercalls as UsercallNrs, WAIT_INDEFINITE, WAIT_NO, accept_stream, alloc, async_queues, + bind_stream, close, connect_stream, do_usercall, exit, flush, free, insecure_time, + launch_thread, read, read_alloc, send, wait, write, }; } } diff --git a/library/std/src/os/mod.rs b/library/std/src/os/mod.rs index a2496baa63fb1..6701173d1e005 100644 --- a/library/std/src/os/mod.rs +++ b/library/std/src/os/mod.rs @@ -139,6 +139,8 @@ pub mod macos; pub mod netbsd; #[cfg(target_os = "nto")] pub mod nto; +#[cfg(target_os = "nuttx")] +pub mod nuttx; #[cfg(target_os = "openbsd")] pub mod openbsd; #[cfg(target_os = "redox")] diff --git a/library/std/src/os/nuttx/fs.rs b/library/std/src/os/nuttx/fs.rs new file mode 100644 index 0000000000000..7d6d8d16eca79 --- /dev/null +++ b/library/std/src/os/nuttx/fs.rs @@ -0,0 +1,92 @@ +#![stable(feature = "metadata_ext", since = "1.1.0")] + +use crate::fs::Metadata; +use crate::sys_common::AsInner; + +#[stable(feature = "metadata_ext", since = "1.1.0")] +pub trait MetadataExt { + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_dev(&self) -> u64; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_ino(&self) -> u64; + #[stable(feature = "metadata_ext", since = "1.1.0")] + fn st_mode(&self) -> u32; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_nlink(&self) -> u64; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_uid(&self) -> u32; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_gid(&self) -> u32; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_rdev(&self) -> u64; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_size(&self) -> u64; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_atime(&self) -> i64; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_atime_nsec(&self) -> i64; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_mtime(&self) -> i64; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_mtime_nsec(&self) -> i64; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_ctime(&self) -> i64; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_ctime_nsec(&self) -> i64; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_blksize(&self) -> u64; + #[stable(feature = "metadata_ext2", since = "1.8.0")] + fn st_blocks(&self) -> u64; +} + +#[stable(feature = "metadata_ext", since = "1.1.0")] +impl MetadataExt for Metadata { + fn st_dev(&self) -> u64 { + self.as_inner().as_inner().st_dev as u64 + } + fn st_ino(&self) -> u64 { + self.as_inner().as_inner().st_ino as u64 + } + fn st_mode(&self) -> u32 { + self.as_inner().as_inner().st_mode as u32 + } + fn st_nlink(&self) -> u64 { + self.as_inner().as_inner().st_nlink as u64 + } + fn st_uid(&self) -> u32 { + self.as_inner().as_inner().st_uid as u32 + } + fn st_gid(&self) -> u32 { + self.as_inner().as_inner().st_gid as u32 + } + fn st_rdev(&self) -> u64 { + self.as_inner().as_inner().st_rdev as u64 + } + fn st_size(&self) -> u64 { + self.as_inner().as_inner().st_size as u64 + } + fn st_atime(&self) -> i64 { + self.as_inner().as_inner().st_atim.tv_sec as i64 + } + fn st_atime_nsec(&self) -> i64 { + self.as_inner().as_inner().st_atim.tv_nsec as i64 + } + fn st_mtime(&self) -> i64 { + self.as_inner().as_inner().st_mtim.tv_sec as i64 + } + fn st_mtime_nsec(&self) -> i64 { + self.as_inner().as_inner().st_mtim.tv_nsec as i64 + } + fn st_ctime(&self) -> i64 { + self.as_inner().as_inner().st_ctim.tv_sec as i64 + } + fn st_ctime_nsec(&self) -> i64 { + self.as_inner().as_inner().st_ctim.tv_nsec as i64 + } + fn st_blksize(&self) -> u64 { + self.as_inner().as_inner().st_blksize as u64 + } + fn st_blocks(&self) -> u64 { + self.as_inner().as_inner().st_blocks as u64 + } +} diff --git a/library/std/src/os/nuttx/mod.rs b/library/std/src/os/nuttx/mod.rs new file mode 100644 index 0000000000000..7275bfd1765d5 --- /dev/null +++ b/library/std/src/os/nuttx/mod.rs @@ -0,0 +1,4 @@ +#![stable(feature = "raw_ext", since = "1.1.0")] +#![forbid(unsafe_op_in_unsafe_fn)] +pub mod fs; +pub(crate) mod raw; diff --git a/library/std/src/os/nuttx/raw.rs b/library/std/src/os/nuttx/raw.rs new file mode 100644 index 0000000000000..113079cf4abdc --- /dev/null +++ b/library/std/src/os/nuttx/raw.rs @@ -0,0 +1,33 @@ +//! rtems raw type definitions + +#![stable(feature = "raw_ext", since = "1.1.0")] +#![deprecated( + since = "1.8.0", + note = "these type aliases are no longer supported by \ + the standard library, the `libc` crate on \ + crates.io should be used instead for the correct \ + definitions" +)] +#![allow(deprecated)] + +#[stable(feature = "pthread_t", since = "1.8.0")] +pub type pthread_t = libc::pthread_t; + +#[stable(feature = "raw_ext", since = "1.1.0")] +pub type blkcnt_t = libc::blkcnt_t; + +#[stable(feature = "raw_ext", since = "1.1.0")] +pub type blksize_t = libc::blksize_t; +#[stable(feature = "raw_ext", since = "1.1.0")] +pub type dev_t = libc::dev_t; +#[stable(feature = "raw_ext", since = "1.1.0")] +pub type ino_t = libc::ino_t; +#[stable(feature = "raw_ext", since = "1.1.0")] +pub type mode_t = libc::mode_t; +#[stable(feature = "raw_ext", since = "1.1.0")] +pub type nlink_t = libc::nlink_t; +#[stable(feature = "raw_ext", since = "1.1.0")] +pub type off_t = libc::off_t; + +#[stable(feature = "raw_ext", since = "1.1.0")] +pub type time_t = libc::time_t; diff --git a/library/std/src/os/unix/fs.rs b/library/std/src/os/unix/fs.rs index caf6980afd91b..ba6481f052cdf 100644 --- a/library/std/src/os/unix/fs.rs +++ b/library/std/src/os/unix/fs.rs @@ -153,7 +153,7 @@ pub trait FileExt { /// /// It is possible to inadvertently set this flag, like in the example below. /// Therefore, it is important to be vigilant while changing options to mitigate - /// unexpected behaviour. + /// unexpected behavior. /// /// ```no_run /// use std::fs::File; @@ -334,6 +334,7 @@ pub trait PermissionsExt { /// assert_eq!(permissions.mode(), 0o644); /// ``` #[stable(feature = "fs_ext", since = "1.1.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "permissions_from_mode")] fn from_mode(mode: u32) -> Self; } diff --git a/library/std/src/os/unix/mod.rs b/library/std/src/os/unix/mod.rs index 7d2f0bd4efea7..5c2ec8ef994d4 100644 --- a/library/std/src/os/unix/mod.rs +++ b/library/std/src/os/unix/mod.rs @@ -69,6 +69,8 @@ mod platform { pub use crate::os::netbsd::*; #[cfg(target_os = "nto")] pub use crate::os::nto::*; + #[cfg(target_os = "nuttx")] + pub use crate::os::nuttx::*; #[cfg(target_os = "openbsd")] pub use crate::os::openbsd::*; #[cfg(target_os = "redox")] diff --git a/library/std/src/os/unix/net/addr.rs b/library/std/src/os/unix/net/addr.rs index 79f2c365025b7..253e1503cf7af 100644 --- a/library/std/src/os/unix/net/addr.rs +++ b/library/std/src/os/unix/net/addr.rs @@ -15,15 +15,12 @@ mod libc { pub type socklen_t = u32; pub struct sockaddr; #[derive(Clone)] - pub struct sockaddr_un; + pub struct sockaddr_un { + pub sun_path: [u8; 1], + } } -fn sun_path_offset(addr: &libc::sockaddr_un) -> usize { - // Work with an actual instance of the type since using a null pointer is UB - let base = (addr as *const libc::sockaddr_un).addr(); - let path = core::ptr::addr_of!(addr.sun_path).addr(); - path - base -} +const SUN_PATH_OFFSET: usize = mem::offset_of!(libc::sockaddr_un, sun_path); pub(super) fn sockaddr_un(path: &Path) -> io::Result<(libc::sockaddr_un, libc::socklen_t)> { // SAFETY: All zeros is a valid representation for `sockaddr_un`. @@ -53,7 +50,7 @@ pub(super) fn sockaddr_un(path: &Path) -> io::Result<(libc::sockaddr_un, libc::s ptr::copy_nonoverlapping(bytes.as_ptr(), addr.sun_path.as_mut_ptr().cast(), bytes.len()) }; - let mut len = sun_path_offset(&addr) + bytes.len(); + let mut len = SUN_PATH_OFFSET + bytes.len(); match bytes.get(0) { Some(&0) | None => {} Some(_) => len += 1, @@ -98,7 +95,7 @@ impl SocketAddr { unsafe { let mut addr: libc::sockaddr_un = mem::zeroed(); let mut len = mem::size_of::() as libc::socklen_t; - cvt(f(core::ptr::addr_of_mut!(addr) as *mut _, &mut len))?; + cvt(f((&raw mut addr) as *mut _, &mut len))?; SocketAddr::from_parts(addr, len) } } @@ -114,13 +111,13 @@ impl SocketAddr { let sun_path: &[u8] = unsafe { mem::transmute::<&[libc::c_char], &[u8]>(&addr.sun_path) }; len = core::slice::memchr::memchr(0, sun_path) - .map_or(len, |new_len| (new_len + sun_path_offset(&addr)) as libc::socklen_t); + .map_or(len, |new_len| (new_len + SUN_PATH_OFFSET) as libc::socklen_t); } if len == 0 { // When there is a datagram from unnamed unix socket // linux returns zero bytes of address - len = sun_path_offset(&addr) as libc::socklen_t; // i.e., zero-length address + len = SUN_PATH_OFFSET as libc::socklen_t; // i.e., zero-length address } else if addr.sun_family != libc::AF_UNIX as libc::sa_family_t { return Err(io::const_io_error!( io::ErrorKind::InvalidInput, @@ -238,7 +235,7 @@ impl SocketAddr { } fn address(&self) -> AddressKind<'_> { - let len = self.len as usize - sun_path_offset(&self.addr); + let len = self.len as usize - SUN_PATH_OFFSET; let path = unsafe { mem::transmute::<&[libc::c_char], &[u8]>(&self.addr.sun_path) }; // macOS seems to return a len of 16 and a zeroed sun_path for unnamed addresses @@ -287,7 +284,7 @@ impl linux_ext::addr::SocketAddrExt for SocketAddr { addr.sun_path.as_mut_ptr().add(1) as *mut u8, name.len(), ); - let len = (sun_path_offset(&addr) + 1 + name.len()) as libc::socklen_t; + let len = (SUN_PATH_OFFSET + 1 + name.len()) as libc::socklen_t; SocketAddr::from_parts(addr, len) } } diff --git a/library/std/src/os/unix/net/ancillary.rs b/library/std/src/os/unix/net/ancillary.rs index 9b487a6298247..36967fc3f98b0 100644 --- a/library/std/src/os/unix/net/ancillary.rs +++ b/library/std/src/os/unix/net/ancillary.rs @@ -1,6 +1,6 @@ // FIXME: This is currently disabled on *BSD. -use super::{sockaddr_un, SocketAddr}; +use super::{SocketAddr, sockaddr_un}; use crate::io::{self, IoSlice, IoSliceMut}; use crate::marker::PhantomData; use crate::mem::zeroed; @@ -37,7 +37,7 @@ pub(super) fn recv_vectored_with_ancillary_from( unsafe { let mut msg_name: libc::sockaddr_un = zeroed(); let mut msg: libc::msghdr = zeroed(); - msg.msg_name = core::ptr::addr_of_mut!(msg_name) as *mut _; + msg.msg_name = (&raw mut msg_name) as *mut _; msg.msg_namelen = size_of::() as libc::socklen_t; msg.msg_iov = bufs.as_mut_ptr().cast(); msg.msg_iovlen = bufs.len() as _; @@ -70,7 +70,7 @@ pub(super) fn send_vectored_with_ancillary_to( if let Some(path) = path { sockaddr_un(path)? } else { (zeroed(), 0) }; let mut msg: libc::msghdr = zeroed(); - msg.msg_name = core::ptr::addr_of_mut!(msg_name) as *mut _; + msg.msg_name = (&raw mut msg_name) as *mut _; msg.msg_namelen = msg_namelen; msg.msg_iov = bufs.as_ptr() as *mut _; msg.msg_iovlen = bufs.len() as _; diff --git a/library/std/src/os/unix/net/datagram.rs b/library/std/src/os/unix/net/datagram.rs index a605c3d4a2602..82446ea107fe5 100644 --- a/library/std/src/os/unix/net/datagram.rs +++ b/library/std/src/os/unix/net/datagram.rs @@ -12,9 +12,9 @@ ))] use libc::MSG_NOSIGNAL; +use super::{SocketAddr, sockaddr_un}; #[cfg(any(doc, target_os = "android", target_os = "linux"))] -use super::{recv_vectored_with_ancillary_from, send_vectored_with_ancillary_to, SocketAncillary}; -use super::{sockaddr_un, SocketAddr}; +use super::{SocketAncillary, recv_vectored_with_ancillary_from, send_vectored_with_ancillary_to}; #[cfg(any(doc, target_os = "android", target_os = "linux"))] use crate::io::{IoSlice, IoSliceMut}; use crate::net::Shutdown; @@ -100,7 +100,7 @@ impl UnixDatagram { let socket = UnixDatagram::unbound()?; let (addr, len) = sockaddr_un(path.as_ref())?; - cvt(libc::bind(socket.as_raw_fd(), core::ptr::addr_of!(addr) as *const _, len as _))?; + cvt(libc::bind(socket.as_raw_fd(), (&raw const addr) as *const _, len as _))?; Ok(socket) } @@ -133,7 +133,7 @@ impl UnixDatagram { let socket = UnixDatagram::unbound()?; cvt(libc::bind( socket.as_raw_fd(), - core::ptr::addr_of!(socket_addr.addr) as *const _, + (&raw const socket_addr.addr) as *const _, socket_addr.len as _, ))?; Ok(socket) @@ -215,7 +215,7 @@ impl UnixDatagram { unsafe { let (addr, len) = sockaddr_un(path.as_ref())?; - cvt(libc::connect(self.as_raw_fd(), core::ptr::addr_of!(addr) as *const _, len))?; + cvt(libc::connect(self.as_raw_fd(), (&raw const addr) as *const _, len))?; } Ok(()) } @@ -247,7 +247,7 @@ impl UnixDatagram { unsafe { cvt(libc::connect( self.as_raw_fd(), - core::ptr::addr_of!(socket_addr.addr) as *const _, + (&raw const socket_addr.addr) as *const _, socket_addr.len, ))?; } @@ -514,7 +514,7 @@ impl UnixDatagram { buf.as_ptr() as *const _, buf.len(), MSG_NOSIGNAL, - core::ptr::addr_of!(addr) as *const _, + (&raw const addr) as *const _, len, ))?; Ok(count as usize) @@ -549,7 +549,7 @@ impl UnixDatagram { buf.as_ptr() as *const _, buf.len(), MSG_NOSIGNAL, - core::ptr::addr_of!(socket_addr.addr) as *const _, + (&raw const socket_addr.addr) as *const _, socket_addr.len, ))?; Ok(count as usize) diff --git a/library/std/src/os/unix/net/listener.rs b/library/std/src/os/unix/net/listener.rs index a55199c82fc10..be236317d047d 100644 --- a/library/std/src/os/unix/net/listener.rs +++ b/library/std/src/os/unix/net/listener.rs @@ -1,4 +1,4 @@ -use super::{sockaddr_un, SocketAddr, UnixStream}; +use super::{SocketAddr, UnixStream, sockaddr_un}; use crate::os::unix::io::{AsFd, AsRawFd, BorrowedFd, FromRawFd, IntoRawFd, OwnedFd, RawFd}; use crate::path::Path; use crate::sys::cvt; @@ -103,11 +103,7 @@ impl UnixListener { )))] const backlog: libc::c_int = libc::SOMAXCONN; - cvt(libc::bind( - inner.as_inner().as_raw_fd(), - core::ptr::addr_of!(addr) as *const _, - len as _, - ))?; + cvt(libc::bind(inner.as_inner().as_raw_fd(), (&raw const addr) as *const _, len as _))?; cvt(libc::listen(inner.as_inner().as_raw_fd(), backlog))?; Ok(UnixListener(inner)) @@ -147,7 +143,7 @@ impl UnixListener { const backlog: core::ffi::c_int = 128; cvt(libc::bind( inner.as_raw_fd(), - core::ptr::addr_of!(socket_addr.addr) as *const _, + (&raw const socket_addr.addr) as *const _, socket_addr.len as _, ))?; cvt(libc::listen(inner.as_raw_fd(), backlog))?; @@ -182,7 +178,7 @@ impl UnixListener { pub fn accept(&self) -> io::Result<(UnixStream, SocketAddr)> { let mut storage: libc::sockaddr_un = unsafe { mem::zeroed() }; let mut len = mem::size_of_val(&storage) as libc::socklen_t; - let sock = self.0.accept(core::ptr::addr_of_mut!(storage) as *mut _, &mut len)?; + let sock = self.0.accept((&raw mut storage) as *mut _, &mut len)?; let addr = SocketAddr::from_parts(storage, len)?; Ok((UnixStream(sock), addr)) } diff --git a/library/std/src/os/unix/net/stream.rs b/library/std/src/os/unix/net/stream.rs index 19fc7b3d8532f..cb210b41eae19 100644 --- a/library/std/src/os/unix/net/stream.rs +++ b/library/std/src/os/unix/net/stream.rs @@ -1,3 +1,6 @@ +use super::{SocketAddr, sockaddr_un}; +#[cfg(any(doc, target_os = "android", target_os = "linux"))] +use super::{SocketAncillary, recv_vectored_with_ancillary_from, send_vectored_with_ancillary_to}; #[cfg(any( target_os = "android", target_os = "linux", @@ -8,10 +11,7 @@ target_os = "nto", target_vendor = "apple", ))] -use super::{peer_cred, UCred}; -#[cfg(any(doc, target_os = "android", target_os = "linux"))] -use super::{recv_vectored_with_ancillary_from, send_vectored_with_ancillary_to, SocketAncillary}; -use super::{sockaddr_un, SocketAddr}; +use super::{UCred, peer_cred}; use crate::fmt; use crate::io::{self, IoSlice, IoSliceMut}; use crate::net::Shutdown; @@ -84,7 +84,7 @@ impl UnixStream { let inner = Socket::new_raw(libc::AF_UNIX, libc::SOCK_STREAM)?; let (addr, len) = sockaddr_un(path.as_ref())?; - cvt(libc::connect(inner.as_raw_fd(), core::ptr::addr_of!(addr) as *const _, len))?; + cvt(libc::connect(inner.as_raw_fd(), (&raw const addr) as *const _, len))?; Ok(UnixStream(inner)) } } @@ -118,7 +118,7 @@ impl UnixStream { let inner = Socket::new_raw(libc::AF_UNIX, libc::SOCK_STREAM)?; cvt(libc::connect( inner.as_raw_fd(), - core::ptr::addr_of!(socket_addr.addr) as *const _, + (&raw const socket_addr.addr) as *const _, socket_addr.len, ))?; Ok(UnixStream(inner)) diff --git a/library/std/src/os/unix/net/ucred.rs b/library/std/src/os/unix/net/ucred.rs index b96e373ad0ae3..e1014a4f296dd 100644 --- a/library/std/src/os/unix/net/ucred.rs +++ b/library/std/src/os/unix/net/ucred.rs @@ -38,7 +38,7 @@ pub(super) use self::impl_linux::peer_cred; #[cfg(any(target_os = "linux", target_os = "android"))] mod impl_linux { - use libc::{c_void, getsockopt, socklen_t, ucred, SOL_SOCKET, SO_PEERCRED}; + use libc::{SO_PEERCRED, SOL_SOCKET, c_void, getsockopt, socklen_t, ucred}; use super::UCred; use crate::os::unix::io::AsRawFd; @@ -60,7 +60,7 @@ mod impl_linux { socket.as_raw_fd(), SOL_SOCKET, SO_PEERCRED, - core::ptr::addr_of_mut!(ucred) as *mut c_void, + (&raw mut ucred) as *mut c_void, &mut ucred_size, ); @@ -98,7 +98,7 @@ mod impl_bsd { #[cfg(target_vendor = "apple")] mod impl_apple { - use libc::{c_void, getpeereid, getsockopt, pid_t, socklen_t, LOCAL_PEERPID, SOL_LOCAL}; + use libc::{LOCAL_PEERPID, SOL_LOCAL, c_void, getpeereid, getsockopt, pid_t, socklen_t}; use super::UCred; use crate::os::unix::io::AsRawFd; @@ -121,7 +121,7 @@ mod impl_apple { socket.as_raw_fd(), SOL_LOCAL, LOCAL_PEERPID, - core::ptr::addr_of_mut!(pid) as *mut c_void, + (&raw mut pid) as *mut c_void, &mut pid_size, ); diff --git a/library/std/src/os/unix/process.rs b/library/std/src/os/unix/process.rs index 9aadd9491169f..ef5adaf229088 100644 --- a/library/std/src/os/unix/process.rs +++ b/library/std/src/os/unix/process.rs @@ -154,6 +154,7 @@ pub trait CommandExt: Sealed { /// required to gracefully handle errors it is recommended to use the /// cross-platform `spawn` instead. #[stable(feature = "process_exec2", since = "1.9.0")] + #[must_use] fn exec(&mut self) -> io::Error; /// Set executable argument diff --git a/library/std/src/os/wasi/mod.rs b/library/std/src/os/wasi/mod.rs index 33b50c9e53b8f..2ee6aa4660094 100644 --- a/library/std/src/os/wasi/mod.rs +++ b/library/std/src/os/wasi/mod.rs @@ -36,6 +36,8 @@ pub mod ffi; pub mod fs; pub mod io; + +#[cfg(all(target_os = "wasi", target_env = "p1"))] pub mod net; /// A prelude for conveniently writing platform-specific code. diff --git a/library/std/src/os/windows/fs.rs b/library/std/src/os/windows/fs.rs index 3dcde43cfec78..ddb8dbd8feea2 100644 --- a/library/std/src/os/windows/fs.rs +++ b/library/std/src/os/windows/fs.rs @@ -298,7 +298,7 @@ impl OpenOptionsExt for OpenOptions { /// of the [`BY_HANDLE_FILE_INFORMATION`] structure. /// /// [`BY_HANDLE_FILE_INFORMATION`]: -/// https://docs.microsoft.com/en-us/windows/win32/api/fileapi/ns-fileapi-by_handle_file_information +/// https://docs.microsoft.com/windows/win32/api/fileapi/ns-fileapi-by_handle_file_information #[stable(feature = "metadata_ext", since = "1.1.0")] pub trait MetadataExt { /// Returns the value of the `dwFileAttributes` field of this metadata. @@ -322,7 +322,7 @@ pub trait MetadataExt { /// ``` /// /// [File Attribute Constants]: - /// https://docs.microsoft.com/en-us/windows/win32/fileio/file-attribute-constants + /// https://docs.microsoft.com/windows/win32/fileio/file-attribute-constants #[stable(feature = "metadata_ext", since = "1.1.0")] fn file_attributes(&self) -> u32; @@ -351,7 +351,7 @@ pub trait MetadataExt { /// } /// ``` /// - /// [`FILETIME`]: https://docs.microsoft.com/en-us/windows/win32/api/minwinbase/ns-minwinbase-filetime + /// [`FILETIME`]: https://docs.microsoft.com/windows/win32/api/minwinbase/ns-minwinbase-filetime #[stable(feature = "metadata_ext", since = "1.1.0")] fn creation_time(&self) -> u64; @@ -386,7 +386,7 @@ pub trait MetadataExt { /// } /// ``` /// - /// [`FILETIME`]: https://docs.microsoft.com/en-us/windows/win32/api/minwinbase/ns-minwinbase-filetime + /// [`FILETIME`]: https://docs.microsoft.com/windows/win32/api/minwinbase/ns-minwinbase-filetime #[stable(feature = "metadata_ext", since = "1.1.0")] fn last_access_time(&self) -> u64; @@ -419,11 +419,11 @@ pub trait MetadataExt { /// } /// ``` /// - /// [`FILETIME`]: https://docs.microsoft.com/en-us/windows/win32/api/minwinbase/ns-minwinbase-filetime + /// [`FILETIME`]: https://docs.microsoft.com/windows/win32/api/minwinbase/ns-minwinbase-filetime #[stable(feature = "metadata_ext", since = "1.1.0")] fn last_write_time(&self) -> u64; - /// Returns the value of the `nFileSize{High,Low}` fields of this + /// Returns the value of the `nFileSize` fields of this /// metadata. /// /// The returned value does not have meaning for directories. @@ -462,7 +462,7 @@ pub trait MetadataExt { #[unstable(feature = "windows_by_handle", issue = "63010")] fn number_of_links(&self) -> Option; - /// Returns the value of the `nFileIndex{Low,High}` fields of this + /// Returns the value of the `nFileIndex` fields of this /// metadata. /// /// This will return `None` if the `Metadata` instance was created from a @@ -471,10 +471,14 @@ pub trait MetadataExt { #[unstable(feature = "windows_by_handle", issue = "63010")] fn file_index(&self) -> Option; - /// Returns the change time, which is the last time file metadata was changed, such as - /// renames, attributes, etc + /// Returns the value of the `ChangeTime` fields of this metadata. /// - /// This will return `None` if the `Metadata` instance was not created using the `FILE_BASIC_INFO` type. + /// `ChangeTime` is the last time file metadata was changed, such as + /// renames, attributes, etc. + /// + /// This will return `None` if `Metadata` instance was created from a call to + /// `DirEntry::metadata` or if the `target_vendor` is outside the current platform + /// support for this api. #[unstable(feature = "windows_change_time", issue = "121478")] fn change_time(&self) -> Option; } diff --git a/library/std/src/os/xous/ffi.rs b/library/std/src/os/xous/ffi.rs index 1a4a940bf358a..1db314e9ddad7 100644 --- a/library/std/src/os/xous/ffi.rs +++ b/library/std/src/os/xous/ffi.rs @@ -615,7 +615,7 @@ pub(crate) fn thread_id() -> Result { /// An error is generated if the `knob` is not a valid limit, or if the call /// would not succeed. pub(crate) fn adjust_limit(knob: Limits, current: usize, new: usize) -> Result { - let mut a0 = Syscall::JoinThread as usize; + let mut a0 = Syscall::AdjustProcessLimit as usize; let mut a1 = knob as usize; let a2 = current; let a3 = new; diff --git a/library/std/src/os/xous/ffi/definitions.rs b/library/std/src/os/xous/ffi/definitions.rs index 345005bcc78d7..1b16849af03b0 100644 --- a/library/std/src/os/xous/ffi/definitions.rs +++ b/library/std/src/os/xous/ffi/definitions.rs @@ -126,42 +126,36 @@ impl From for Error { #[stable(feature = "rust1", since = "1.0.0")] impl core::fmt::Display for Error { fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { - write!( - f, - "{}", - match self { - Error::NoError => "no error occurred", - Error::BadAlignment => "memory was not properly aligned", - Error::BadAddress => "an invalid address was supplied", - Error::OutOfMemory => "the process or service has run out of memory", - Error::MemoryInUse => "the requested address is in use", - Error::InterruptNotFound => - "the requested interrupt does not exist on this platform", - Error::InterruptInUse => "the requested interrupt is currently in use", - Error::InvalidString => "the specified string was not formatted correctly", - Error::ServerExists => "a server with that address already exists", - Error::ServerNotFound => "the requetsed server could not be found", - Error::ProcessNotFound => "the target process does not exist", - Error::ProcessNotChild => - "the requested operation can only be done on child processes", - Error::ProcessTerminated => "the target process has crashed", - Error::Timeout => "the requested operation timed out", - Error::InternalError => "an internal error occurred", - Error::ServerQueueFull => "the server has too many pending messages", - Error::ThreadNotAvailable => "the specified thread does not exist", - Error::UnhandledSyscall => "the kernel did not recognize that syscall", - Error::InvalidSyscall => "the syscall had incorrect parameters", - Error::ShareViolation => "an attempt was made to share memory twice", - Error::InvalidThread => "tried to resume a thread that was not ready", - Error::InvalidPid => "kernel attempted to use a pid that was not valid", - Error::AccessDenied => "no permission to perform the requested operation", - Error::UseBeforeInit => "attempt to use a service before initialization finished", - Error::DoubleFree => "the requested resource was freed twice", - Error::DebugInProgress => "kernel attempted to activate a thread being debugged", - Error::InvalidLimit => "process attempted to adjust an invalid limit", - Error::UnknownError => "an unknown error occurred", - } - ) + write!(f, "{}", match self { + Error::NoError => "no error occurred", + Error::BadAlignment => "memory was not properly aligned", + Error::BadAddress => "an invalid address was supplied", + Error::OutOfMemory => "the process or service has run out of memory", + Error::MemoryInUse => "the requested address is in use", + Error::InterruptNotFound => "the requested interrupt does not exist on this platform", + Error::InterruptInUse => "the requested interrupt is currently in use", + Error::InvalidString => "the specified string was not formatted correctly", + Error::ServerExists => "a server with that address already exists", + Error::ServerNotFound => "the requetsed server could not be found", + Error::ProcessNotFound => "the target process does not exist", + Error::ProcessNotChild => "the requested operation can only be done on child processes", + Error::ProcessTerminated => "the target process has crashed", + Error::Timeout => "the requested operation timed out", + Error::InternalError => "an internal error occurred", + Error::ServerQueueFull => "the server has too many pending messages", + Error::ThreadNotAvailable => "the specified thread does not exist", + Error::UnhandledSyscall => "the kernel did not recognize that syscall", + Error::InvalidSyscall => "the syscall had incorrect parameters", + Error::ShareViolation => "an attempt was made to share memory twice", + Error::InvalidThread => "tried to resume a thread that was not ready", + Error::InvalidPid => "kernel attempted to use a pid that was not valid", + Error::AccessDenied => "no permission to perform the requested operation", + Error::UseBeforeInit => "attempt to use a service before initialization finished", + Error::DoubleFree => "the requested resource was freed twice", + Error::DebugInProgress => "kernel attempted to activate a thread being debugged", + Error::InvalidLimit => "process attempted to adjust an invalid limit", + Error::UnknownError => "an unknown error occurred", + }) } } diff --git a/library/std/src/os/xous/services.rs b/library/std/src/os/xous/services.rs index ddf0236f5ad74..93916750c0547 100644 --- a/library/std/src/os/xous/services.rs +++ b/library/std/src/os/xous/services.rs @@ -19,7 +19,7 @@ pub(crate) use ticktimer::*; mod ns { const NAME_MAX_LENGTH: usize = 64; - use crate::os::xous::ffi::{lend_mut, Connection}; + use crate::os::xous::ffi::{Connection, lend_mut}; // By making this repr(C), the layout of this struct becomes well-defined // and no longer shifts around. // By marking it as `align(4096)` we define that it will be page-aligned, diff --git a/library/std/src/os/xous/services/systime.rs b/library/std/src/os/xous/services/systime.rs index 079ede7aa86c7..de87694b4cdca 100644 --- a/library/std/src/os/xous/services/systime.rs +++ b/library/std/src/os/xous/services/systime.rs @@ -1,6 +1,6 @@ use core::sync::atomic::{AtomicU32, Ordering}; -use crate::os::xous::ffi::{connect, Connection}; +use crate::os::xous::ffi::{Connection, connect}; pub(crate) enum SystimeScalar { GetUtcTimeMs, diff --git a/library/std/src/panic.rs b/library/std/src/panic.rs index 6f0952c41ede5..d649357a56d71 100644 --- a/library/std/src/panic.rs +++ b/library/std/src/panic.rs @@ -231,11 +231,11 @@ pub macro panic_2015 { }), } +#[stable(feature = "panic_hooks", since = "1.10.0")] +pub use core::panic::Location; #[doc(hidden)] #[unstable(feature = "edition_panic", issue = "none", reason = "use panic!() instead")] pub use core::panic::panic_2021; -#[stable(feature = "panic_hooks", since = "1.10.0")] -pub use core::panic::Location; #[stable(feature = "catch_unwind", since = "1.9.0")] pub use core::panic::{AssertUnwindSafe, RefUnwindSafe, UnwindSafe}; @@ -283,47 +283,60 @@ where { } +#[unstable(feature = "abort_unwind", issue = "130338")] +pub use core::panic::abort_unwind; + /// Invokes a closure, capturing the cause of an unwinding panic if one occurs. /// -/// This function will return `Ok` with the closure's result if the closure -/// does not panic, and will return `Err(cause)` if the closure panics. The -/// `cause` returned is the object with which panic was originally invoked. +/// This function will return `Ok` with the closure's result if the closure does +/// not panic, and will return `Err(cause)` if the closure panics. The `cause` +/// returned is the object with which panic was originally invoked. /// -/// It is currently undefined behavior to unwind from Rust code into foreign -/// code, so this function is particularly useful when Rust is called from -/// another language (normally C). This can run arbitrary Rust code, capturing a -/// panic and allowing a graceful handling of the error. +/// Rust functions that are expected to be called from foreign code that does +/// not support unwinding (such as C compiled with `-fno-exceptions`) should be +/// defined using `extern "C"`, which ensures that if the Rust code panics, it +/// is automatically caught and the process is aborted. If this is the desired +/// behavior, it is not necessary to use `catch_unwind` explicitly. This +/// function should instead be used when more graceful error-handling is needed. /// /// It is **not** recommended to use this function for a general try/catch /// mechanism. The [`Result`] type is more appropriate to use for functions that /// can fail on a regular basis. Additionally, this function is not guaranteed /// to catch all panics, see the "Notes" section below. /// -/// The closure provided is required to adhere to the [`UnwindSafe`] trait to ensure -/// that all captured variables are safe to cross this boundary. The purpose of -/// this bound is to encode the concept of [exception safety][rfc] in the type -/// system. Most usage of this function should not need to worry about this -/// bound as programs are naturally unwind safe without `unsafe` code. If it -/// becomes a problem the [`AssertUnwindSafe`] wrapper struct can be used to quickly -/// assert that the usage here is indeed unwind safe. +/// The closure provided is required to adhere to the [`UnwindSafe`] trait to +/// ensure that all captured variables are safe to cross this boundary. The +/// purpose of this bound is to encode the concept of [exception safety][rfc] in +/// the type system. Most usage of this function should not need to worry about +/// this bound as programs are naturally unwind safe without `unsafe` code. If +/// it becomes a problem the [`AssertUnwindSafe`] wrapper struct can be used to +/// quickly assert that the usage here is indeed unwind safe. /// /// [rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1236-stabilize-catch-panic.md /// /// # Notes /// -/// Note that this function **might not catch all panics** in Rust. A panic in -/// Rust is not always implemented via unwinding, but can be implemented by -/// aborting the process as well. This function *only* catches unwinding panics, -/// not those that abort the process. +/// This function **might not catch all Rust panics**. A Rust panic is not +/// always implemented via unwinding, but can be implemented by aborting the +/// process as well. This function *only* catches unwinding panics, not those +/// that abort the process. +/// +/// If a custom panic hook has been set, it will be invoked before the panic is +/// caught, before unwinding. /// -/// Note that if a custom panic hook has been set, it will be invoked before -/// the panic is caught, before unwinding. +/// Although unwinding into Rust code with a foreign exception (e.g. an +/// exception thrown from C++ code, or a `panic!` in Rust code compiled or +/// linked with a different runtime) via an appropriate ABI (e.g. `"C-unwind"`) +/// is permitted, catching such an exception using this function will have one +/// of two behaviors, and it is unspecified which will occur: /// -/// Also note that unwinding into Rust code with a foreign exception (e.g. -/// an exception thrown from C++ code) is undefined behavior. +/// * The process aborts, after executing all destructors of `f` and the +/// functions it called. +/// * The function returns a `Result::Err` containing an opaque type. /// -/// Finally, be **careful in how you drop the result of this function**. -/// If it is `Err`, it contains the panic payload, and dropping that may in turn panic! +/// Finally, be **careful in how you drop the result of this function**. If it +/// is `Err`, it contains the panic payload, and dropping that may in turn +/// panic! /// /// # Examples /// diff --git a/library/std/src/panicking.rs b/library/std/src/panicking.rs index 1c972d3810036..ac1f547c9143f 100644 --- a/library/std/src/panicking.rs +++ b/library/std/src/panicking.rs @@ -506,7 +506,7 @@ pub unsafe fn r#try R>(f: F) -> Result> // method of calling a catch panic whilst juggling ownership. let mut data = Data { f: ManuallyDrop::new(f) }; - let data_ptr = core::ptr::addr_of_mut!(data) as *mut u8; + let data_ptr = (&raw mut data) as *mut u8; // SAFETY: // // Access to the union's fields: this is `std` and we know that the `r#try` diff --git a/library/std/src/path.rs b/library/std/src/path.rs index 506ad445b6bed..62125f885b2ff 100644 --- a/library/std/src/path.rs +++ b/library/std/src/path.rs @@ -75,14 +75,14 @@ use core::clone::CloneToUninit; use crate::borrow::{Borrow, Cow}; use crate::collections::TryReserveError; use crate::error::Error; -use crate::ffi::{os_str, OsStr, OsString}; +use crate::ffi::{OsStr, OsString, os_str}; use crate::hash::{Hash, Hasher}; use crate::iter::FusedIterator; use crate::ops::{self, Deref}; use crate::rc::Rc; use crate::str::FromStr; use crate::sync::Arc; -use crate::sys::path::{is_sep_byte, is_verbatim_sep, parse_prefix, MAIN_SEP_STR}; +use crate::sys::path::{MAIN_SEP_STR, is_sep_byte, is_verbatim_sep, parse_prefix}; use crate::{cmp, fmt, fs, io, sys}; //////////////////////////////////////////////////////////////////////////////// @@ -263,6 +263,7 @@ pub fn is_separator(c: char) -> bool { /// /// For example, `/` on Unix and `\` on Windows. #[stable(feature = "rust1", since = "1.0.0")] +#[cfg_attr(not(test), rustc_diagnostic_item = "path_main_separator")] pub const MAIN_SEPARATOR: char = crate::sys::path::MAIN_SEP; /// The primary separator of path components for the current platform. @@ -1153,6 +1154,21 @@ impl FusedIterator for Ancestors<'_> {} /// ``` /// /// Which method works best depends on what kind of situation you're in. +/// +/// Note that `PathBuf` does not always sanitize arguments, for example +/// [`push`] allows paths built from strings which include separators: +/// +/// use std::path::PathBuf; +/// +/// let mut path = PathBuf::new(); +/// +/// path.push(r"C:\"); +/// path.push("windows"); +/// path.push(r"..\otherdir"); +/// path.push("system32"); +/// +/// The behavior of `PathBuf` may be changed to a panic on such inputs +/// in the future. [`Extend::extend`] should be used to add multi-part paths. #[cfg_attr(not(test), rustc_diagnostic_item = "PathBuf")] #[stable(feature = "rust1", since = "1.0.0")] pub struct PathBuf { @@ -1211,6 +1227,7 @@ impl PathBuf { /// let p = PathBuf::from("/test"); /// assert_eq!(Path::new("/test"), p.as_path()); /// ``` + #[cfg_attr(not(test), rustc_diagnostic_item = "pathbuf_as_path")] #[stable(feature = "rust1", since = "1.0.0")] #[must_use] #[inline] @@ -1391,6 +1408,9 @@ impl PathBuf { /// `file_name`. The new path will be a sibling of the original path. /// (That is, it will have the same parent.) /// + /// The argument is not sanitized, so can include separators. This + /// behavior may be changed to a panic in the future. + /// /// [`self.file_name`]: Path::file_name /// [`pop`]: PathBuf::pop /// @@ -1411,6 +1431,12 @@ impl PathBuf { /// /// buf.set_file_name("baz"); /// assert!(buf == PathBuf::from("/baz")); + /// + /// buf.set_file_name("../b/c.txt"); + /// assert!(buf == PathBuf::from("/../b/c.txt")); + /// + /// buf.set_file_name("baz"); + /// assert!(buf == PathBuf::from("/../b/baz")); /// ``` #[stable(feature = "rust1", since = "1.0.0")] pub fn set_file_name>(&mut self, file_name: S) { @@ -2240,6 +2266,7 @@ impl Path { #[must_use = "this returns the result of the operation, \ without modifying the original"] #[stable(feature = "rust1", since = "1.0.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "path_to_pathbuf")] pub fn to_path_buf(&self) -> PathBuf { PathBuf::from(self.inner.to_os_string()) } @@ -3117,7 +3144,7 @@ unsafe impl CloneToUninit for Path { #[cfg_attr(debug_assertions, track_caller)] unsafe fn clone_to_uninit(&self, dst: *mut Self) { // SAFETY: Path is just a wrapper around OsStr - unsafe { self.inner.clone_to_uninit(core::ptr::addr_of_mut!((*dst).inner)) } + unsafe { self.inner.clone_to_uninit(&raw mut (*dst).inner) } } } diff --git a/library/std/src/path/tests.rs b/library/std/src/path/tests.rs index 6436872087d6c..b75793d2bc990 100644 --- a/library/std/src/path/tests.rs +++ b/library/std/src/path/tests.rs @@ -139,7 +139,7 @@ fn test_pathbuf_leak() { } #[test] -#[cfg(unix)] +#[cfg(any(unix, target_os = "wasi"))] pub fn test_decompositions_unix() { t!("", iter: [], @@ -1201,7 +1201,10 @@ pub fn test_push() { }); ); - if cfg!(unix) || cfg!(all(target_env = "sgx", target_vendor = "fortanix")) { + if cfg!(unix) + || cfg!(target_os = "wasi") + || cfg!(all(target_env = "sgx", target_vendor = "fortanix")) + { tp!("", "foo", "foo"); tp!("foo", "bar", "foo/bar"); tp!("foo/", "bar", "foo/bar"); @@ -1358,7 +1361,10 @@ pub fn test_set_file_name() { tfn!("foo", "bar", "bar"); tfn!("foo", "", ""); tfn!("", "foo", "foo"); - if cfg!(unix) || cfg!(all(target_env = "sgx", target_vendor = "fortanix")) { + if cfg!(unix) + || cfg!(target_os = "wasi") + || cfg!(all(target_env = "sgx", target_vendor = "fortanix")) + { tfn!(".", "foo", "./foo"); tfn!("foo/", "bar", "bar"); tfn!("foo/.", "bar", "bar"); @@ -1758,7 +1764,7 @@ fn test_components_debug() { assert_eq!(expected, actual); } -#[cfg(unix)] +#[cfg(any(unix, target_os = "wasi"))] #[test] fn test_iter_debug() { let path = Path::new("/tmp"); @@ -1859,7 +1865,7 @@ fn test_ord() { } #[test] -#[cfg(unix)] +#[cfg(any(unix, target_os = "wasi"))] fn test_unix_absolute() { use crate::path::absolute; diff --git a/library/std/src/pipe.rs b/library/std/src/pipe.rs index aa4c7014fe918..891032e94a669 100644 --- a/library/std/src/pipe.rs +++ b/library/std/src/pipe.rs @@ -12,7 +12,7 @@ //! ``` use crate::io; -use crate::sys::anonymous_pipe::{pipe as pipe_inner, AnonPipe}; +use crate::sys::anonymous_pipe::{AnonPipe, pipe as pipe_inner}; /// Create anonymous pipe that is close-on-exec and blocking. #[unstable(feature = "anonymous_pipe", issue = "127154")] diff --git a/library/std/src/process.rs b/library/std/src/process.rs index a155855029e70..6933528cdbd0a 100644 --- a/library/std/src/process.rs +++ b/library/std/src/process.rs @@ -119,7 +119,7 @@ //! when given a `.bat` file as the application to run, it will automatically //! convert that into running `cmd.exe /c` with the batch file as the next argument. //! -//! For historical reasons Rust currently preserves this behaviour when using +//! For historical reasons Rust currently preserves this behavior when using //! [`Command::new`], and escapes the arguments according to `cmd.exe` rules. //! Due to the complexity of `cmd.exe` argument handling, it might not be //! possible to safely escape some special characters, and using them will result @@ -148,7 +148,15 @@ #![stable(feature = "process", since = "1.0.0")] #![deny(unsafe_op_in_unsafe_fn)] -#[cfg(all(test, not(any(target_os = "emscripten", target_env = "sgx", target_os = "xous"))))] +#[cfg(all( + test, + not(any( + target_os = "emscripten", + target_os = "wasi", + target_env = "sgx", + target_os = "xous" + )) +))] mod tests; use crate::convert::Infallible; @@ -157,7 +165,7 @@ use crate::io::prelude::*; use crate::io::{self, BorrowedCursor, IoSlice, IoSliceMut}; use crate::num::NonZero; use crate::path::Path; -use crate::sys::pipe::{read2, AnonPipe}; +use crate::sys::pipe::{AnonPipe, read2}; use crate::sys::process as imp; #[stable(feature = "command_access", since = "1.57.0")] pub use crate::sys_common::process::CommandEnvs; @@ -1910,10 +1918,14 @@ impl crate::error::Error for ExitStatusError {} /// to its parent under normal termination. /// /// `ExitCode` is intended to be consumed only by the standard library (via -/// [`Termination::report()`]), and intentionally does not provide accessors like -/// `PartialEq`, `Eq`, or `Hash`. Instead the standard library provides the -/// canonical `SUCCESS` and `FAILURE` exit codes as well as `From for -/// ExitCode` for constructing other arbitrary exit codes. +/// [`Termination::report()`]). For forwards compatibility with potentially +/// unusual targets, this type currently does not provide `Eq`, `Hash`, or +/// access to the raw value. This type does provide `PartialEq` for +/// comparison, but note that there may potentially be multiple failure +/// codes, some of which will _not_ compare equal to `ExitCode::FAILURE`. +/// The standard library provides the canonical `SUCCESS` and `FAILURE` +/// exit codes as well as `From for ExitCode` for constructing other +/// arbitrary exit codes. /// /// # Portability /// @@ -1952,7 +1964,7 @@ impl crate::error::Error for ExitStatusError {} /// ExitCode::SUCCESS /// } /// ``` -#[derive(Clone, Copy, Debug)] +#[derive(Clone, Copy, Debug, PartialEq)] #[stable(feature = "process_exitcode", since = "1.61.0")] pub struct ExitCode(imp::ExitCode); @@ -2306,7 +2318,7 @@ pub fn exit(code: i32) -> ! { /// Rust IO buffers (eg, from `BufWriter`) will not be flushed. /// Likewise, C stdio buffers will (on most platforms) not be flushed. /// -/// This is in contrast to the default behaviour of [`panic!`] which unwinds +/// This is in contrast to the default behavior of [`panic!`] which unwinds /// the current thread's stack and calls all destructors. /// When `panic="abort"` is set, either as an argument to `rustc` or in a /// crate's Cargo.toml, [`panic!`] and `abort` are similar. However, diff --git a/library/std/src/process/tests.rs b/library/std/src/process/tests.rs index f8e8e0dea553b..fb0b495961c36 100644 --- a/library/std/src/process/tests.rs +++ b/library/std/src/process/tests.rs @@ -96,9 +96,23 @@ fn stdout_works() { #[test] #[cfg_attr(any(windows, target_os = "vxworks"), ignore)] fn set_current_dir_works() { + // On many Unix platforms this will use the posix_spawn path. let mut cmd = shell_cmd(); cmd.arg("-c").arg("pwd").current_dir("/").stdout(Stdio::piped()); assert_eq!(run_output(cmd), "/\n"); + + // Also test the fork/exec path by setting a pre_exec function. + #[cfg(unix)] + { + use crate::os::unix::process::CommandExt; + + let mut cmd = shell_cmd(); + cmd.arg("-c").arg("pwd").current_dir("/").stdout(Stdio::piped()); + unsafe { + cmd.pre_exec(|| Ok(())); + } + assert_eq!(run_output(cmd), "/\n"); + } } #[test] @@ -437,7 +451,7 @@ fn test_proc_thread_attributes() { use crate::mem; use crate::os::windows::io::AsRawHandle; use crate::os::windows::process::CommandExt; - use crate::sys::c::{CloseHandle, BOOL, HANDLE}; + use crate::sys::c::{BOOL, CloseHandle, HANDLE}; use crate::sys::cvt; #[repr(C)] diff --git a/library/std/src/random.rs b/library/std/src/random.rs new file mode 100644 index 0000000000000..45f51dd37b041 --- /dev/null +++ b/library/std/src/random.rs @@ -0,0 +1,106 @@ +//! Random value generation. +//! +//! The [`Random`] trait allows generating a random value for a type using a +//! given [`RandomSource`]. + +#[unstable(feature = "random", issue = "130703")] +pub use core::random::*; + +use crate::sys::random as sys; + +/// The default random source. +/// +/// This asks the system for random data suitable for cryptographic purposes +/// such as key generation. If security is a concern, consult the platform +/// documentation below for the specific guarantees your target provides. +/// +/// The high quality of randomness provided by this source means it can be quite +/// slow on some targets. If you need a large quantity of random numbers and +/// security is not a concern, consider using an alternative random number +/// generator (potentially seeded from this one). +/// +/// # Underlying sources +/// +/// Platform | Source +/// -----------------------|--------------------------------------------------------------- +/// Linux | [`getrandom`] or [`/dev/urandom`] after polling `/dev/random` +/// Windows | [`ProcessPrng`](https://learn.microsoft.com/en-us/windows/win32/seccng/processprng) +/// Apple | `CCRandomGenerateBytes` +/// DragonFly | [`arc4random_buf`](https://man.dragonflybsd.org/?command=arc4random) +/// ESP-IDF | [`esp_fill_random`](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/random.html#_CPPv415esp_fill_randomPv6size_t) +/// FreeBSD | [`arc4random_buf`](https://man.freebsd.org/cgi/man.cgi?query=arc4random) +/// Fuchsia | [`cprng_draw`](https://fuchsia.dev/reference/syscalls/cprng_draw) +/// Haiku | `arc4random_buf` +/// Illumos | [`arc4random_buf`](https://www.illumos.org/man/3C/arc4random) +/// NetBSD | [`arc4random_buf`](https://man.netbsd.org/arc4random.3) +/// OpenBSD | [`arc4random_buf`](https://man.openbsd.org/arc4random.3) +/// Solaris | [`arc4random_buf`](https://docs.oracle.com/cd/E88353_01/html/E37843/arc4random-3c.html) +/// Vita | `arc4random_buf` +/// Hermit | `read_entropy` +/// Horizon | `getrandom` shim +/// AIX, Hurd, L4Re, QNX | `/dev/urandom` +/// Redox | `/scheme/rand` +/// RTEMS | [`arc4random_buf`](https://docs.rtems.org/branches/master/bsp-howto/getentropy.html) +/// SGX | [`rdrand`](https://en.wikipedia.org/wiki/RDRAND) +/// SOLID | `SOLID_RNG_SampleRandomBytes` +/// TEEOS | `TEE_GenerateRandom` +/// UEFI | [`EFI_RNG_PROTOCOL`](https://uefi.org/specs/UEFI/2.10/37_Secure_Technologies.html#random-number-generator-protocol) +/// VxWorks | `randABytes` after waiting for `randSecure` to become ready +/// WASI | [`random_get`](https://github.com/WebAssembly/WASI/blob/main/legacy/preview1/docs.md#-random_getbuf-pointeru8-buf_len-size---result-errno) +/// ZKVM | `sys_rand` +/// +/// Note that the sources used might change over time. +/// +/// Consult the documentation for the underlying operations on your supported +/// targets to determine whether they provide any particular desired properties, +/// such as support for reseeding on VM fork operations. +/// +/// [`getrandom`]: https://www.man7.org/linux/man-pages/man2/getrandom.2.html +/// [`/dev/urandom`]: https://www.man7.org/linux/man-pages/man4/random.4.html +#[derive(Default, Debug, Clone, Copy)] +#[unstable(feature = "random", issue = "130703")] +pub struct DefaultRandomSource; + +#[unstable(feature = "random", issue = "130703")] +impl RandomSource for DefaultRandomSource { + fn fill_bytes(&mut self, bytes: &mut [u8]) { + sys::fill_bytes(bytes) + } +} + +/// Generates a random value with the default random source. +/// +/// This is a convenience function for `T::random(&mut DefaultRandomSource)` and +/// will sample according to the same distribution as the underlying [`Random`] +/// trait implementation. See [`DefaultRandomSource`] for more information about +/// how randomness is sourced. +/// +/// **Warning:** Be careful when manipulating random values! The +/// [`random`](Random::random) method on integers samples them with a uniform +/// distribution, so a value of 1 is just as likely as [`i32::MAX`]. By using +/// modulo operations, some of the resulting values can become more likely than +/// others. Use audited crates when in doubt. +/// +/// # Examples +/// +/// Generating a [version 4/variant 1 UUID] represented as text: +/// ``` +/// #![feature(random)] +/// +/// use std::random::random; +/// +/// let bits: u128 = random(); +/// let g1 = (bits >> 96) as u32; +/// let g2 = (bits >> 80) as u16; +/// let g3 = (0x4000 | (bits >> 64) & 0x0fff) as u16; +/// let g4 = (0x8000 | (bits >> 48) & 0x3fff) as u16; +/// let g5 = (bits & 0xffffffffffff) as u64; +/// let uuid = format!("{g1:08x}-{g2:04x}-{g3:04x}-{g4:04x}-{g5:012x}"); +/// println!("{uuid}"); +/// ``` +/// +/// [version 4/variant 1 UUID]: https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random) +#[unstable(feature = "random", issue = "130703")] +pub fn random() -> T { + T::random(&mut DefaultRandomSource) +} diff --git a/library/std/src/rt.rs b/library/std/src/rt.rs index b6f36931ec28a..b2492238bd37b 100644 --- a/library/std/src/rt.rs +++ b/library/std/src/rt.rs @@ -21,9 +21,10 @@ pub use crate::panicking::{begin_panic, panic_count}; pub use core::panicking::{panic_display, panic_fmt}; #[rustfmt::skip] +use crate::any::Any; use crate::sync::Once; -use crate::sys; use crate::thread::{self, Thread}; +use crate::{mem, panic, sys}; // Prints to the "panic output", depending on the platform this may be: // - the standard error output @@ -66,6 +67,11 @@ macro_rules! rtunwrap { }; } +fn handle_rt_panic(e: Box) { + mem::forget(e); + rtabort!("initialization or cleanup bug"); +} + // One-time runtime initialization. // Runs before `main`. // SAFETY: must be called only once during runtime initialization. @@ -96,9 +102,38 @@ unsafe fn init(argc: isize, argv: *const *const u8, sigpipe: u8) { sys::init(argc, argv, sigpipe) }; - // Set up the current thread to give it the right name. - let thread = Thread::new_main(); - thread::set_current(thread); + // Set up the current thread handle to give it the right name. + // + // When code running before main uses `ReentrantLock` (for example by + // using `println!`), the thread ID can become initialized before we + // create this handle. Since `set_current` fails when the ID of the + // handle does not match the current ID, we should attempt to use the + // current thread ID here instead of unconditionally creating a new + // one. Also see #130210. + let thread = unsafe { Thread::new_main(thread::current_id()) }; + if let Err(_thread) = thread::set_current(thread) { + // `thread::current` will create a new handle if none has been set yet. + // Thus, if someone uses it before main, this call will fail. That's a + // bad idea though, as we then cannot set the main thread name here. + // + // FIXME: detect the main thread in `thread::current` and use the + // correct name there. + rtabort!("code running before main must not use thread::current"); + } +} + +/// Clean up the thread-local runtime state. This *should* be run after all other +/// code managed by the Rust runtime, but will not cause UB if that condition is +/// not fulfilled. Also note that this function is not guaranteed to be run, but +/// skipping it will cause leaks and therefore is to be avoided. +pub(crate) fn thread_cleanup() { + // This function is run in situations where unwinding leads to an abort + // (think `extern "C"` functions). Abort here instead so that we can + // print a nice message. + panic::catch_unwind(|| { + crate::thread::drop_current(); + }) + .unwrap_or_else(handle_rt_panic); } // One-time runtime cleanup. @@ -123,11 +158,6 @@ fn lang_start_internal( argv: *const *const u8, sigpipe: u8, ) -> Result { - use crate::{mem, panic}; - let rt_abort = move |e| { - mem::forget(e); - rtabort!("initialization or cleanup bug"); - }; // Guard against the code called by this function from unwinding outside of the Rust-controlled // code, which is UB. This is a requirement imposed by a combination of how the // `#[lang="start"]` attribute is implemented as well as by the implementation of the panicking @@ -139,16 +169,17 @@ fn lang_start_internal( // prevent std from accidentally introducing a panic to these functions. Another is from // user code from `main` or, more nefariously, as described in e.g. issue #86030. // SAFETY: Only called once during runtime initialization. - panic::catch_unwind(move || unsafe { init(argc, argv, sigpipe) }).map_err(rt_abort)?; + panic::catch_unwind(move || unsafe { init(argc, argv, sigpipe) }) + .unwrap_or_else(handle_rt_panic); let ret_code = panic::catch_unwind(move || panic::catch_unwind(main).unwrap_or(101) as isize) .map_err(move |e| { mem::forget(e); rtabort!("drop of the panic payload panicked"); }); - panic::catch_unwind(cleanup).map_err(rt_abort)?; + panic::catch_unwind(cleanup).unwrap_or_else(handle_rt_panic); // Guard against multiple threads calling `libc::exit` concurrently. // See the documentation for `unique_thread_exit` for more information. - panic::catch_unwind(|| crate::sys::exit_guard::unique_thread_exit()).map_err(rt_abort)?; + panic::catch_unwind(crate::sys::exit_guard::unique_thread_exit).unwrap_or_else(handle_rt_panic); ret_code } diff --git a/library/std/src/sync/barrier/tests.rs b/library/std/src/sync/barrier/tests.rs index 834a3e75158a7..0fbcd9988127b 100644 --- a/library/std/src/sync/barrier/tests.rs +++ b/library/std/src/sync/barrier/tests.rs @@ -1,9 +1,9 @@ -use crate::sync::mpsc::{channel, TryRecvError}; +use crate::sync::mpsc::{TryRecvError, channel}; use crate::sync::{Arc, Barrier}; use crate::thread; #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn test_barrier() { const N: usize = 10; diff --git a/library/std/src/sync/condvar.rs b/library/std/src/sync/condvar.rs index e41cbc1a65c0f..44ffcb528d937 100644 --- a/library/std/src/sync/condvar.rs +++ b/library/std/src/sync/condvar.rs @@ -2,7 +2,7 @@ mod tests; use crate::fmt; -use crate::sync::{mutex, poison, LockResult, MutexGuard, PoisonError}; +use crate::sync::{LockResult, MutexGuard, PoisonError, mutex, poison}; use crate::sys::sync as sys; use crate::time::{Duration, Instant}; diff --git a/library/std/src/sync/condvar/tests.rs b/library/std/src/sync/condvar/tests.rs index 12d13a6b20be3..f9e9066bc92a2 100644 --- a/library/std/src/sync/condvar/tests.rs +++ b/library/std/src/sync/condvar/tests.rs @@ -12,7 +12,7 @@ fn smoke() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn notify_one() { let m = Arc::new(Mutex::new(())); let m2 = m.clone(); @@ -29,7 +29,7 @@ fn notify_one() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn notify_all() { const N: usize = 10; @@ -66,7 +66,7 @@ fn notify_all() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn wait_while() { let pair = Arc::new((Mutex::new(false), Condvar::new())); let pair2 = pair.clone(); @@ -87,7 +87,7 @@ fn wait_while() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // condvar wait not supported fn wait_timeout_wait() { let m = Arc::new(Mutex::new(())); let c = Arc::new(Condvar::new()); @@ -106,7 +106,7 @@ fn wait_timeout_wait() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // condvar wait not supported fn wait_timeout_while_wait() { let m = Arc::new(Mutex::new(())); let c = Arc::new(Condvar::new()); @@ -118,7 +118,7 @@ fn wait_timeout_while_wait() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // condvar wait not supported fn wait_timeout_while_instant_satisfy() { let m = Arc::new(Mutex::new(())); let c = Arc::new(Condvar::new()); @@ -130,7 +130,7 @@ fn wait_timeout_while_instant_satisfy() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn wait_timeout_while_wake() { let pair = Arc::new((Mutex::new(false), Condvar::new())); let pair_copy = pair.clone(); @@ -153,7 +153,7 @@ fn wait_timeout_while_wake() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn wait_timeout_wake() { let m = Arc::new(Mutex::new(())); let c = Arc::new(Condvar::new()); diff --git a/library/std/src/sync/lazy_lock.rs b/library/std/src/sync/lazy_lock.rs index 953aef40e7b76..b05615035d7b1 100644 --- a/library/std/src/sync/lazy_lock.rs +++ b/library/std/src/sync/lazy_lock.rs @@ -44,8 +44,6 @@ union Data { /// /// // The `String` is built, stored in the `LazyLock`, and returned as `&String`. /// let _ = &*DEEP_THOUGHT; -/// // The `String` is retrieved from the `LazyLock` and returned as `&String`. -/// let _ = &*DEEP_THOUGHT; /// ``` /// /// Initialize fields with `LazyLock`. @@ -121,7 +119,7 @@ impl T> LazyLock { pub fn into_inner(mut this: Self) -> Result { let state = this.once.state(); match state { - ExclusiveState::Poisoned => panic!("LazyLock instance has previously been poisoned"), + ExclusiveState::Poisoned => panic_poisoned(), state => { let this = ManuallyDrop::new(this); let data = unsafe { ptr::read(&this.data) }.into_inner(); @@ -134,6 +132,60 @@ impl T> LazyLock { } } + /// Forces the evaluation of this lazy value and returns a mutable reference to + /// the result. + /// + /// # Examples + /// + /// ``` + /// #![feature(lazy_get)] + /// use std::sync::LazyLock; + /// + /// let mut lazy = LazyLock::new(|| 92); + /// + /// let p = LazyLock::force_mut(&mut lazy); + /// assert_eq!(*p, 92); + /// *p = 44; + /// assert_eq!(*lazy, 44); + /// ``` + #[inline] + #[unstable(feature = "lazy_get", issue = "129333")] + pub fn force_mut(this: &mut LazyLock) -> &mut T { + #[cold] + /// # Safety + /// May only be called when the state is `Incomplete`. + unsafe fn really_init_mut T>(this: &mut LazyLock) -> &mut T { + struct PoisonOnPanic<'a, T, F>(&'a mut LazyLock); + impl Drop for PoisonOnPanic<'_, T, F> { + #[inline] + fn drop(&mut self) { + self.0.once.set_state(ExclusiveState::Poisoned); + } + } + + // SAFETY: We always poison if the initializer panics (then we never check the data), + // or set the data on success. + let f = unsafe { ManuallyDrop::take(&mut this.data.get_mut().f) }; + // INVARIANT: Initiated from mutable reference, don't drop because we read it. + let guard = PoisonOnPanic(this); + let data = f(); + guard.0.data.get_mut().value = ManuallyDrop::new(data); + guard.0.once.set_state(ExclusiveState::Complete); + core::mem::forget(guard); + // SAFETY: We put the value there above. + unsafe { &mut this.data.get_mut().value } + } + + let state = this.once.state(); + match state { + ExclusiveState::Poisoned => panic_poisoned(), + // SAFETY: The `Once` states we completed the initialization. + ExclusiveState::Complete => unsafe { &mut this.data.get_mut().value }, + // SAFETY: The state is `Incomplete`. + ExclusiveState::Incomplete => unsafe { really_init_mut(this) }, + } + } + /// Forces the evaluation of this lazy value and returns a reference to /// result. This is equivalent to the `Deref` impl, but is explicit. /// @@ -174,13 +226,58 @@ impl T> LazyLock { } impl LazyLock { - /// Gets the inner value if it has already been initialized. - fn get(&self) -> Option<&T> { - if self.once.is_completed() { + /// Returns a reference to the value if initialized, or `None` if not. + /// + /// # Examples + /// + /// ``` + /// #![feature(lazy_get)] + /// + /// use std::sync::LazyLock; + /// + /// let mut lazy = LazyLock::new(|| 92); + /// + /// assert_eq!(LazyLock::get_mut(&mut lazy), None); + /// let _ = LazyLock::force(&lazy); + /// *LazyLock::get_mut(&mut lazy).unwrap() = 44; + /// assert_eq!(*lazy, 44); + /// ``` + #[inline] + #[unstable(feature = "lazy_get", issue = "129333")] + pub fn get_mut(this: &mut LazyLock) -> Option<&mut T> { + // `state()` does not perform an atomic load, so prefer it over `is_complete()`. + let state = this.once.state(); + match state { + // SAFETY: + // The closure has been run successfully, so `value` has been initialized. + ExclusiveState::Complete => Some(unsafe { &mut this.data.get_mut().value }), + _ => None, + } + } + + /// Returns a mutable reference to the value if initialized, or `None` if not. + /// + /// # Examples + /// + /// ``` + /// #![feature(lazy_get)] + /// + /// use std::sync::LazyLock; + /// + /// let lazy = LazyLock::new(|| 92); + /// + /// assert_eq!(LazyLock::get(&lazy), None); + /// let _ = LazyLock::force(&lazy); + /// assert_eq!(LazyLock::get(&lazy), Some(&92)); + /// ``` + #[inline] + #[unstable(feature = "lazy_get", issue = "129333")] + pub fn get(this: &LazyLock) -> Option<&T> { + if this.once.is_completed() { // SAFETY: // The closure has been run successfully, so `value` has been initialized // and will not be modified again. - Some(unsafe { &*(*self.data.get()).value }) + Some(unsafe { &(*this.data.get()).value }) } else { None } @@ -228,7 +325,7 @@ impl Default for LazyLock { impl fmt::Debug for LazyLock { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { let mut d = f.debug_tuple("LazyLock"); - match self.get() { + match LazyLock::get(self) { Some(v) => d.field(v), None => d.field(&format_args!("")), }; @@ -236,6 +333,12 @@ impl fmt::Debug for LazyLock { } } +#[cold] +#[inline(never)] +fn panic_poisoned() -> ! { + panic!("LazyLock instance has previously been poisoned") +} + // We never create a `&F` from a `&LazyLock` so it is fine // to not impl `Sync` for `F`. #[stable(feature = "lazy_cell", since = "1.80.0")] diff --git a/library/std/src/sync/lazy_lock/tests.rs b/library/std/src/sync/lazy_lock/tests.rs index 8a6ab4ac4fd99..7d7dde5434990 100644 --- a/library/std/src/sync/lazy_lock/tests.rs +++ b/library/std/src/sync/lazy_lock/tests.rs @@ -34,6 +34,7 @@ fn lazy_default() { } #[test] +#[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] fn lazy_poisoning() { let x: LazyCell = LazyCell::new(|| panic!("kaboom")); for _ in 0..2 { @@ -43,7 +44,7 @@ fn lazy_poisoning() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn sync_lazy_new() { static CALLED: AtomicUsize = AtomicUsize::new(0); static SYNC_LAZY: LazyLock = LazyLock::new(|| { @@ -90,7 +91,7 @@ fn sync_lazy_default() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn static_sync_lazy() { static XS: LazyLock> = LazyLock::new(|| { let mut xs = Vec::new(); @@ -123,6 +124,7 @@ fn static_sync_lazy_via_fn() { } #[test] +#[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] fn sync_lazy_poisoning() { let x: LazyLock = LazyLock::new(|| panic!("kaboom")); for _ in 0..2 { @@ -142,3 +144,24 @@ fn is_sync_send() { fn assert_traits() {} assert_traits::>(); } + +#[test] +#[should_panic = "has previously been poisoned"] +fn lazy_force_mut_panic() { + let mut lazy = LazyLock::::new(|| panic!()); + crate::panic::catch_unwind(crate::panic::AssertUnwindSafe(|| { + let _ = LazyLock::force_mut(&mut lazy); + })) + .unwrap_err(); + let _ = &*lazy; +} + +#[test] +fn lazy_force_mut() { + let s = "abc".to_owned(); + let mut lazy = LazyLock::new(move || s); + LazyLock::force_mut(&mut lazy); + let p = LazyLock::force_mut(&mut lazy); + p.clear(); + LazyLock::force_mut(&mut lazy); +} diff --git a/library/std/src/sync/mod.rs b/library/std/src/sync/mod.rs index d0ba8cc3b47df..0fb77331293fe 100644 --- a/library/std/src/sync/mod.rs +++ b/library/std/src/sync/mod.rs @@ -9,6 +9,9 @@ //! Consider the following code, operating on some global static variables: //! //! ```rust +//! // FIXME(static_mut_refs): Do not allow `static_mut_refs` lint +//! #![allow(static_mut_refs)] +//! //! static mut A: u32 = 0; //! static mut B: u32 = 0; //! static mut C: u32 = 0; @@ -130,6 +133,11 @@ //! inter-thread synchronisation mechanism, at the cost of some //! extra memory. //! +//! - [`mpmc`]: Multi-producer, multi-consumer queues, used for +//! message-based communication. Can provide a lightweight +//! inter-thread synchronisation mechanism, at the cost of some +//! extra memory. +//! //! - [`Mutex`]: Mutual Exclusion mechanism, which ensures that at //! most one thread at a time is able to access some data. //! @@ -150,6 +158,7 @@ //! [`Arc`]: crate::sync::Arc //! [`Barrier`]: crate::sync::Barrier //! [`Condvar`]: crate::sync::Condvar +//! [`mpmc`]: crate::sync::mpmc //! [`mpsc`]: crate::sync::mpsc //! [`Mutex`]: crate::sync::Mutex //! [`Once`]: crate::sync::Once @@ -158,10 +167,10 @@ #![stable(feature = "rust1", since = "1.0.0")] -#[stable(feature = "rust1", since = "1.0.0")] -pub use core::sync::atomic; #[unstable(feature = "exclusive_wrapper", issue = "98407")] pub use core::sync::Exclusive; +#[stable(feature = "rust1", since = "1.0.0")] +pub use core::sync::atomic; #[stable(feature = "rust1", since = "1.0.0")] pub use alloc_crate::sync::{Arc, Weak}; @@ -178,7 +187,7 @@ pub use self::mutex::MappedMutexGuard; pub use self::mutex::{Mutex, MutexGuard}; #[stable(feature = "rust1", since = "1.0.0")] #[allow(deprecated)] -pub use self::once::{Once, OnceState, ONCE_INIT}; +pub use self::once::{ONCE_INIT, Once, OnceState}; #[stable(feature = "once_cell", since = "1.70.0")] pub use self::once_lock::OnceLock; #[stable(feature = "rust1", since = "1.0.0")] @@ -190,12 +199,13 @@ pub use self::rwlock::{MappedRwLockReadGuard, MappedRwLockWriteGuard}; #[stable(feature = "rust1", since = "1.0.0")] pub use self::rwlock::{RwLock, RwLockReadGuard, RwLockWriteGuard}; +#[unstable(feature = "mpmc_channel", issue = "126840")] +pub mod mpmc; pub mod mpsc; mod barrier; mod condvar; mod lazy_lock; -mod mpmc; mod mutex; pub(crate) mod once; mod once_lock; diff --git a/library/std/src/sync/mpmc/array.rs b/library/std/src/sync/mpmc/array.rs index 34acd9c9a943b..2c8ba411f3023 100644 --- a/library/std/src/sync/mpmc/array.rs +++ b/library/std/src/sync/mpmc/array.rs @@ -484,7 +484,7 @@ impl Channel { /// /// # Panicking /// If a destructor panics, the remaining messages are leaked, matching the - /// behaviour of the unbounded channel. + /// behavior of the unbounded channel. /// /// # Safety /// This method must only be called when dropping the last receiver. The diff --git a/library/std/src/sync/mpmc/context.rs b/library/std/src/sync/mpmc/context.rs index 8db3c9896eb77..2371d32d4ea0d 100644 --- a/library/std/src/sync/mpmc/context.rs +++ b/library/std/src/sync/mpmc/context.rs @@ -4,8 +4,8 @@ use super::select::Selected; use super::waker::current_thread_id; use crate::cell::Cell; use crate::ptr; -use crate::sync::atomic::{AtomicPtr, AtomicUsize, Ordering}; use crate::sync::Arc; +use crate::sync::atomic::{AtomicPtr, AtomicUsize, Ordering}; use crate::thread::{self, Thread}; use crate::time::Instant; diff --git a/library/std/src/sync/mpmc/error.rs b/library/std/src/sync/mpmc/error.rs index e3aec7e76232f..e34b56d08312b 100644 --- a/library/std/src/sync/mpmc/error.rs +++ b/library/std/src/sync/mpmc/error.rs @@ -7,6 +7,7 @@ use crate::{error, fmt}; /// /// [`send_timeout`]: super::Sender::send_timeout #[derive(PartialEq, Eq, Clone, Copy)] +#[unstable(feature = "mpmc_channel", issue = "126840")] pub enum SendTimeoutError { /// The message could not be sent because the channel is full and the operation timed out. /// @@ -18,12 +19,14 @@ pub enum SendTimeoutError { Disconnected(T), } +#[unstable(feature = "mpmc_channel", issue = "126840")] impl fmt::Debug for SendTimeoutError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { "SendTimeoutError(..)".fmt(f) } } +#[unstable(feature = "mpmc_channel", issue = "126840")] impl fmt::Display for SendTimeoutError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { match *self { @@ -33,8 +36,10 @@ impl fmt::Display for SendTimeoutError { } } +#[unstable(feature = "mpmc_channel", issue = "126840")] impl error::Error for SendTimeoutError {} +#[unstable(feature = "mpmc_channel", issue = "126840")] impl From> for SendTimeoutError { fn from(err: SendError) -> SendTimeoutError { match err { diff --git a/library/std/src/sync/mpmc/mod.rs b/library/std/src/sync/mpmc/mod.rs index c640e07348ea0..44e146a89bafb 100644 --- a/library/std/src/sync/mpmc/mod.rs +++ b/library/std/src/sync/mpmc/mod.rs @@ -1,8 +1,114 @@ -//! Multi-producer multi-consumer channels. +//! Multi-producer, multi-consumer FIFO queue communication primitives. +//! +//! This module provides message-based communication over channels, concretely +//! defined by two types: +//! +//! * [`Sender`] +//! * [`Receiver`] +//! +//! [`Sender`]s are used to send data to a set of [`Receiver`]s. Both +//! sender and receiver are cloneable (multi-producer) such that many threads can send +//! simultaneously to receivers (multi-consumer). +//! +//! These channels come in two flavors: +//! +//! 1. An asynchronous, infinitely buffered channel. The [`channel`] function +//! will return a `(Sender, Receiver)` tuple where all sends will be +//! **asynchronous** (they never block). The channel conceptually has an +//! infinite buffer. +//! +//! 2. A synchronous, bounded channel. The [`sync_channel`] function will +//! return a `(SyncSender, Receiver)` tuple where the storage for pending +//! messages is a pre-allocated buffer of a fixed size. All sends will be +//! **synchronous** by blocking until there is buffer space available. Note +//! that a bound of 0 is allowed, causing the channel to become a "rendezvous" +//! channel where each sender atomically hands off a message to a receiver. +//! +//! [`send`]: Sender::send +//! +//! ## Disconnection +//! +//! The send and receive operations on channels will all return a [`Result`] +//! indicating whether the operation succeeded or not. An unsuccessful operation +//! is normally indicative of the other half of a channel having "hung up" by +//! being dropped in its corresponding thread. +//! +//! Once half of a channel has been deallocated, most operations can no longer +//! continue to make progress, so [`Err`] will be returned. Many applications +//! will continue to [`unwrap`] the results returned from this module, +//! instigating a propagation of failure among threads if one unexpectedly dies. +//! +//! [`unwrap`]: Result::unwrap +//! +//! # Examples +//! +//! Simple usage: +//! +//! ``` +//! #![feature(mpmc_channel)] +//! +//! use std::thread; +//! use std::sync::mpmc::channel; +//! +//! // Create a simple streaming channel +//! let (tx, rx) = channel(); +//! thread::spawn(move || { +//! tx.send(10).unwrap(); +//! }); +//! assert_eq!(rx.recv().unwrap(), 10); +//! ``` +//! +//! Shared usage: +//! +//! ``` +//! #![feature(mpmc_channel)] +//! +//! use std::thread; +//! use std::sync::mpmc::channel; +//! +//! thread::scope(|s| { +//! // Create a shared channel that can be sent along from many threads +//! // where tx is the sending half (tx for transmission), and rx is the receiving +//! // half (rx for receiving). +//! let (tx, rx) = channel(); +//! for i in 0..10 { +//! let tx = tx.clone(); +//! s.spawn(move || { +//! tx.send(i).unwrap(); +//! }); +//! } +//! +//! for _ in 0..5 { +//! let rx1 = rx.clone(); +//! let rx2 = rx.clone(); +//! s.spawn(move || { +//! let j = rx1.recv().unwrap(); +//! assert!(0 <= j && j < 10); +//! }); +//! s.spawn(move || { +//! let j = rx2.recv().unwrap(); +//! assert!(0 <= j && j < 10); +//! }); +//! } +//! }) +//! ``` +//! +//! Propagating panics: +//! +//! ``` +//! #![feature(mpmc_channel)] +//! +//! use std::sync::mpmc::channel; +//! +//! // The call to recv() will return an error because the channel has already +//! // hung up (or been deallocated) +//! let (tx, rx) = channel::(); +//! drop(tx); +//! assert!(rx.recv().is_err()); +//! ``` -// This module is not currently exposed publicly, but is used -// as the implementation for the channels in `sync::mpsc`. The -// implementation comes from the crossbeam-channel crate: +// This module is used as the implementation for the channels in `sync::mpsc`. +// The implementation comes from the crossbeam-channel crate: // // Copyright (c) 2019 The Crossbeam Project Developers // @@ -46,9 +152,47 @@ use crate::fmt; use crate::panic::{RefUnwindSafe, UnwindSafe}; use crate::time::{Duration, Instant}; -/// Creates a channel of unbounded capacity. +/// Creates a new asynchronous channel, returning the sender/receiver halves. +/// All data sent on the [`Sender`] will become available on the [`Receiver`] in +/// the same order as it was sent, and no [`send`] will block the calling thread +/// (this channel has an "infinite buffer", unlike [`sync_channel`], which will +/// block after its buffer limit is reached). [`recv`] will block until a message +/// is available while there is at least one [`Sender`] alive (including clones). /// -/// This channel has a growable buffer that can hold any number of messages at a time. +/// The [`Sender`] can be cloned to [`send`] to the same channel multiple times. +/// The [`Receiver`] also can be cloned to have multi receivers. +/// +/// If the [`Receiver`] is disconnected while trying to [`send`] with the +/// [`Sender`], the [`send`] method will return a [`SendError`]. Similarly, if the +/// [`Sender`] is disconnected while trying to [`recv`], the [`recv`] method will +/// return a [`RecvError`]. +/// +/// [`send`]: Sender::send +/// [`recv`]: Receiver::recv +/// +/// # Examples +/// +/// ``` +/// #![feature(mpmc_channel)] +/// +/// use std::sync::mpmc::channel; +/// use std::thread; +/// +/// let (sender, receiver) = channel(); +/// +/// // Spawn off an expensive computation +/// thread::spawn(move || { +/// # fn expensive_computation() {} +/// sender.send(expensive_computation()).unwrap(); +/// }); +/// +/// // Do some useful work for awhile +/// +/// // Let's see what that answer was +/// println!("{:?}", receiver.recv().unwrap()); +/// ``` +#[must_use] +#[unstable(feature = "mpmc_channel", issue = "126840")] pub fn channel() -> (Sender, Receiver) { let (s, r) = counter::new(list::Channel::new()); let s = Sender { flavor: SenderFlavor::List(s) }; @@ -56,12 +200,50 @@ pub fn channel() -> (Sender, Receiver) { (s, r) } -/// Creates a channel of bounded capacity. +/// Creates a new synchronous, bounded channel. +/// All data sent on the [`Sender`] will become available on the [`Receiver`] +/// in the same order as it was sent. Like asynchronous [`channel`]s, the +/// [`Receiver`] will block until a message becomes available. `sync_channel` +/// differs greatly in the semantics of the sender, however. +/// +/// This channel has an internal buffer on which messages will be queued. +/// `bound` specifies the buffer size. When the internal buffer becomes full, +/// future sends will *block* waiting for the buffer to open up. Note that a +/// buffer size of 0 is valid, in which case this becomes "rendezvous channel" +/// where each [`send`] will not return until a [`recv`] is paired with it. +/// +/// The [`Sender`] can be cloned to [`send`] to the same channel multiple +/// times. The [`Receiver`] also can be cloned to have multi receivers. +/// +/// Like asynchronous channels, if the [`Receiver`] is disconnected while trying +/// to [`send`] with the [`Sender`], the [`send`] method will return a +/// [`SendError`]. Similarly, If the [`Sender`] is disconnected while trying +/// to [`recv`], the [`recv`] method will return a [`RecvError`]. +/// +/// [`send`]: Sender::send +/// [`recv`]: Receiver::recv +/// +/// # Examples +/// +/// ``` +/// use std::sync::mpsc::sync_channel; +/// use std::thread; +/// +/// let (sender, receiver) = sync_channel(1); /// -/// This channel has a buffer that can hold at most `cap` messages at a time. +/// // this returns immediately +/// sender.send(1).unwrap(); /// -/// A special case is zero-capacity channel, which cannot hold any messages. Instead, send and -/// receive operations must appear at the same time in order to pair up and pass the message over. +/// thread::spawn(move || { +/// // this will block until the previous message has been received +/// sender.send(2).unwrap(); +/// }); +/// +/// assert_eq!(receiver.recv().unwrap(), 1); +/// assert_eq!(receiver.recv().unwrap(), 2); +/// ``` +#[must_use] +#[unstable(feature = "mpmc_channel", issue = "126840")] pub fn sync_channel(cap: usize) -> (Sender, Receiver) { if cap == 0 { let (s, r) = counter::new(zero::Channel::new()); @@ -76,7 +258,42 @@ pub fn sync_channel(cap: usize) -> (Sender, Receiver) { } } -/// The sending side of a channel. +/// The sending-half of Rust's synchronous [`channel`] type. +/// +/// Messages can be sent through this channel with [`send`]. +/// +/// Note: all senders (the original and its clones) need to be dropped for the receiver +/// to stop blocking to receive messages with [`Receiver::recv`]. +/// +/// [`send`]: Sender::send +/// +/// # Examples +/// +/// ```rust +/// #![feature(mpmc_channel)] +/// +/// use std::sync::mpmc::channel; +/// use std::thread; +/// +/// let (sender, receiver) = channel(); +/// let sender2 = sender.clone(); +/// +/// // First thread owns sender +/// thread::spawn(move || { +/// sender.send(1).unwrap(); +/// }); +/// +/// // Second thread owns sender2 +/// thread::spawn(move || { +/// sender2.send(2).unwrap(); +/// }); +/// +/// let msg = receiver.recv().unwrap(); +/// let msg2 = receiver.recv().unwrap(); +/// +/// assert_eq!(3, msg + msg2); +/// ``` +#[unstable(feature = "mpmc_channel", issue = "126840")] pub struct Sender { flavor: SenderFlavor, } @@ -93,10 +310,14 @@ enum SenderFlavor { Zero(counter::Sender>), } +#[unstable(feature = "mpmc_channel", issue = "126840")] unsafe impl Send for Sender {} +#[unstable(feature = "mpmc_channel", issue = "126840")] unsafe impl Sync for Sender {} +#[unstable(feature = "mpmc_channel", issue = "126840")] impl UnwindSafe for Sender {} +#[unstable(feature = "mpmc_channel", issue = "126840")] impl RefUnwindSafe for Sender {} impl Sender { @@ -107,6 +328,19 @@ impl Sender { /// /// If called on a zero-capacity channel, this method will send the message only if there /// happens to be a receive operation on the other side of the channel at the same time. + /// + /// # Examples + /// + /// ```rust + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc::{channel, Receiver, Sender}; + /// + /// let (sender, _receiver): (Sender, Receiver) = channel(); + /// + /// assert!(sender.try_send(1).is_ok()); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn try_send(&self, msg: T) -> Result<(), TrySendError> { match &self.flavor { SenderFlavor::Array(chan) => chan.try_send(msg), @@ -115,14 +349,36 @@ impl Sender { } } - /// Blocks the current thread until a message is sent or the channel is disconnected. + /// Attempts to send a value on this channel, returning it back if it could + /// not be sent. /// - /// If the channel is full and not disconnected, this call will block until the send operation - /// can proceed. If the channel becomes disconnected, this call will wake up and return an - /// error. The returned error contains the original message. + /// A successful send occurs when it is determined that the other end of + /// the channel has not hung up already. An unsuccessful send would be one + /// where the corresponding receiver has already been deallocated. Note + /// that a return value of [`Err`] means that the data will never be + /// received, but a return value of [`Ok`] does *not* mean that the data + /// will be received. It is possible for the corresponding receiver to + /// hang up immediately after this function returns [`Ok`]. /// - /// If called on a zero-capacity channel, this method will wait for a receive operation to - /// appear on the other side of the channel. + /// This method will never block the current thread. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc::channel; + /// + /// let (tx, rx) = channel(); + /// + /// // This send is always successful + /// tx.send(1).unwrap(); + /// + /// // This send will fail because the receiver is gone + /// drop(rx); + /// assert!(tx.send(1).is_err()); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn send(&self, msg: T) -> Result<(), SendError> { match &self.flavor { SenderFlavor::Array(chan) => chan.send(msg, None), @@ -136,10 +392,6 @@ impl Sender { } } -// The methods below are not used by `sync::mpsc`, but -// are useful and we'll likely want to expose them -// eventually -#[allow(unused)] impl Sender { /// Waits for a message to be sent into the channel, but only for a limited time. /// @@ -149,6 +401,20 @@ impl Sender { /// /// If called on a zero-capacity channel, this method will wait for a receive operation to /// appear on the other side of the channel. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc::channel; + /// use std::time::Duration; + /// + /// let (tx, rx) = channel(); + /// + /// tx.send_timeout(1, Duration::from_millis(400)).unwrap(); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn send_timeout(&self, msg: T, timeout: Duration) -> Result<(), SendTimeoutError> { match Instant::now().checked_add(timeout) { Some(deadline) => self.send_deadline(msg, deadline), @@ -165,6 +431,21 @@ impl Sender { /// /// If called on a zero-capacity channel, this method will wait for a receive operation to /// appear on the other side of the channel. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc::channel; + /// use std::time::{Duration, Instant}; + /// + /// let (tx, rx) = channel(); + /// + /// let t = Instant::now() + Duration::from_millis(400); + /// tx.send_deadline(1, t).unwrap(); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn send_deadline(&self, msg: T, deadline: Instant) -> Result<(), SendTimeoutError> { match &self.flavor { SenderFlavor::Array(chan) => chan.send(msg, Some(deadline)), @@ -176,6 +457,31 @@ impl Sender { /// Returns `true` if the channel is empty. /// /// Note: Zero-capacity channels are always empty. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc; + /// use std::thread; + /// + /// let (send, _recv) = mpmc::channel(); + /// + /// let tx1 = send.clone(); + /// let tx2 = send.clone(); + /// + /// assert!(tx1.is_empty()); + /// + /// let handle = thread::spawn(move || { + /// tx2.send(1u8).unwrap(); + /// }); + /// + /// handle.join().unwrap(); + /// + /// assert!(!tx1.is_empty()); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn is_empty(&self) -> bool { match &self.flavor { SenderFlavor::Array(chan) => chan.is_empty(), @@ -187,6 +493,29 @@ impl Sender { /// Returns `true` if the channel is full. /// /// Note: Zero-capacity channels are always full. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc; + /// use std::thread; + /// + /// let (send, _recv) = mpmc::sync_channel(1); + /// + /// let (tx1, tx2) = (send.clone(), send.clone()); + /// assert!(!tx1.is_full()); + /// + /// let handle = thread::spawn(move || { + /// tx2.send(1u8).unwrap(); + /// }); + /// + /// handle.join().unwrap(); + /// + /// assert!(tx1.is_full()); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn is_full(&self) -> bool { match &self.flavor { SenderFlavor::Array(chan) => chan.is_full(), @@ -196,6 +525,29 @@ impl Sender { } /// Returns the number of messages in the channel. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc; + /// use std::thread; + /// + /// let (send, _recv) = mpmc::channel(); + /// let (tx1, tx2) = (send.clone(), send.clone()); + /// + /// assert_eq!(tx1.len(), 0); + /// + /// let handle = thread::spawn(move || { + /// tx2.send(1u8).unwrap(); + /// }); + /// + /// handle.join().unwrap(); + /// + /// assert_eq!(tx1.len(), 1); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn len(&self) -> usize { match &self.flavor { SenderFlavor::Array(chan) => chan.len(), @@ -205,6 +557,29 @@ impl Sender { } /// If the channel is bounded, returns its capacity. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc; + /// use std::thread; + /// + /// let (send, _recv) = mpmc::sync_channel(3); + /// let (tx1, tx2) = (send.clone(), send.clone()); + /// + /// assert_eq!(tx1.capacity(), Some(3)); + /// + /// let handle = thread::spawn(move || { + /// tx2.send(1u8).unwrap(); + /// }); + /// + /// handle.join().unwrap(); + /// + /// assert_eq!(tx1.capacity(), Some(3)); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn capacity(&self) -> Option { match &self.flavor { SenderFlavor::Array(chan) => chan.capacity(), @@ -214,6 +589,21 @@ impl Sender { } /// Returns `true` if senders belong to the same channel. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc; + /// + /// let (tx1, _) = mpmc::channel::(); + /// let (tx2, _) = mpmc::channel::(); + /// + /// assert!(tx1.same_channel(&tx1)); + /// assert!(!tx1.same_channel(&tx2)); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn same_channel(&self, other: &Sender) -> bool { match (&self.flavor, &other.flavor) { (SenderFlavor::Array(ref a), SenderFlavor::Array(ref b)) => a == b, @@ -224,6 +614,7 @@ impl Sender { } } +#[unstable(feature = "mpmc_channel", issue = "126840")] impl Drop for Sender { fn drop(&mut self) { unsafe { @@ -236,6 +627,7 @@ impl Drop for Sender { } } +#[unstable(feature = "mpmc_channel", issue = "126840")] impl Clone for Sender { fn clone(&self) -> Self { let flavor = match &self.flavor { @@ -248,17 +640,216 @@ impl Clone for Sender { } } +#[unstable(feature = "mpmc_channel", issue = "126840")] impl fmt::Debug for Sender { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { f.pad("Sender { .. }") } } -/// The receiving side of a channel. +/// The receiving half of Rust's [`channel`] (or [`sync_channel`]) type. +/// Different threads can share this [`Sender`] by cloning it. +/// +/// Messages sent to the channel can be retrieved using [`recv`]. +/// +/// [`recv`]: Receiver::recv +/// +/// # Examples +/// +/// ```rust +/// #![feature(mpmc_channel)] +/// +/// use std::sync::mpmc::channel; +/// use std::thread; +/// use std::time::Duration; +/// +/// let (send, recv) = channel(); +/// +/// let tx_thread = thread::spawn(move || { +/// send.send("Hello world!").unwrap(); +/// thread::sleep(Duration::from_secs(2)); // block for two seconds +/// send.send("Delayed for 2 seconds").unwrap(); +/// }); +/// +/// let (rx1, rx2) = (recv.clone(), recv.clone()); +/// let rx_thread_1 = thread::spawn(move || { +/// println!("{}", rx1.recv().unwrap()); // Received immediately +/// }); +/// let rx_thread_2 = thread::spawn(move || { +/// println!("{}", rx2.recv().unwrap()); // Received after 2 seconds +/// }); +/// +/// tx_thread.join().unwrap(); +/// rx_thread_1.join().unwrap(); +/// rx_thread_2.join().unwrap(); +/// ``` +#[unstable(feature = "mpmc_channel", issue = "126840")] pub struct Receiver { flavor: ReceiverFlavor, } +/// An iterator over messages on a [`Receiver`], created by [`iter`]. +/// +/// This iterator will block whenever [`next`] is called, +/// waiting for a new message, and [`None`] will be returned +/// when the corresponding channel has hung up. +/// +/// [`iter`]: Receiver::iter +/// [`next`]: Iterator::next +/// +/// # Examples +/// +/// ```rust +/// #![feature(mpmc_channel)] +/// +/// use std::sync::mpmc::channel; +/// use std::thread; +/// +/// let (send, recv) = channel(); +/// +/// thread::spawn(move || { +/// send.send(1u8).unwrap(); +/// send.send(2u8).unwrap(); +/// send.send(3u8).unwrap(); +/// }); +/// +/// for x in recv.iter() { +/// println!("Got: {x}"); +/// } +/// ``` +#[unstable(feature = "mpmc_channel", issue = "126840")] +#[derive(Debug)] +pub struct Iter<'a, T: 'a> { + rx: &'a Receiver, +} + +/// An iterator that attempts to yield all pending values for a [`Receiver`], +/// created by [`try_iter`]. +/// +/// [`None`] will be returned when there are no pending values remaining or +/// if the corresponding channel has hung up. +/// +/// This iterator will never block the caller in order to wait for data to +/// become available. Instead, it will return [`None`]. +/// +/// [`try_iter`]: Receiver::try_iter +/// +/// # Examples +/// +/// ```rust +/// #![feature(mpmc_channel)] +/// +/// use std::sync::mpmc::channel; +/// use std::thread; +/// use std::time::Duration; +/// +/// let (sender, receiver) = channel(); +/// +/// // Nothing is in the buffer yet +/// assert!(receiver.try_iter().next().is_none()); +/// println!("Nothing in the buffer..."); +/// +/// thread::spawn(move || { +/// sender.send(1).unwrap(); +/// sender.send(2).unwrap(); +/// sender.send(3).unwrap(); +/// }); +/// +/// println!("Going to sleep..."); +/// thread::sleep(Duration::from_secs(2)); // block for two seconds +/// +/// for x in receiver.try_iter() { +/// println!("Got: {x}"); +/// } +/// ``` +#[unstable(feature = "mpmc_channel", issue = "126840")] +#[derive(Debug)] +pub struct TryIter<'a, T: 'a> { + rx: &'a Receiver, +} + +/// An owning iterator over messages on a [`Receiver`], +/// created by [`into_iter`]. +/// +/// This iterator will block whenever [`next`] +/// is called, waiting for a new message, and [`None`] will be +/// returned if the corresponding channel has hung up. +/// +/// [`into_iter`]: Receiver::into_iter +/// [`next`]: Iterator::next +/// +/// # Examples +/// +/// ```rust +/// #![feature(mpmc_channel)] +/// +/// use std::sync::mpmc::channel; +/// use std::thread; +/// +/// let (send, recv) = channel(); +/// +/// thread::spawn(move || { +/// send.send(1u8).unwrap(); +/// send.send(2u8).unwrap(); +/// send.send(3u8).unwrap(); +/// }); +/// +/// for x in recv.into_iter() { +/// println!("Got: {x}"); +/// } +/// ``` +#[unstable(feature = "mpmc_channel", issue = "126840")] +#[derive(Debug)] +pub struct IntoIter { + rx: Receiver, +} + +#[unstable(feature = "mpmc_channel", issue = "126840")] +impl<'a, T> Iterator for Iter<'a, T> { + type Item = T; + + fn next(&mut self) -> Option { + self.rx.recv().ok() + } +} + +#[unstable(feature = "mpmc_channel", issue = "126840")] +impl<'a, T> Iterator for TryIter<'a, T> { + type Item = T; + + fn next(&mut self) -> Option { + self.rx.try_recv().ok() + } +} + +#[unstable(feature = "mpmc_channel", issue = "126840")] +impl<'a, T> IntoIterator for &'a Receiver { + type Item = T; + type IntoIter = Iter<'a, T>; + + fn into_iter(self) -> Iter<'a, T> { + self.iter() + } +} + +#[unstable(feature = "mpmc_channel", issue = "126840")] +impl Iterator for IntoIter { + type Item = T; + fn next(&mut self) -> Option { + self.rx.recv().ok() + } +} + +#[unstable(feature = "mpmc_channel", issue = "126840")] +impl IntoIterator for Receiver { + type Item = T; + type IntoIter = IntoIter; + + fn into_iter(self) -> IntoIter { + IntoIter { rx: self } + } +} + /// Receiver flavors. enum ReceiverFlavor { /// Bounded channel based on a preallocated array. @@ -271,20 +862,46 @@ enum ReceiverFlavor { Zero(counter::Receiver>), } +#[unstable(feature = "mpmc_channel", issue = "126840")] unsafe impl Send for Receiver {} +#[unstable(feature = "mpmc_channel", issue = "126840")] unsafe impl Sync for Receiver {} +#[unstable(feature = "mpmc_channel", issue = "126840")] impl UnwindSafe for Receiver {} +#[unstable(feature = "mpmc_channel", issue = "126840")] impl RefUnwindSafe for Receiver {} impl Receiver { /// Attempts to receive a message from the channel without blocking. /// - /// This method will either receive a message from the channel immediately or return an error - /// if the channel is empty. + /// This method will never block the caller in order to wait for data to + /// become available. Instead, this will always return immediately with a + /// possible option of pending data on the channel. /// /// If called on a zero-capacity channel, this method will receive a message only if there /// happens to be a send operation on the other side of the channel at the same time. + /// + /// This is useful for a flavor of "optimistic check" before deciding to + /// block on a receiver. + /// + /// Compared with [`recv`], this function has two failure cases instead of one + /// (one for disconnection, one for an empty buffer). + /// + /// [`recv`]: Self::recv + /// + /// # Examples + /// + /// ```rust + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc::{Receiver, channel}; + /// + /// let (_, receiver): (_, Receiver) = channel(); + /// + /// assert!(receiver.try_recv().is_err()); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn try_recv(&self) -> Result { match &self.flavor { ReceiverFlavor::Array(chan) => chan.try_recv(), @@ -293,15 +910,64 @@ impl Receiver { } } - /// Blocks the current thread until a message is received or the channel is empty and - /// disconnected. + /// Attempts to wait for a value on this receiver, returning an error if the + /// corresponding channel has hung up. /// - /// If the channel is empty and not disconnected, this call will block until the receive - /// operation can proceed. If the channel is empty and becomes disconnected, this call will - /// wake up and return an error. + /// This function will always block the current thread if there is no data + /// available and it's possible for more data to be sent (at least one sender + /// still exists). Once a message is sent to the corresponding [`Sender`], + /// this receiver will wake up and return that message. /// - /// If called on a zero-capacity channel, this method will wait for a send operation to appear - /// on the other side of the channel. + /// If the corresponding [`Sender`] has disconnected, or it disconnects while + /// this call is blocking, this call will wake up and return [`Err`] to + /// indicate that no more messages can ever be received on this channel. + /// However, since channels are buffered, messages sent before the disconnect + /// will still be properly received. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc; + /// use std::thread; + /// + /// let (send, recv) = mpmc::channel(); + /// let handle = thread::spawn(move || { + /// send.send(1u8).unwrap(); + /// }); + /// + /// handle.join().unwrap(); + /// + /// assert_eq!(Ok(1), recv.recv()); + /// ``` + /// + /// Buffering behavior: + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc; + /// use std::thread; + /// use std::sync::mpmc::RecvError; + /// + /// let (send, recv) = mpmc::channel(); + /// let handle = thread::spawn(move || { + /// send.send(1u8).unwrap(); + /// send.send(2).unwrap(); + /// send.send(3).unwrap(); + /// drop(send); + /// }); + /// + /// // wait for the thread to join so we ensure the sender is dropped + /// handle.join().unwrap(); + /// + /// assert_eq!(Ok(1), recv.recv()); + /// assert_eq!(Ok(2), recv.recv()); + /// assert_eq!(Ok(3), recv.recv()); + /// assert_eq!(Err(RecvError), recv.recv()); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn recv(&self) -> Result { match &self.flavor { ReceiverFlavor::Array(chan) => chan.recv(None), @@ -311,14 +977,65 @@ impl Receiver { .map_err(|_| RecvError) } - /// Waits for a message to be received from the channel, but only for a limited time. + /// Attempts to wait for a value on this receiver, returning an error if the + /// corresponding channel has hung up, or if it waits more than `timeout`. + /// + /// This function will always block the current thread if there is no data + /// available and it's possible for more data to be sent (at least one sender + /// still exists). Once a message is sent to the corresponding [`Sender`], + /// this receiver will wake up and return that message. + /// + /// If the corresponding [`Sender`] has disconnected, or it disconnects while + /// this call is blocking, this call will wake up and return [`Err`] to + /// indicate that no more messages can ever be received on this channel. + /// However, since channels are buffered, messages sent before the disconnect + /// will still be properly received. + /// + /// # Examples + /// + /// Successfully receiving value before encountering timeout: /// - /// If the channel is empty and not disconnected, this call will block until the receive - /// operation can proceed or the operation times out. If the channel is empty and becomes - /// disconnected, this call will wake up and return an error. + /// ```no_run + /// #![feature(mpmc_channel)] /// - /// If called on a zero-capacity channel, this method will wait for a send operation to appear - /// on the other side of the channel. + /// use std::thread; + /// use std::time::Duration; + /// use std::sync::mpmc; + /// + /// let (send, recv) = mpmc::channel(); + /// + /// thread::spawn(move || { + /// send.send('a').unwrap(); + /// }); + /// + /// assert_eq!( + /// recv.recv_timeout(Duration::from_millis(400)), + /// Ok('a') + /// ); + /// ``` + /// + /// Receiving an error upon reaching timeout: + /// + /// ```no_run + /// #![feature(mpmc_channel)] + /// + /// use std::thread; + /// use std::time::Duration; + /// use std::sync::mpmc; + /// + /// let (send, recv) = mpmc::channel(); + /// + /// thread::spawn(move || { + /// thread::sleep(Duration::from_millis(800)); + /// send.send('a').unwrap(); + /// }); + /// + /// assert_eq!( + /// recv.recv_timeout(Duration::from_millis(400)), + /// Err(mpmc::RecvTimeoutError::Timeout) + /// ); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn recv_timeout(&self, timeout: Duration) -> Result { match Instant::now().checked_add(timeout) { Some(deadline) => self.recv_deadline(deadline), @@ -327,14 +1044,65 @@ impl Receiver { } } - /// Waits for a message to be received from the channel, but only for a limited time. + /// Attempts to wait for a value on this receiver, returning an error if the + /// corresponding channel has hung up, or if `deadline` is reached. + /// + /// This function will always block the current thread if there is no data + /// available and it's possible for more data to be sent. Once a message is + /// sent to the corresponding [`Sender`], then this receiver will wake up + /// and return that message. + /// + /// If the corresponding [`Sender`] has disconnected, or it disconnects while + /// this call is blocking, this call will wake up and return [`Err`] to + /// indicate that no more messages can ever be received on this channel. + /// However, since channels are buffered, messages sent before the disconnect + /// will still be properly received. + /// + /// # Examples + /// + /// Successfully receiving value before reaching deadline: + /// + /// ```no_run + /// #![feature(mpmc_channel)] + /// + /// use std::thread; + /// use std::time::{Duration, Instant}; + /// use std::sync::mpmc; /// - /// If the channel is empty and not disconnected, this call will block until the receive - /// operation can proceed or the operation times out. If the channel is empty and becomes - /// disconnected, this call will wake up and return an error. + /// let (send, recv) = mpmc::channel(); /// - /// If called on a zero-capacity channel, this method will wait for a send operation to appear - /// on the other side of the channel. + /// thread::spawn(move || { + /// send.send('a').unwrap(); + /// }); + /// + /// assert_eq!( + /// recv.recv_deadline(Instant::now() + Duration::from_millis(400)), + /// Ok('a') + /// ); + /// ``` + /// + /// Receiving an error upon reaching deadline: + /// + /// ```no_run + /// #![feature(mpmc_channel)] + /// + /// use std::thread; + /// use std::time::{Duration, Instant}; + /// use std::sync::mpmc; + /// + /// let (send, recv) = mpmc::channel(); + /// + /// thread::spawn(move || { + /// thread::sleep(Duration::from_millis(800)); + /// send.send('a').unwrap(); + /// }); + /// + /// assert_eq!( + /// recv.recv_deadline(Instant::now() + Duration::from_millis(400)), + /// Err(mpmc::RecvTimeoutError::Timeout) + /// ); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn recv_deadline(&self, deadline: Instant) -> Result { match &self.flavor { ReceiverFlavor::Array(chan) => chan.recv(Some(deadline)), @@ -342,16 +1110,77 @@ impl Receiver { ReceiverFlavor::Zero(chan) => chan.recv(Some(deadline)), } } + + /// Returns an iterator that will attempt to yield all pending values. + /// It will return `None` if there are no more pending values or if the + /// channel has hung up. The iterator will never [`panic!`] or block the + /// user by waiting for values. + /// + /// # Examples + /// + /// ```no_run + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc::channel; + /// use std::thread; + /// use std::time::Duration; + /// + /// let (sender, receiver) = channel(); + /// + /// // nothing is in the buffer yet + /// assert!(receiver.try_iter().next().is_none()); + /// + /// thread::spawn(move || { + /// thread::sleep(Duration::from_secs(1)); + /// sender.send(1).unwrap(); + /// sender.send(2).unwrap(); + /// sender.send(3).unwrap(); + /// }); + /// + /// // nothing is in the buffer yet + /// assert!(receiver.try_iter().next().is_none()); + /// + /// // block for two seconds + /// thread::sleep(Duration::from_secs(2)); + /// + /// let mut iter = receiver.try_iter(); + /// assert_eq!(iter.next(), Some(1)); + /// assert_eq!(iter.next(), Some(2)); + /// assert_eq!(iter.next(), Some(3)); + /// assert_eq!(iter.next(), None); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] + pub fn try_iter(&self) -> TryIter<'_, T> { + TryIter { rx: self } + } } -// The methods below are not used by `sync::mpsc`, but -// are useful and we'll likely want to expose them -// eventually -#[allow(unused)] impl Receiver { /// Returns `true` if the channel is empty. /// /// Note: Zero-capacity channels are always empty. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc; + /// use std::thread; + /// + /// let (send, recv) = mpmc::channel(); + /// + /// assert!(recv.is_empty()); + /// + /// let handle = thread::spawn(move || { + /// send.send(1u8).unwrap(); + /// }); + /// + /// handle.join().unwrap(); + /// + /// assert!(!recv.is_empty()); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn is_empty(&self) -> bool { match &self.flavor { ReceiverFlavor::Array(chan) => chan.is_empty(), @@ -363,6 +1192,28 @@ impl Receiver { /// Returns `true` if the channel is full. /// /// Note: Zero-capacity channels are always full. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc; + /// use std::thread; + /// + /// let (send, recv) = mpmc::sync_channel(1); + /// + /// assert!(!recv.is_full()); + /// + /// let handle = thread::spawn(move || { + /// send.send(1u8).unwrap(); + /// }); + /// + /// handle.join().unwrap(); + /// + /// assert!(recv.is_full()); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn is_full(&self) -> bool { match &self.flavor { ReceiverFlavor::Array(chan) => chan.is_full(), @@ -372,6 +1223,28 @@ impl Receiver { } /// Returns the number of messages in the channel. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc; + /// use std::thread; + /// + /// let (send, recv) = mpmc::channel(); + /// + /// assert_eq!(recv.len(), 0); + /// + /// let handle = thread::spawn(move || { + /// send.send(1u8).unwrap(); + /// }); + /// + /// handle.join().unwrap(); + /// + /// assert_eq!(recv.len(), 1); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn len(&self) -> usize { match &self.flavor { ReceiverFlavor::Array(chan) => chan.len(), @@ -381,6 +1254,28 @@ impl Receiver { } /// If the channel is bounded, returns its capacity. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc; + /// use std::thread; + /// + /// let (send, recv) = mpmc::sync_channel(3); + /// + /// assert_eq!(recv.capacity(), Some(3)); + /// + /// let handle = thread::spawn(move || { + /// send.send(1u8).unwrap(); + /// }); + /// + /// handle.join().unwrap(); + /// + /// assert_eq!(recv.capacity(), Some(3)); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn capacity(&self) -> Option { match &self.flavor { ReceiverFlavor::Array(chan) => chan.capacity(), @@ -390,6 +1285,21 @@ impl Receiver { } /// Returns `true` if receivers belong to the same channel. + /// + /// # Examples + /// + /// ``` + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc; + /// + /// let (_, rx1) = mpmc::channel::(); + /// let (_, rx2) = mpmc::channel::(); + /// + /// assert!(rx1.same_channel(&rx1)); + /// assert!(!rx1.same_channel(&rx2)); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] pub fn same_channel(&self, other: &Receiver) -> bool { match (&self.flavor, &other.flavor) { (ReceiverFlavor::Array(a), ReceiverFlavor::Array(b)) => a == b, @@ -398,8 +1308,39 @@ impl Receiver { _ => false, } } + + /// Returns an iterator that will block waiting for messages, but never + /// [`panic!`]. It will return [`None`] when the channel has hung up. + /// + /// # Examples + /// + /// ```rust + /// #![feature(mpmc_channel)] + /// + /// use std::sync::mpmc::channel; + /// use std::thread; + /// + /// let (send, recv) = channel(); + /// + /// thread::spawn(move || { + /// send.send(1).unwrap(); + /// send.send(2).unwrap(); + /// send.send(3).unwrap(); + /// }); + /// + /// let mut iter = recv.iter(); + /// assert_eq!(iter.next(), Some(1)); + /// assert_eq!(iter.next(), Some(2)); + /// assert_eq!(iter.next(), Some(3)); + /// assert_eq!(iter.next(), None); + /// ``` + #[unstable(feature = "mpmc_channel", issue = "126840")] + pub fn iter(&self) -> Iter<'_, T> { + Iter { rx: self } + } } +#[unstable(feature = "mpmc_channel", issue = "126840")] impl Drop for Receiver { fn drop(&mut self) { unsafe { @@ -412,6 +1353,7 @@ impl Drop for Receiver { } } +#[unstable(feature = "mpmc_channel", issue = "126840")] impl Clone for Receiver { fn clone(&self) -> Self { let flavor = match &self.flavor { @@ -424,6 +1366,7 @@ impl Clone for Receiver { } } +#[unstable(feature = "mpmc_channel", issue = "126840")] impl fmt::Debug for Receiver { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { f.pad("Receiver { .. }") diff --git a/library/std/src/sync/mpmc/tests.rs b/library/std/src/sync/mpmc/tests.rs new file mode 100644 index 0000000000000..ab14050df6c98 --- /dev/null +++ b/library/std/src/sync/mpmc/tests.rs @@ -0,0 +1,728 @@ +use super::*; +use crate::{env, thread}; + +pub fn stress_factor() -> usize { + match env::var("RUST_TEST_STRESS") { + Ok(val) => val.parse().unwrap(), + Err(..) => 1, + } +} + +#[test] +fn smoke() { + let (tx, rx) = channel::(); + tx.send(1).unwrap(); + assert_eq!(rx.recv().unwrap(), 1); +} + +#[test] +fn drop_full() { + let (tx, _rx) = channel::>(); + tx.send(Box::new(1)).unwrap(); +} + +#[test] +fn drop_full_shared() { + let (tx, _rx) = channel::>(); + drop(tx.clone()); + drop(tx.clone()); + tx.send(Box::new(1)).unwrap(); +} + +#[test] +fn smoke_shared() { + let (tx, rx) = channel::(); + tx.send(1).unwrap(); + assert_eq!(rx.recv().unwrap(), 1); + let tx = tx.clone(); + tx.send(1).unwrap(); + assert_eq!(rx.recv().unwrap(), 1); +} + +#[test] +fn smoke_threads() { + let (tx, rx) = channel::(); + let t1 = thread::spawn(move || { + for i in 0..2 { + tx.send(i).unwrap(); + } + }); + let t2 = thread::spawn(move || { + assert_eq!(rx.recv().unwrap(), 0); + assert_eq!(rx.recv().unwrap(), 1); + }); + t1.join().unwrap(); + t2.join().unwrap(); +} + +#[test] +fn smoke_port_gone() { + let (tx, rx) = channel::(); + drop(rx); + assert!(tx.send(1).is_err()); +} + +#[test] +fn smoke_shared_port_gone() { + let (tx, rx) = channel::(); + drop(rx); + assert!(tx.send(1).is_err()) +} + +#[test] +fn smoke_shared_port_gone2() { + let (tx, rx) = channel::(); + drop(rx); + let tx2 = tx.clone(); + drop(tx); + assert!(tx2.send(1).is_err()); +} + +#[test] +fn port_gone_concurrent() { + let (tx, rx) = channel::(); + let _t = thread::spawn(move || { + rx.recv().unwrap(); + }); + while tx.send(1).is_ok() {} +} + +#[test] +fn port_gone_concurrent_shared() { + let (tx, rx) = channel::(); + let tx2 = tx.clone(); + let _t = thread::spawn(move || { + rx.recv().unwrap(); + }); + while tx.send(1).is_ok() && tx2.send(1).is_ok() {} +} + +#[test] +fn smoke_chan_gone() { + let (tx, rx) = channel::(); + drop(tx); + assert!(rx.recv().is_err()); +} + +#[test] +fn smoke_chan_gone_shared() { + let (tx, rx) = channel::<()>(); + let tx2 = tx.clone(); + drop(tx); + drop(tx2); + assert!(rx.recv().is_err()); +} + +#[test] +fn chan_gone_concurrent() { + let (tx, rx) = channel::(); + let _t = thread::spawn(move || { + tx.send(1).unwrap(); + tx.send(1).unwrap(); + }); + while rx.recv().is_ok() {} +} + +#[test] +fn stress() { + let count = if cfg!(miri) { 100 } else { 10000 }; + let (tx, rx) = channel::(); + let t = thread::spawn(move || { + for _ in 0..count { + tx.send(1).unwrap(); + } + }); + for _ in 0..count { + assert_eq!(rx.recv().unwrap(), 1); + } + t.join().ok().expect("thread panicked"); +} + +#[test] +fn stress_shared() { + const AMT: u32 = if cfg!(miri) { 100 } else { 10000 }; + const NTHREADS: u32 = 8; + let (tx, rx) = channel::(); + + let t = thread::spawn(move || { + for _ in 0..AMT * NTHREADS { + assert_eq!(rx.recv().unwrap(), 1); + } + match rx.try_recv() { + Ok(..) => panic!(), + _ => {} + } + }); + + for _ in 0..NTHREADS { + let tx = tx.clone(); + thread::spawn(move || { + for _ in 0..AMT { + tx.send(1).unwrap(); + } + }); + } + drop(tx); + t.join().ok().expect("thread panicked"); +} + +#[test] +fn send_from_outside_runtime() { + let (tx1, rx1) = channel::<()>(); + let (tx2, rx2) = channel::(); + let t1 = thread::spawn(move || { + tx1.send(()).unwrap(); + for _ in 0..40 { + assert_eq!(rx2.recv().unwrap(), 1); + } + }); + rx1.recv().unwrap(); + let t2 = thread::spawn(move || { + for _ in 0..40 { + tx2.send(1).unwrap(); + } + }); + t1.join().ok().expect("thread panicked"); + t2.join().ok().expect("thread panicked"); +} + +#[test] +fn recv_from_outside_runtime() { + let (tx, rx) = channel::(); + let t = thread::spawn(move || { + for _ in 0..40 { + assert_eq!(rx.recv().unwrap(), 1); + } + }); + for _ in 0..40 { + tx.send(1).unwrap(); + } + t.join().ok().expect("thread panicked"); +} + +#[test] +fn no_runtime() { + let (tx1, rx1) = channel::(); + let (tx2, rx2) = channel::(); + let t1 = thread::spawn(move || { + assert_eq!(rx1.recv().unwrap(), 1); + tx2.send(2).unwrap(); + }); + let t2 = thread::spawn(move || { + tx1.send(1).unwrap(); + assert_eq!(rx2.recv().unwrap(), 2); + }); + t1.join().ok().expect("thread panicked"); + t2.join().ok().expect("thread panicked"); +} + +#[test] +fn oneshot_single_thread_close_port_first() { + // Simple test of closing without sending + let (_tx, rx) = channel::(); + drop(rx); +} + +#[test] +fn oneshot_single_thread_close_chan_first() { + // Simple test of closing without sending + let (tx, _rx) = channel::(); + drop(tx); +} + +#[test] +fn oneshot_single_thread_send_port_close() { + // Testing that the sender cleans up the payload if receiver is closed + let (tx, rx) = channel::>(); + drop(rx); + assert!(tx.send(Box::new(0)).is_err()); +} + +#[test] +fn oneshot_single_thread_recv_chan_close() { + // Receiving on a closed chan will panic + let res = thread::spawn(move || { + let (tx, rx) = channel::(); + drop(tx); + rx.recv().unwrap(); + }) + .join(); + // What is our res? + assert!(res.is_err()); +} + +#[test] +fn oneshot_single_thread_send_then_recv() { + let (tx, rx) = channel::>(); + tx.send(Box::new(10)).unwrap(); + assert!(*rx.recv().unwrap() == 10); +} + +#[test] +fn oneshot_single_thread_try_send_open() { + let (tx, rx) = channel::(); + assert!(tx.send(10).is_ok()); + assert!(rx.recv().unwrap() == 10); +} + +#[test] +fn oneshot_single_thread_try_send_closed() { + let (tx, rx) = channel::(); + drop(rx); + assert!(tx.send(10).is_err()); +} + +#[test] +fn oneshot_single_thread_try_recv_open() { + let (tx, rx) = channel::(); + tx.send(10).unwrap(); + assert!(rx.recv() == Ok(10)); +} + +#[test] +fn oneshot_single_thread_try_recv_closed() { + let (tx, rx) = channel::(); + drop(tx); + assert!(rx.recv().is_err()); +} + +#[test] +fn oneshot_single_thread_peek_data() { + let (tx, rx) = channel::(); + assert_eq!(rx.try_recv(), Err(TryRecvError::Empty)); + tx.send(10).unwrap(); + assert_eq!(rx.try_recv(), Ok(10)); +} + +#[test] +fn oneshot_single_thread_peek_close() { + let (tx, rx) = channel::(); + drop(tx); + assert_eq!(rx.try_recv(), Err(TryRecvError::Disconnected)); + assert_eq!(rx.try_recv(), Err(TryRecvError::Disconnected)); +} + +#[test] +fn oneshot_single_thread_peek_open() { + let (_tx, rx) = channel::(); + assert_eq!(rx.try_recv(), Err(TryRecvError::Empty)); +} + +#[test] +fn oneshot_multi_task_recv_then_send() { + let (tx, rx) = channel::>(); + let _t = thread::spawn(move || { + assert!(*rx.recv().unwrap() == 10); + }); + + tx.send(Box::new(10)).unwrap(); +} + +#[test] +fn oneshot_multi_task_recv_then_close() { + let (tx, rx) = channel::>(); + let _t = thread::spawn(move || { + drop(tx); + }); + let res = thread::spawn(move || { + assert!(*rx.recv().unwrap() == 10); + }) + .join(); + assert!(res.is_err()); +} + +#[test] +fn oneshot_multi_thread_close_stress() { + for _ in 0..stress_factor() { + let (tx, rx) = channel::(); + let _t = thread::spawn(move || { + drop(rx); + }); + drop(tx); + } +} + +#[test] +fn oneshot_multi_thread_send_close_stress() { + for _ in 0..stress_factor() { + let (tx, rx) = channel::(); + let _t = thread::spawn(move || { + drop(rx); + }); + let _ = thread::spawn(move || { + tx.send(1).unwrap(); + }) + .join(); + } +} + +#[test] +fn oneshot_multi_thread_recv_close_stress() { + for _ in 0..stress_factor() { + let (tx, rx) = channel::(); + thread::spawn(move || { + let res = thread::spawn(move || { + rx.recv().unwrap(); + }) + .join(); + assert!(res.is_err()); + }); + let _t = thread::spawn(move || { + thread::spawn(move || { + drop(tx); + }); + }); + } +} + +#[test] +fn oneshot_multi_thread_send_recv_stress() { + for _ in 0..stress_factor() { + let (tx, rx) = channel::>(); + let _t = thread::spawn(move || { + tx.send(Box::new(10)).unwrap(); + }); + assert!(*rx.recv().unwrap() == 10); + } +} + +#[test] +fn stream_send_recv_stress() { + for _ in 0..stress_factor() { + let (tx, rx) = channel(); + + send(tx, 0); + recv(rx, 0); + + fn send(tx: Sender>, i: i32) { + if i == 10 { + return; + } + + thread::spawn(move || { + tx.send(Box::new(i)).unwrap(); + send(tx, i + 1); + }); + } + + fn recv(rx: Receiver>, i: i32) { + if i == 10 { + return; + } + + thread::spawn(move || { + assert!(*rx.recv().unwrap() == i); + recv(rx, i + 1); + }); + } + } +} + +#[test] +fn oneshot_single_thread_recv_timeout() { + let (tx, rx) = channel(); + tx.send(()).unwrap(); + assert_eq!(rx.recv_timeout(Duration::from_millis(1)), Ok(())); + assert_eq!(rx.recv_timeout(Duration::from_millis(1)), Err(RecvTimeoutError::Timeout)); + tx.send(()).unwrap(); + assert_eq!(rx.recv_timeout(Duration::from_millis(1)), Ok(())); +} + +#[test] +fn stress_recv_timeout_two_threads() { + let (tx, rx) = channel(); + let stress = stress_factor() + 100; + let timeout = Duration::from_millis(100); + + thread::spawn(move || { + for i in 0..stress { + if i % 2 == 0 { + thread::sleep(timeout * 2); + } + tx.send(1usize).unwrap(); + } + }); + + let mut recv_count = 0; + loop { + match rx.recv_timeout(timeout) { + Ok(n) => { + assert_eq!(n, 1usize); + recv_count += 1; + } + Err(RecvTimeoutError::Timeout) => continue, + Err(RecvTimeoutError::Disconnected) => break, + } + } + + assert_eq!(recv_count, stress); +} + +#[test] +fn recv_timeout_upgrade() { + let (tx, rx) = channel::<()>(); + let timeout = Duration::from_millis(1); + let _tx_clone = tx.clone(); + + let start = Instant::now(); + assert_eq!(rx.recv_timeout(timeout), Err(RecvTimeoutError::Timeout)); + assert!(Instant::now() >= start + timeout); +} + +#[test] +fn stress_recv_timeout_shared() { + let (tx, rx) = channel(); + let stress = stress_factor() + 100; + + for i in 0..stress { + let tx = tx.clone(); + thread::spawn(move || { + thread::sleep(Duration::from_millis(i as u64 * 10)); + tx.send(1usize).unwrap(); + }); + } + + drop(tx); + + let mut recv_count = 0; + loop { + match rx.recv_timeout(Duration::from_millis(10)) { + Ok(n) => { + assert_eq!(n, 1usize); + recv_count += 1; + } + Err(RecvTimeoutError::Timeout) => continue, + Err(RecvTimeoutError::Disconnected) => break, + } + } + + assert_eq!(recv_count, stress); +} + +#[test] +fn very_long_recv_timeout_wont_panic() { + let (tx, rx) = channel::<()>(); + let join_handle = thread::spawn(move || rx.recv_timeout(Duration::from_secs(u64::MAX))); + thread::sleep(Duration::from_secs(1)); + assert!(tx.send(()).is_ok()); + assert_eq!(join_handle.join().unwrap(), Ok(())); +} + +#[test] +fn recv_a_lot() { + let count = if cfg!(miri) { 1000 } else { 10000 }; + // Regression test that we don't run out of stack in scheduler context + let (tx, rx) = channel(); + for _ in 0..count { + tx.send(()).unwrap(); + } + for _ in 0..count { + rx.recv().unwrap(); + } +} + +#[test] +fn shared_recv_timeout() { + let (tx, rx) = channel(); + let total = 5; + for _ in 0..total { + let tx = tx.clone(); + thread::spawn(move || { + tx.send(()).unwrap(); + }); + } + + for _ in 0..total { + rx.recv().unwrap(); + } + + assert_eq!(rx.recv_timeout(Duration::from_millis(1)), Err(RecvTimeoutError::Timeout)); + tx.send(()).unwrap(); + assert_eq!(rx.recv_timeout(Duration::from_millis(1)), Ok(())); +} + +#[test] +fn shared_chan_stress() { + let (tx, rx) = channel(); + let total = stress_factor() + 100; + for _ in 0..total { + let tx = tx.clone(); + thread::spawn(move || { + tx.send(()).unwrap(); + }); + } + + for _ in 0..total { + rx.recv().unwrap(); + } +} + +#[test] +fn test_nested_recv_iter() { + let (tx, rx) = channel::(); + let (total_tx, total_rx) = channel::(); + + let _t = thread::spawn(move || { + let mut acc = 0; + for x in rx.iter() { + acc += x; + } + total_tx.send(acc).unwrap(); + }); + + tx.send(3).unwrap(); + tx.send(1).unwrap(); + tx.send(2).unwrap(); + drop(tx); + assert_eq!(total_rx.recv().unwrap(), 6); +} + +#[test] +fn test_recv_iter_break() { + let (tx, rx) = channel::(); + let (count_tx, count_rx) = channel(); + + let _t = thread::spawn(move || { + let mut count = 0; + for x in rx.iter() { + if count >= 3 { + break; + } else { + count += x; + } + } + count_tx.send(count).unwrap(); + }); + + tx.send(2).unwrap(); + tx.send(2).unwrap(); + tx.send(2).unwrap(); + let _ = tx.send(2); + drop(tx); + assert_eq!(count_rx.recv().unwrap(), 4); +} + +#[test] +fn test_recv_try_iter() { + let (request_tx, request_rx) = channel(); + let (response_tx, response_rx) = channel(); + + // Request `x`s until we have `6`. + let t = thread::spawn(move || { + let mut count = 0; + loop { + for x in response_rx.try_iter() { + count += x; + if count == 6 { + return count; + } + } + request_tx.send(()).unwrap(); + } + }); + + for _ in request_rx.iter() { + if response_tx.send(2).is_err() { + break; + } + } + + assert_eq!(t.join().unwrap(), 6); +} + +#[test] +fn test_recv_into_iter_owned() { + let mut iter = { + let (tx, rx) = channel::(); + tx.send(1).unwrap(); + tx.send(2).unwrap(); + + rx.into_iter() + }; + assert_eq!(iter.next().unwrap(), 1); + assert_eq!(iter.next().unwrap(), 2); + assert_eq!(iter.next().is_none(), true); +} + +#[test] +fn test_recv_into_iter_borrowed() { + let (tx, rx) = channel::(); + tx.send(1).unwrap(); + tx.send(2).unwrap(); + drop(tx); + let mut iter = (&rx).into_iter(); + assert_eq!(iter.next().unwrap(), 1); + assert_eq!(iter.next().unwrap(), 2); + assert_eq!(iter.next().is_none(), true); +} + +#[test] +fn try_recv_states() { + let (tx1, rx1) = channel::(); + let (tx2, rx2) = channel::<()>(); + let (tx3, rx3) = channel::<()>(); + let _t = thread::spawn(move || { + rx2.recv().unwrap(); + tx1.send(1).unwrap(); + tx3.send(()).unwrap(); + rx2.recv().unwrap(); + drop(tx1); + tx3.send(()).unwrap(); + }); + + assert_eq!(rx1.try_recv(), Err(TryRecvError::Empty)); + tx2.send(()).unwrap(); + rx3.recv().unwrap(); + assert_eq!(rx1.try_recv(), Ok(1)); + assert_eq!(rx1.try_recv(), Err(TryRecvError::Empty)); + tx2.send(()).unwrap(); + rx3.recv().unwrap(); + assert_eq!(rx1.try_recv(), Err(TryRecvError::Disconnected)); +} + +// This bug used to end up in a livelock inside of the Receiver destructor +// because the internal state of the Shared packet was corrupted +#[test] +fn destroy_upgraded_shared_port_when_sender_still_active() { + let (tx, rx) = channel(); + let (tx2, rx2) = channel(); + let _t = thread::spawn(move || { + rx.recv().unwrap(); // wait on a oneshot + drop(rx); // destroy a shared + tx2.send(()).unwrap(); + }); + // make sure the other thread has gone to sleep + for _ in 0..5000 { + thread::yield_now(); + } + + // upgrade to a shared chan and send a message + let t = tx.clone(); + drop(tx); + t.send(()).unwrap(); + + // wait for the child thread to exit before we exit + rx2.recv().unwrap(); +} + +#[test] +fn issue_32114() { + let (tx, _) = channel(); + let _ = tx.send(123); + assert_eq!(tx.send(123), Err(SendError(123))); +} + +#[test] +fn issue_39364() { + let (tx, rx) = channel::<()>(); + let t = thread::spawn(move || { + thread::sleep(Duration::from_millis(300)); + let _ = tx.clone(); + // Don't drop; hand back to caller. + tx + }); + + let _ = rx.recv_timeout(Duration::from_millis(500)); + let _tx = t.join().unwrap(); // delay dropping until end of test + let _ = rx.recv_timeout(Duration::from_millis(500)); +} diff --git a/library/std/src/sync/mpmc/waker.rs b/library/std/src/sync/mpmc/waker.rs index fb877887f9c9d..1895466f95d45 100644 --- a/library/std/src/sync/mpmc/waker.rs +++ b/library/std/src/sync/mpmc/waker.rs @@ -3,8 +3,8 @@ use super::context::Context; use super::select::{Operation, Selected}; use crate::ptr; -use crate::sync::atomic::{AtomicBool, Ordering}; use crate::sync::Mutex; +use crate::sync::atomic::{AtomicBool, Ordering}; /// Represents a thread blocked on a specific channel operation. pub(crate) struct Entry { diff --git a/library/std/src/sync/mpmc/zero.rs b/library/std/src/sync/mpmc/zero.rs index 2b82eeda3d5fb..446881291e68e 100644 --- a/library/std/src/sync/mpmc/zero.rs +++ b/library/std/src/sync/mpmc/zero.rs @@ -9,8 +9,8 @@ use super::utils::Backoff; use super::waker::Waker; use crate::cell::UnsafeCell; use crate::marker::PhantomData; -use crate::sync::atomic::{AtomicBool, Ordering}; use crate::sync::Mutex; +use crate::sync::atomic::{AtomicBool, Ordering}; use crate::time::Instant; use crate::{fmt, ptr}; @@ -185,11 +185,7 @@ impl Channel { // Prepare for blocking until a receiver wakes us up. let oper = Operation::hook(token); let mut packet = Packet::::message_on_stack(msg); - inner.senders.register_with_packet( - oper, - core::ptr::addr_of_mut!(packet) as *mut (), - cx, - ); + inner.senders.register_with_packet(oper, (&raw mut packet) as *mut (), cx); inner.receivers.notify(); drop(inner); @@ -256,11 +252,7 @@ impl Channel { // Prepare for blocking until a sender wakes us up. let oper = Operation::hook(token); let mut packet = Packet::::empty_on_stack(); - inner.receivers.register_with_packet( - oper, - core::ptr::addr_of_mut!(packet) as *mut (), - cx, - ); + inner.receivers.register_with_packet(oper, (&raw mut packet) as *mut (), cx); inner.senders.notify(); drop(inner); diff --git a/library/std/src/sync/mpsc/mod.rs b/library/std/src/sync/mpsc/mod.rs index 26d5b9515a244..83a93a0636956 100644 --- a/library/std/src/sync/mpsc/mod.rs +++ b/library/std/src/sync/mpsc/mod.rs @@ -137,10 +137,10 @@ #![stable(feature = "rust1", since = "1.0.0")] -#[cfg(all(test, not(target_os = "emscripten")))] +#[cfg(all(test, not(any(target_os = "emscripten", target_os = "wasi"))))] mod tests; -#[cfg(all(test, not(target_os = "emscripten")))] +#[cfg(all(test, not(any(target_os = "emscripten", target_os = "wasi"))))] mod sync_tests; // MPSC channels are built as a wrapper around MPMC channels, which diff --git a/library/std/src/sync/mutex.rs b/library/std/src/sync/mutex.rs index d417034f5af85..fe2aca031a248 100644 --- a/library/std/src/sync/mutex.rs +++ b/library/std/src/sync/mutex.rs @@ -1,4 +1,4 @@ -#[cfg(all(test, not(target_os = "emscripten")))] +#[cfg(all(test, not(any(target_os = "emscripten", target_os = "wasi"))))] mod tests; use crate::cell::UnsafeCell; @@ -7,7 +7,7 @@ use crate::marker::PhantomData; use crate::mem::ManuallyDrop; use crate::ops::{Deref, DerefMut}; use crate::ptr::NonNull; -use crate::sync::{poison, LockResult, TryLockError, TryLockResult}; +use crate::sync::{LockResult, TryLockError, TryLockResult, poison}; use crate::sys::sync as sys; /// A mutual exclusion primitive useful for protecting shared data diff --git a/library/std/src/sync/once.rs b/library/std/src/sync/once.rs index bf595fdea2d25..27db4b634fb28 100644 --- a/library/std/src/sync/once.rs +++ b/library/std/src/sync/once.rs @@ -3,7 +3,7 @@ //! This primitive is meant to be used to run one-time initialization. An //! example use case would be for initializing an FFI library. -#[cfg(all(test, not(target_os = "emscripten")))] +#[cfg(all(test, not(any(target_os = "emscripten", target_os = "wasi"))))] mod tests; use crate::fmt; @@ -288,7 +288,7 @@ impl Once { /// /// If this [`Once`] has been poisoned because an initialization closure has /// panicked, this method will also panic. Use [`wait_force`](Self::wait_force) - /// if this behaviour is not desired. + /// if this behavior is not desired. #[unstable(feature = "once_wait", issue = "127527")] pub fn wait(&self) { if !self.inner.is_completed() { @@ -314,6 +314,16 @@ impl Once { pub(crate) fn state(&mut self) -> ExclusiveState { self.inner.state() } + + /// Sets current state of the `Once` instance. + /// + /// Since this takes a mutable reference, no initialization can currently + /// be running, so the state must be either "incomplete", "poisoned" or + /// "complete". + #[inline] + pub(crate) fn set_state(&mut self, new_state: ExclusiveState) { + self.inner.set_state(new_state); + } } #[stable(feature = "std_debug", since = "1.16.0")] diff --git a/library/std/src/sync/once_lock.rs b/library/std/src/sync/once_lock.rs index be615a5a8ef37..0ae3cf4df3614 100644 --- a/library/std/src/sync/once_lock.rs +++ b/library/std/src/sync/once_lock.rs @@ -634,6 +634,26 @@ impl From for OnceLock { #[stable(feature = "once_cell", since = "1.70.0")] impl PartialEq for OnceLock { + /// Equality for two `OnceLock`s. + /// + /// Two `OnceLock`s are equal if they either both contain values and their + /// values are equal, or if neither contains a value. + /// + /// # Examples + /// + /// ``` + /// use std::sync::OnceLock; + /// + /// let five = OnceLock::new(); + /// five.set(5).unwrap(); + /// + /// let also_five = OnceLock::new(); + /// also_five.set(5).unwrap(); + /// + /// assert!(five == also_five); + /// + /// assert!(OnceLock::::new() == OnceLock::::new()); + /// ``` #[inline] fn eq(&self, other: &OnceLock) -> bool { self.get() == other.get() diff --git a/library/std/src/sync/once_lock/tests.rs b/library/std/src/sync/once_lock/tests.rs index 176830c6748b2..5113d436c3c99 100644 --- a/library/std/src/sync/once_lock/tests.rs +++ b/library/std/src/sync/once_lock/tests.rs @@ -1,7 +1,7 @@ +use crate::sync::OnceLock; use crate::sync::atomic::AtomicUsize; use crate::sync::atomic::Ordering::SeqCst; use crate::sync::mpsc::channel; -use crate::sync::OnceLock; use crate::{panic, thread}; fn spawn_and_wait(f: impl FnOnce() -> R + Send + 'static) -> R { @@ -9,7 +9,7 @@ fn spawn_and_wait(f: impl FnOnce() -> R + Send + 'static) -> } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn sync_once_cell() { static ONCE_CELL: OnceLock = OnceLock::new(); @@ -43,7 +43,7 @@ fn sync_once_cell_get_unchecked() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn sync_once_cell_drop() { static DROP_CNT: AtomicUsize = AtomicUsize::new(0); struct Dropper; @@ -81,6 +81,7 @@ fn clone() { } #[test] +#[cfg_attr(not(panic = "unwind"), ignore = "test requires unwinding support")] fn get_or_try_init() { let cell: OnceLock = OnceLock::new(); assert!(cell.get().is_none()); @@ -154,7 +155,7 @@ fn eval_once_macro() { } #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads fn sync_once_cell_does_not_leak_partially_constructed_boxes() { static ONCE_CELL: OnceLock = OnceLock::new(); diff --git a/library/std/src/sync/reentrant_lock.rs b/library/std/src/sync/reentrant_lock.rs index 0b23681e90726..0140e0d21299f 100644 --- a/library/std/src/sync/reentrant_lock.rs +++ b/library/std/src/sync/reentrant_lock.rs @@ -1,4 +1,4 @@ -#[cfg(all(test, not(target_os = "emscripten")))] +#[cfg(all(test, not(any(target_os = "emscripten", target_os = "wasi"))))] mod tests; use cfg_if::cfg_if; @@ -8,7 +8,7 @@ use crate::fmt; use crate::ops::Deref; use crate::panic::{RefUnwindSafe, UnwindSafe}; use crate::sys::sync as sys; -use crate::thread::{current_id, ThreadId}; +use crate::thread::{ThreadId, current_id}; /// A re-entrant mutual exclusion lock /// diff --git a/library/std/src/sync/rwlock.rs b/library/std/src/sync/rwlock.rs index d995a16e056d6..da2da6f9dfc53 100644 --- a/library/std/src/sync/rwlock.rs +++ b/library/std/src/sync/rwlock.rs @@ -1,4 +1,4 @@ -#[cfg(all(test, not(target_os = "emscripten")))] +#[cfg(all(test, not(any(target_os = "emscripten", target_os = "wasi"))))] mod tests; use crate::cell::UnsafeCell; @@ -7,7 +7,7 @@ use crate::marker::PhantomData; use crate::mem::ManuallyDrop; use crate::ops::{Deref, DerefMut}; use crate::ptr::NonNull; -use crate::sync::{poison, LockResult, TryLockError, TryLockResult}; +use crate::sync::{LockResult, TryLockError, TryLockResult, poison}; use crate::sys::sync as sys; /// A reader-writer lock diff --git a/library/std/src/sys/alloc/solid.rs b/library/std/src/sys/alloc/solid.rs index abb534a1c5cf4..47cfa2eb1162b 100644 --- a/library/std/src/sys/alloc/solid.rs +++ b/library/std/src/sys/alloc/solid.rs @@ -1,4 +1,4 @@ -use super::{realloc_fallback, MIN_ALIGN}; +use super::{MIN_ALIGN, realloc_fallback}; use crate::alloc::{GlobalAlloc, Layout, System}; #[stable(feature = "alloc_system_type", since = "1.28.0")] diff --git a/library/std/src/sys/alloc/unix.rs b/library/std/src/sys/alloc/unix.rs index 46ed7de7162f8..1af9d76629014 100644 --- a/library/std/src/sys/alloc/unix.rs +++ b/library/std/src/sys/alloc/unix.rs @@ -1,4 +1,4 @@ -use super::{realloc_fallback, MIN_ALIGN}; +use super::{MIN_ALIGN, realloc_fallback}; use crate::alloc::{GlobalAlloc, Layout, System}; use crate::ptr; @@ -71,6 +71,7 @@ cfg_if::cfg_if! { } } else { #[inline] + #[cfg_attr(target_os = "vxworks", allow(unused_unsafe))] unsafe fn aligned_malloc(layout: &Layout) -> *mut u8 { let mut out = ptr::null_mut(); // We prefer posix_memalign over aligned_alloc since it is more widely available, and diff --git a/library/std/src/sys/alloc/wasm.rs b/library/std/src/sys/alloc/wasm.rs index ef9d753d7f86c..a308fafc68b15 100644 --- a/library/std/src/sys/alloc/wasm.rs +++ b/library/std/src/sys/alloc/wasm.rs @@ -16,6 +16,9 @@ //! The crate itself provides a global allocator which on wasm has no //! synchronization as there are no threads! +// FIXME(static_mut_refs): Do not allow `static_mut_refs` lint +#![allow(static_mut_refs)] + use crate::alloc::{GlobalAlloc, Layout, System}; static mut DLMALLOC: dlmalloc::Dlmalloc = dlmalloc::Dlmalloc::new(); diff --git a/library/std/src/sys/alloc/windows.rs b/library/std/src/sys/alloc/windows.rs index e91956966aa73..a77dda6e817ba 100644 --- a/library/std/src/sys/alloc/windows.rs +++ b/library/std/src/sys/alloc/windows.rs @@ -1,4 +1,4 @@ -use super::{realloc_fallback, MIN_ALIGN}; +use super::{MIN_ALIGN, realloc_fallback}; use crate::alloc::{GlobalAlloc, Layout, System}; use crate::ffi::c_void; use crate::mem::MaybeUninit; diff --git a/library/std/src/sys/alloc/xous.rs b/library/std/src/sys/alloc/xous.rs index 9ea43445d0206..321d30e0b11b2 100644 --- a/library/std/src/sys/alloc/xous.rs +++ b/library/std/src/sys/alloc/xous.rs @@ -1,3 +1,6 @@ +// FIXME(static_mut_refs): Do not allow `static_mut_refs` lint +#![allow(static_mut_refs)] + use crate::alloc::{GlobalAlloc, Layout, System}; #[cfg(not(test))] diff --git a/library/std/src/sys/mod.rs b/library/std/src/sys/mod.rs index 1ef17dd530fd2..f17dd47decedc 100644 --- a/library/std/src/sys/mod.rs +++ b/library/std/src/sys/mod.rs @@ -14,6 +14,7 @@ pub mod cmath; pub mod exit_guard; pub mod os_str; pub mod path; +pub mod random; pub mod sync; pub mod thread_local; diff --git a/library/std/src/sys/os_str/bytes.rs b/library/std/src/sys/os_str/bytes.rs index 992767211d083..8e0609fe48c53 100644 --- a/library/std/src/sys/os_str/bytes.rs +++ b/library/std/src/sys/os_str/bytes.rs @@ -2,7 +2,6 @@ //! systems: just a `Vec`/`[u8]`. use core::clone::CloneToUninit; -use core::ptr::addr_of_mut; use crate::borrow::Cow; use crate::collections::TryReserveError; @@ -355,6 +354,6 @@ unsafe impl CloneToUninit for Slice { #[cfg_attr(debug_assertions, track_caller)] unsafe fn clone_to_uninit(&self, dst: *mut Self) { // SAFETY: we're just a wrapper around [u8] - unsafe { self.inner.clone_to_uninit(addr_of_mut!((*dst).inner)) } + unsafe { self.inner.clone_to_uninit(&raw mut (*dst).inner) } } } diff --git a/library/std/src/sys/os_str/wtf8.rs b/library/std/src/sys/os_str/wtf8.rs index 433237aa6e7bf..b3834388df68a 100644 --- a/library/std/src/sys/os_str/wtf8.rs +++ b/library/std/src/sys/os_str/wtf8.rs @@ -1,13 +1,12 @@ //! The underlying OsString/OsStr implementation on Windows is a //! wrapper around the "WTF-8" encoding; see the `wtf8` module for more. use core::clone::CloneToUninit; -use core::ptr::addr_of_mut; use crate::borrow::Cow; use crate::collections::TryReserveError; use crate::rc::Rc; use crate::sync::Arc; -use crate::sys_common::wtf8::{check_utf8_boundary, Wtf8, Wtf8Buf}; +use crate::sys_common::wtf8::{Wtf8, Wtf8Buf, check_utf8_boundary}; use crate::sys_common::{AsInner, FromInner, IntoInner}; use crate::{fmt, mem}; @@ -278,6 +277,6 @@ unsafe impl CloneToUninit for Slice { #[cfg_attr(debug_assertions, track_caller)] unsafe fn clone_to_uninit(&self, dst: *mut Self) { // SAFETY: we're just a wrapper around Wtf8 - unsafe { self.inner.clone_to_uninit(addr_of_mut!((*dst).inner)) } + unsafe { self.inner.clone_to_uninit(&raw mut (*dst).inner) } } } diff --git a/library/std/src/sys/pal/hermit/args.rs b/library/std/src/sys/pal/hermit/args.rs index 51afe3434aedc..4402426027730 100644 --- a/library/std/src/sys/pal/hermit/args.rs +++ b/library/std/src/sys/pal/hermit/args.rs @@ -1,4 +1,4 @@ -use crate::ffi::{c_char, CStr, OsString}; +use crate::ffi::{CStr, OsString, c_char}; use crate::os::hermit::ffi::OsStringExt; use crate::sync::atomic::Ordering::{Acquire, Relaxed, Release}; use crate::sync::atomic::{AtomicIsize, AtomicPtr}; diff --git a/library/std/src/sys/pal/hermit/fs.rs b/library/std/src/sys/pal/hermit/fs.rs index aaf1a044d0613..70f6981f7175b 100644 --- a/library/std/src/sys/pal/hermit/fs.rs +++ b/library/std/src/sys/pal/hermit/fs.rs @@ -1,7 +1,7 @@ use super::fd::FileDesc; use super::hermit_abi::{ - self, dirent64, stat as stat_struct, DT_DIR, DT_LNK, DT_REG, DT_UNKNOWN, O_APPEND, O_CREAT, - O_DIRECTORY, O_EXCL, O_RDONLY, O_RDWR, O_TRUNC, O_WRONLY, S_IFDIR, S_IFLNK, S_IFMT, S_IFREG, + self, DT_DIR, DT_LNK, DT_REG, DT_UNKNOWN, O_APPEND, O_CREAT, O_DIRECTORY, O_EXCL, O_RDONLY, + O_RDWR, O_TRUNC, O_WRONLY, S_IFDIR, S_IFLNK, S_IFMT, S_IFREG, dirent64, stat as stat_struct, }; use crate::ffi::{CStr, OsStr, OsString}; use crate::io::{self, BorrowedCursor, Error, ErrorKind, IoSlice, IoSliceMut, SeekFrom}; diff --git a/library/std/src/sys/pal/hermit/futex.rs b/library/std/src/sys/pal/hermit/futex.rs index 21c5facd52fbd..670383b45aca9 100644 --- a/library/std/src/sys/pal/hermit/futex.rs +++ b/library/std/src/sys/pal/hermit/futex.rs @@ -3,9 +3,14 @@ use crate::ptr::null; use crate::sync::atomic::AtomicU32; use crate::time::Duration; +/// An atomic for use as a futex that is at least 32-bits but may be larger +pub type Futex = AtomicU32; +/// Must be the underlying type of Futex +pub type Primitive = u32; + /// An atomic for use as a futex that is at least 8-bits but may be larger. -pub type SmallAtomic = AtomicU32; -/// Must be the underlying type of SmallAtomic +pub type SmallFutex = AtomicU32; +/// Must be the underlying type of SmallFutex pub type SmallPrimitive = u32; pub fn futex_wait(futex: &AtomicU32, expected: u32, timeout: Option) -> bool { diff --git a/library/std/src/sys/pal/hermit/mod.rs b/library/std/src/sys/pal/hermit/mod.rs index 1f2e5d9469f5c..b62afb40a615f 100644 --- a/library/std/src/sys/pal/hermit/mod.rs +++ b/library/std/src/sys/pal/hermit/mod.rs @@ -52,20 +52,6 @@ pub fn abort_internal() -> ! { unsafe { hermit_abi::abort() } } -pub fn hashmap_random_keys() -> (u64, u64) { - let mut buf = [0; 16]; - let mut slice = &mut buf[..]; - while !slice.is_empty() { - let res = cvt(unsafe { hermit_abi::read_entropy(slice.as_mut_ptr(), slice.len(), 0) }) - .expect("failed to generate random hashmap keys"); - slice = &mut slice[res as usize..]; - } - - let key1 = buf[..8].try_into().unwrap(); - let key2 = buf[8..].try_into().unwrap(); - (u64::from_ne_bytes(key1), u64::from_ne_bytes(key2)) -} - // This function is needed by the panic runtime. The symbol is named in // pre-link args for the target specification, so keep that in sync. #[cfg(not(test))] @@ -106,7 +92,11 @@ pub unsafe extern "C" fn runtime_entry( unsafe { crate::sys::thread_local::destructors::run(); } - unsafe { hermit_abi::exit(result) } + crate::rt::thread_cleanup(); + + unsafe { + hermit_abi::exit(result); + } } #[inline] diff --git a/library/std/src/sys/pal/hermit/net.rs b/library/std/src/sys/pal/hermit/net.rs index 416469c003738..d9baa091a2321 100644 --- a/library/std/src/sys/pal/hermit/net.rs +++ b/library/std/src/sys/pal/hermit/net.rs @@ -192,7 +192,7 @@ impl Socket { buf.as_mut_ptr(), buf.len(), flags, - core::ptr::addr_of_mut!(storage) as *mut _, + (&raw mut storage) as *mut _, &mut addrlen, ) })?; @@ -298,7 +298,7 @@ impl Socket { netc::ioctl( self.as_raw_fd(), netc::FIONBIO, - core::ptr::addr_of_mut!(nonblocking) as *mut core::ffi::c_void, + (&raw mut nonblocking) as *mut core::ffi::c_void, ) }) .map(drop) diff --git a/library/std/src/sys/pal/hermit/thread.rs b/library/std/src/sys/pal/hermit/thread.rs index 4c0c0919f4799..41f2c3e212355 100644 --- a/library/std/src/sys/pal/hermit/thread.rs +++ b/library/std/src/sys/pal/hermit/thread.rs @@ -53,6 +53,7 @@ impl Thread { // run all destructors crate::sys::thread_local::destructors::run(); + crate::rt::thread_cleanup(); } } } diff --git a/library/std/src/sys/pal/hermit/time.rs b/library/std/src/sys/pal/hermit/time.rs index 2c87c4860a27a..e0b6eb76b03af 100644 --- a/library/std/src/sys/pal/hermit/time.rs +++ b/library/std/src/sys/pal/hermit/time.rs @@ -2,7 +2,7 @@ use core::hash::{Hash, Hasher}; -use super::hermit_abi::{self, timespec, CLOCK_MONOTONIC, CLOCK_REALTIME}; +use super::hermit_abi::{self, CLOCK_MONOTONIC, CLOCK_REALTIME, timespec}; use crate::cmp::Ordering; use crate::ops::{Add, AddAssign, Sub, SubAssign}; use crate::time::Duration; @@ -107,8 +107,7 @@ pub struct Instant(Timespec); impl Instant { pub fn now() -> Instant { let mut time: Timespec = Timespec::zero(); - let _ = - unsafe { hermit_abi::clock_gettime(CLOCK_MONOTONIC, core::ptr::addr_of_mut!(time.t)) }; + let _ = unsafe { hermit_abi::clock_gettime(CLOCK_MONOTONIC, &raw mut time.t) }; Instant(time) } @@ -209,8 +208,7 @@ impl SystemTime { pub fn now() -> SystemTime { let mut time: Timespec = Timespec::zero(); - let _ = - unsafe { hermit_abi::clock_gettime(CLOCK_REALTIME, core::ptr::addr_of_mut!(time.t)) }; + let _ = unsafe { hermit_abi::clock_gettime(CLOCK_REALTIME, &raw mut time.t) }; SystemTime(time) } diff --git a/library/std/src/sys/pal/itron/task.rs b/library/std/src/sys/pal/itron/task.rs index 5da0c5917885f..49c420baca2f3 100644 --- a/library/std/src/sys/pal/itron/task.rs +++ b/library/std/src/sys/pal/itron/task.rs @@ -1,5 +1,5 @@ use super::abi; -use super::error::{fail, fail_aborting, ItronError}; +use super::error::{ItronError, fail, fail_aborting}; use crate::mem::MaybeUninit; /// Gets the ID of the task in Running state. Panics on failure. diff --git a/library/std/src/sys/pal/itron/thread.rs b/library/std/src/sys/pal/itron/thread.rs index 01e69afa99e13..04095e1a7cf99 100644 --- a/library/std/src/sys/pal/itron/thread.rs +++ b/library/std/src/sys/pal/itron/thread.rs @@ -1,7 +1,7 @@ //! Thread implementation backed by μITRON tasks. Assumes `acre_tsk` and //! `exd_tsk` are available. -use super::error::{expect_success, expect_success_aborting, ItronError}; +use super::error::{ItronError, expect_success, expect_success_aborting}; use super::time::dur2reltims; use super::{abi, task}; use crate::cell::UnsafeCell; diff --git a/library/std/src/sys/pal/itron/time/tests.rs b/library/std/src/sys/pal/itron/time/tests.rs index d14035d9da49f..28db4f8b6799f 100644 --- a/library/std/src/sys/pal/itron/time/tests.rs +++ b/library/std/src/sys/pal/itron/time/tests.rs @@ -8,26 +8,24 @@ fn reltim2dur(t: u64) -> Duration { fn test_dur2reltims() { assert_eq!(dur2reltims(reltim2dur(0)).collect::>(), vec![]); assert_eq!(dur2reltims(reltim2dur(42)).collect::>(), vec![42]); - assert_eq!( - dur2reltims(reltim2dur(abi::TMAX_RELTIM as u64)).collect::>(), - vec![abi::TMAX_RELTIM] - ); - assert_eq!( - dur2reltims(reltim2dur(abi::TMAX_RELTIM as u64 + 10000)).collect::>(), - vec![abi::TMAX_RELTIM, 10000] - ); + assert_eq!(dur2reltims(reltim2dur(abi::TMAX_RELTIM as u64)).collect::>(), vec![ + abi::TMAX_RELTIM + ]); + assert_eq!(dur2reltims(reltim2dur(abi::TMAX_RELTIM as u64 + 10000)).collect::>(), vec![ + abi::TMAX_RELTIM, + 10000 + ]); } #[test] fn test_dur2tmos() { assert_eq!(dur2tmos(reltim2dur(0)).collect::>(), vec![0]); assert_eq!(dur2tmos(reltim2dur(42)).collect::>(), vec![42]); - assert_eq!( - dur2tmos(reltim2dur(abi::TMAX_RELTIM as u64)).collect::>(), - vec![abi::TMAX_RELTIM] - ); - assert_eq!( - dur2tmos(reltim2dur(abi::TMAX_RELTIM as u64 + 10000)).collect::>(), - vec![abi::TMAX_RELTIM, 10000] - ); + assert_eq!(dur2tmos(reltim2dur(abi::TMAX_RELTIM as u64)).collect::>(), vec![ + abi::TMAX_RELTIM + ]); + assert_eq!(dur2tmos(reltim2dur(abi::TMAX_RELTIM as u64 + 10000)).collect::>(), vec![ + abi::TMAX_RELTIM, + 10000 + ]); } diff --git a/library/std/src/sys/pal/sgx/abi/usercalls/mod.rs b/library/std/src/sys/pal/sgx/abi/usercalls/mod.rs index def1ccdf81ac0..90b9b59471a52 100644 --- a/library/std/src/sys/pal/sgx/abi/usercalls/mod.rs +++ b/library/std/src/sys/pal/sgx/abi/usercalls/mod.rs @@ -1,6 +1,6 @@ use crate::cmp; use crate::io::{Error as IoError, ErrorKind, IoSlice, IoSliceMut, Result as IoResult}; -use crate::sys::rand::rdrand64; +use crate::random::{DefaultRandomSource, Random}; use crate::time::{Duration, Instant}; pub(crate) mod alloc; @@ -164,7 +164,7 @@ pub fn wait(event_mask: u64, mut timeout: u64) -> IoResult { // trusted to ensure accurate timeouts. if let Ok(timeout_signed) = i64::try_from(timeout) { let tenth = timeout_signed / 10; - let deviation = (rdrand64() as i64).checked_rem(tenth).unwrap_or(0); + let deviation = i64::random(&mut DefaultRandomSource).checked_rem(tenth).unwrap_or(0); timeout = timeout_signed.saturating_add(deviation) as _; } } diff --git a/library/std/src/sys/pal/sgx/abi/usercalls/tests.rs b/library/std/src/sys/pal/sgx/abi/usercalls/tests.rs index ef824d35f4a16..5978ae5a0fac9 100644 --- a/library/std/src/sys/pal/sgx/abi/usercalls/tests.rs +++ b/library/std/src/sys/pal/sgx/abi/usercalls/tests.rs @@ -1,4 +1,4 @@ -use super::alloc::{copy_from_userspace, copy_to_userspace, User}; +use super::alloc::{User, copy_from_userspace, copy_to_userspace}; #[test] fn test_copy_to_userspace_function() { diff --git a/library/std/src/sys/pal/sgx/mod.rs b/library/std/src/sys/pal/sgx/mod.rs index 8d29b2ec6193e..586ccd18c2f57 100644 --- a/library/std/src/sys/pal/sgx/mod.rs +++ b/library/std/src/sys/pal/sgx/mod.rs @@ -132,24 +132,6 @@ pub extern "C" fn __rust_abort() { abort_internal(); } -pub mod rand { - pub fn rdrand64() -> u64 { - unsafe { - let mut ret: u64 = 0; - for _ in 0..10 { - if crate::arch::x86_64::_rdrand64_step(&mut ret) == 1 { - return ret; - } - } - rtabort!("Failed to obtain random data"); - } - } -} - -pub fn hashmap_random_keys() -> (u64, u64) { - (self::rand::rdrand64(), self::rand::rdrand64()) -} - pub use crate::sys_common::{AsInner, FromInner, IntoInner}; pub trait TryIntoInner: Sized { diff --git a/library/std/src/sys/pal/sgx/net.rs b/library/std/src/sys/pal/sgx/net.rs index f2e751c51194d..c966886d16344 100644 --- a/library/std/src/sys/pal/sgx/net.rs +++ b/library/std/src/sys/pal/sgx/net.rs @@ -3,7 +3,7 @@ use crate::io::{self, BorrowedCursor, IoSlice, IoSliceMut}; use crate::net::{Ipv4Addr, Ipv6Addr, Shutdown, SocketAddr, ToSocketAddrs}; use crate::sync::Arc; use crate::sys::fd::FileDesc; -use crate::sys::{sgx_ineffective, unsupported, AsInner, FromInner, IntoInner, TryIntoInner}; +use crate::sys::{AsInner, FromInner, IntoInner, TryIntoInner, sgx_ineffective, unsupported}; use crate::time::Duration; use crate::{error, fmt}; @@ -78,9 +78,8 @@ fn io_err_to_addr(result: io::Result<&SocketAddr>) -> io::Result { } } -fn addr_to_sockaddr(addr: &Option) -> io::Result { - addr.as_ref() - .ok_or(io::ErrorKind::AddrNotAvailable)? +fn addr_to_sockaddr(addr: Option<&str>) -> io::Result { + addr.ok_or(io::ErrorKind::AddrNotAvailable)? .to_socket_addrs() // unwrap OK: if an iterator is returned, we're guaranteed to get exactly one entry .map(|mut it| it.next().unwrap()) @@ -161,11 +160,11 @@ impl TcpStream { } pub fn peer_addr(&self) -> io::Result { - addr_to_sockaddr(&self.peer_addr) + addr_to_sockaddr(self.peer_addr.as_deref()) } pub fn socket_addr(&self) -> io::Result { - addr_to_sockaddr(&self.inner.local_addr) + addr_to_sockaddr(self.inner.local_addr.as_deref()) } pub fn shutdown(&self, _: Shutdown) -> io::Result<()> { @@ -255,13 +254,14 @@ impl TcpListener { } pub fn socket_addr(&self) -> io::Result { - addr_to_sockaddr(&self.inner.local_addr) + addr_to_sockaddr(self.inner.local_addr.as_deref()) } pub fn accept(&self) -> io::Result<(TcpStream, SocketAddr)> { let (fd, local_addr, peer_addr) = usercalls::accept_stream(self.inner.inner.raw())?; let peer_addr = Some(peer_addr); - let ret_peer = addr_to_sockaddr(&peer_addr).unwrap_or_else(|_| ([0; 4], 0).into()); + let ret_peer = + addr_to_sockaddr(peer_addr.as_deref()).unwrap_or_else(|_| ([0; 4], 0).into()); Ok((TcpStream { inner: Socket::new(fd, local_addr), peer_addr }, ret_peer)) } diff --git a/library/std/src/sys/pal/sgx/waitqueue/mod.rs b/library/std/src/sys/pal/sgx/waitqueue/mod.rs index bd114523fe80b..41d1413fcdee9 100644 --- a/library/std/src/sys/pal/sgx/waitqueue/mod.rs +++ b/library/std/src/sys/pal/sgx/waitqueue/mod.rs @@ -16,9 +16,9 @@ mod tests; mod spin_mutex; mod unsafe_list; -use fortanix_sgx_abi::{Tcs, EV_UNPARK, WAIT_INDEFINITE}; +use fortanix_sgx_abi::{EV_UNPARK, Tcs, WAIT_INDEFINITE}; -pub use self::spin_mutex::{try_lock_or_false, SpinMutex, SpinMutexGuard}; +pub use self::spin_mutex::{SpinMutex, SpinMutexGuard, try_lock_or_false}; use self::unsafe_list::{UnsafeList, UnsafeListEntry}; use super::abi::{thread, usercalls}; use crate::num::NonZero; diff --git a/library/std/src/sys/pal/solid/abi/fs.rs b/library/std/src/sys/pal/solid/abi/fs.rs index 394be15b0064b..6864a3e7745b9 100644 --- a/library/std/src/sys/pal/solid/abi/fs.rs +++ b/library/std/src/sys/pal/solid/abi/fs.rs @@ -1,8 +1,8 @@ //! `solid_fs.h` pub use libc::{ - ino_t, off_t, stat, time_t, O_APPEND, O_CREAT, O_EXCL, O_RDONLY, O_RDWR, O_TRUNC, O_WRONLY, - SEEK_CUR, SEEK_END, SEEK_SET, S_IFBLK, S_IFCHR, S_IFDIR, S_IFIFO, S_IFMT, S_IFREG, S_IWRITE, + O_APPEND, O_CREAT, O_EXCL, O_RDONLY, O_RDWR, O_TRUNC, O_WRONLY, S_IFBLK, S_IFCHR, S_IFDIR, + S_IFIFO, S_IFMT, S_IFREG, S_IWRITE, SEEK_CUR, SEEK_END, SEEK_SET, ino_t, off_t, stat, time_t, }; use crate::os::raw::{c_char, c_int, c_uchar}; diff --git a/library/std/src/sys/pal/solid/abi/mod.rs b/library/std/src/sys/pal/solid/abi/mod.rs index 62cd86236bbd3..4d09705721748 100644 --- a/library/std/src/sys/pal/solid/abi/mod.rs +++ b/library/std/src/sys/pal/solid/abi/mod.rs @@ -4,7 +4,7 @@ mod fs; pub mod sockets; pub use self::fs::*; // `solid_types.h` -pub use super::itron::abi::{ER, ER_ID, E_TMOUT, ID}; +pub use super::itron::abi::{E_TMOUT, ER, ER_ID, ID}; pub const SOLID_ERR_NOTFOUND: ER = -1000; pub const SOLID_ERR_NOTSUPPORTED: ER = -1001; diff --git a/library/std/src/sys/pal/solid/error.rs b/library/std/src/sys/pal/solid/error.rs index 66c4d8a0ea27a..e092497856d43 100644 --- a/library/std/src/sys/pal/solid/error.rs +++ b/library/std/src/sys/pal/solid/error.rs @@ -1,4 +1,4 @@ -pub use self::itron::error::{expect_success, ItronError as SolidError}; +pub use self::itron::error::{ItronError as SolidError, expect_success}; use super::{abi, itron, net}; use crate::io::ErrorKind; diff --git a/library/std/src/sys/pal/solid/mod.rs b/library/std/src/sys/pal/solid/mod.rs index 6ebcf5b7c48c8..d41042be51844 100644 --- a/library/std/src/sys/pal/solid/mod.rs +++ b/library/std/src/sys/pal/solid/mod.rs @@ -62,13 +62,3 @@ pub fn decode_error_kind(code: i32) -> crate::io::ErrorKind { pub fn abort_internal() -> ! { unsafe { libc::abort() } } - -pub fn hashmap_random_keys() -> (u64, u64) { - unsafe { - let mut out = crate::mem::MaybeUninit::<[u64; 2]>::uninit(); - let result = abi::SOLID_RNG_SampleRandomBytes(out.as_mut_ptr() as *mut u8, 16); - assert_eq!(result, 0, "SOLID_RNG_SampleRandomBytes failed: {result}"); - let [x1, x2] = out.assume_init(); - (x1, x2) - } -} diff --git a/library/std/src/sys/pal/solid/net.rs b/library/std/src/sys/pal/solid/net.rs index b6a31395095d9..c0818ecd856d2 100644 --- a/library/std/src/sys/pal/solid/net.rs +++ b/library/std/src/sys/pal/solid/net.rs @@ -1,6 +1,6 @@ use libc::{c_int, c_void, size_t}; -use self::netc::{sockaddr, socklen_t, MSG_PEEK}; +use self::netc::{MSG_PEEK, sockaddr, socklen_t}; use super::abi; use crate::ffi::CStr; use crate::io::{self, BorrowedBuf, BorrowedCursor, ErrorKind, IoSlice, IoSliceMut}; diff --git a/library/std/src/sys/pal/teeos/mod.rs b/library/std/src/sys/pal/teeos/mod.rs index 00e3860424006..60a227afb84e3 100644 --- a/library/std/src/sys/pal/teeos/mod.rs +++ b/library/std/src/sys/pal/teeos/mod.rs @@ -6,8 +6,6 @@ #![allow(unused_variables)] #![allow(dead_code)] -pub use self::rand::hashmap_random_keys; - #[path = "../unsupported/args.rs"] pub mod args; #[path = "../unsupported/env.rs"] @@ -23,7 +21,6 @@ pub mod os; pub mod pipe; #[path = "../unsupported/process.rs"] pub mod process; -mod rand; pub mod stdio; pub mod thread; #[allow(non_upper_case_globals)] diff --git a/library/std/src/sys/pal/teeos/rand.rs b/library/std/src/sys/pal/teeos/rand.rs deleted file mode 100644 index b45c3bb40e782..0000000000000 --- a/library/std/src/sys/pal/teeos/rand.rs +++ /dev/null @@ -1,21 +0,0 @@ -pub fn hashmap_random_keys() -> (u64, u64) { - const KEY_LEN: usize = core::mem::size_of::(); - - let mut v = [0u8; KEY_LEN * 2]; - imp::fill_bytes(&mut v); - - let key1 = v[0..KEY_LEN].try_into().unwrap(); - let key2 = v[KEY_LEN..].try_into().unwrap(); - - (u64::from_ne_bytes(key1), u64::from_ne_bytes(key2)) -} - -mod imp { - extern "C" { - fn TEE_GenerateRandom(randomBuffer: *mut core::ffi::c_void, randomBufferLen: libc::size_t); - } - - pub fn fill_bytes(v: &mut [u8]) { - unsafe { TEE_GenerateRandom(v.as_mut_ptr() as _, v.len() * crate::mem::size_of::()) } - } -} diff --git a/library/std/src/sys/pal/uefi/helpers.rs b/library/std/src/sys/pal/uefi/helpers.rs index 4031d33ba8066..abc8e69a285f3 100644 --- a/library/std/src/sys/pal/uefi/helpers.rs +++ b/library/std/src/sys/pal/uefi/helpers.rs @@ -10,11 +10,11 @@ //! - More information about protocols can be found [here](https://edk2-docs.gitbook.io/edk-ii-uefi-driver-writer-s-guide/3_foundation/36_protocols_and_handles) use r_efi::efi::{self, Guid}; -use r_efi::protocols::{device_path, device_path_to_text}; +use r_efi::protocols::{device_path, device_path_to_text, shell}; use crate::ffi::{OsStr, OsString}; use crate::io::{self, const_io_error}; -use crate::mem::{size_of, MaybeUninit}; +use crate::mem::{MaybeUninit, size_of}; use crate::os::uefi::env::boot_services; use crate::os::uefi::ffi::{OsStrExt, OsStringExt}; use crate::os::uefi::{self}; @@ -177,16 +177,8 @@ pub(crate) fn device_path_to_text(path: NonNull) -> io::R ) }; - // SAFETY: `convert_device_path_to_text` returns a pointer to a null-terminated UTF-16 - // string, and that string cannot be deallocated prior to dropping the `WStrUnits`, so - // it's safe for `WStrUnits` to use. - let path_len = unsafe { - WStrUnits::new(path_ptr) - .ok_or(io::const_io_error!(io::ErrorKind::InvalidData, "Invalid path"))? - .count() - }; - - let path = OsString::from_wide(unsafe { slice::from_raw_parts(path_ptr.cast(), path_len) }); + let path = os_string_from_raw(path_ptr) + .ok_or(io::const_io_error!(io::ErrorKind::InvalidData, "Invalid path"))?; if let Some(boot_services) = crate::os::uefi::env::boot_services() { let boot_services: NonNull = boot_services.cast(); @@ -420,3 +412,36 @@ impl Drop for OwnedTable { unsafe { crate::alloc::dealloc(self.ptr as *mut u8, self.layout) }; } } + +/// Create OsString from a pointer to NULL terminated UTF-16 string +pub(crate) fn os_string_from_raw(ptr: *mut r_efi::efi::Char16) -> Option { + let path_len = unsafe { WStrUnits::new(ptr)?.count() }; + Some(OsString::from_wide(unsafe { slice::from_raw_parts(ptr.cast(), path_len) })) +} + +/// Create NULL terminated UTF-16 string +pub(crate) fn os_string_to_raw(s: &OsStr) -> Option> { + let temp = s.encode_wide().chain(Some(0)).collect::>(); + if temp[..temp.len() - 1].contains(&0) { None } else { Some(temp) } +} + +pub(crate) fn open_shell() -> Option> { + static LAST_VALID_HANDLE: AtomicPtr = + AtomicPtr::new(crate::ptr::null_mut()); + + if let Some(handle) = NonNull::new(LAST_VALID_HANDLE.load(Ordering::Acquire)) { + if let Ok(protocol) = open_protocol::(handle, shell::PROTOCOL_GUID) { + return Some(protocol); + } + } + + let handles = locate_handles(shell::PROTOCOL_GUID).ok()?; + for handle in handles { + if let Ok(protocol) = open_protocol::(handle, shell::PROTOCOL_GUID) { + LAST_VALID_HANDLE.store(handle.as_ptr(), Ordering::Release); + return Some(protocol); + } + } + + None +} diff --git a/library/std/src/sys/pal/uefi/mod.rs b/library/std/src/sys/pal/uefi/mod.rs index ac22f4ded8855..c0ab52f650aa5 100644 --- a/library/std/src/sys/pal/uefi/mod.rs +++ b/library/std/src/sys/pal/uefi/mod.rs @@ -179,39 +179,6 @@ pub extern "C" fn __rust_abort() { abort_internal(); } -#[inline] -pub fn hashmap_random_keys() -> (u64, u64) { - get_random().unwrap() -} - -fn get_random() -> Option<(u64, u64)> { - use r_efi::protocols::rng; - - let mut buf = [0u8; 16]; - let handles = helpers::locate_handles(rng::PROTOCOL_GUID).ok()?; - for handle in handles { - if let Ok(protocol) = helpers::open_protocol::(handle, rng::PROTOCOL_GUID) { - let r = unsafe { - ((*protocol.as_ptr()).get_rng)( - protocol.as_ptr(), - crate::ptr::null_mut(), - buf.len(), - buf.as_mut_ptr(), - ) - }; - if r.is_error() { - continue; - } else { - return Some(( - u64::from_le_bytes(buf[..8].try_into().ok()?), - u64::from_le_bytes(buf[8..].try_into().ok()?), - )); - } - } - } - None -} - /// Disable access to BootServices if `EVT_SIGNAL_EXIT_BOOT_SERVICES` is signaled extern "efiapi" fn exit_boot_service_handler(_e: r_efi::efi::Event, _ctx: *mut crate::ffi::c_void) { uefi::env::disable_boot_services(); diff --git a/library/std/src/sys/pal/uefi/os.rs b/library/std/src/sys/pal/uefi/os.rs index 4d2d7264704b2..27395f7c3c0b3 100644 --- a/library/std/src/sys/pal/uefi/os.rs +++ b/library/std/src/sys/pal/uefi/os.rs @@ -1,7 +1,7 @@ -use r_efi::efi::protocols::{device_path, loaded_image_device_path}; use r_efi::efi::Status; +use r_efi::efi::protocols::{device_path, loaded_image_device_path}; -use super::{helpers, unsupported, RawOsError}; +use super::{RawOsError, helpers, unsupported_err}; use crate::error::Error as StdError; use crate::ffi::{OsStr, OsString}; use crate::marker::PhantomData; @@ -125,11 +125,32 @@ pub fn error_string(errno: RawOsError) -> String { } pub fn getcwd() -> io::Result { - unsupported() + match helpers::open_shell() { + Some(shell) => { + // SAFETY: path_ptr is managed by UEFI shell and should not be deallocated + let path_ptr = unsafe { ((*shell.as_ptr()).get_cur_dir)(crate::ptr::null_mut()) }; + helpers::os_string_from_raw(path_ptr) + .map(PathBuf::from) + .ok_or(io::const_io_error!(io::ErrorKind::InvalidData, "Invalid path")) + } + None => { + let mut t = current_exe()?; + // SAFETY: This should never fail since the disk prefix will be present even for root + // executables + assert!(t.pop()); + Ok(t) + } + } } -pub fn chdir(_: &path::Path) -> io::Result<()> { - unsupported() +pub fn chdir(p: &path::Path) -> io::Result<()> { + let shell = helpers::open_shell().ok_or(unsupported_err())?; + + let mut p = helpers::os_string_to_raw(p.as_os_str()) + .ok_or(io::const_io_error!(io::ErrorKind::InvalidData, "Invalid path"))?; + + let r = unsafe { ((*shell.as_ptr()).set_cur_dir)(crate::ptr::null_mut(), p.as_mut_ptr()) }; + if r.is_error() { Err(io::Error::from_raw_os_error(r.as_usize())) } else { Ok(()) } } pub struct SplitPaths<'a>(!, PhantomData<&'a ()>); @@ -171,44 +192,58 @@ pub fn current_exe() -> io::Result { helpers::device_path_to_text(protocol).map(PathBuf::from) } -pub struct Env(!); +pub struct EnvStrDebug<'a> { + iter: &'a [(OsString, OsString)], +} + +impl fmt::Debug for EnvStrDebug<'_> { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + let mut list = f.debug_list(); + for (a, b) in self.iter { + list.entry(&(a.to_str().unwrap(), b.to_str().unwrap())); + } + list.finish() + } +} + +pub struct Env(crate::vec::IntoIter<(OsString, OsString)>); impl Env { // FIXME(https://github.com/rust-lang/rust/issues/114583): Remove this when ::fmt matches ::fmt. pub fn str_debug(&self) -> impl fmt::Debug + '_ { - let Self(inner) = self; - match *inner {} + EnvStrDebug { iter: self.0.as_slice() } } } impl Iterator for Env { type Item = (OsString, OsString); + fn next(&mut self) -> Option<(OsString, OsString)> { - self.0 + self.0.next() } } impl fmt::Debug for Env { - fn fmt(&self, _: &mut fmt::Formatter<'_>) -> fmt::Result { - let Self(inner) = self; - match *inner {} + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + self.0.fmt(f) } } pub fn env() -> Env { - panic!("not supported on this platform") + let env = uefi_env::get_all().expect("not supported on this platform"); + Env(env.into_iter()) } -pub fn getenv(_: &OsStr) -> Option { - None +pub fn getenv(key: &OsStr) -> Option { + uefi_env::get(key) } -pub unsafe fn setenv(_: &OsStr, _: &OsStr) -> io::Result<()> { - Err(io::const_io_error!(io::ErrorKind::Unsupported, "cannot set env vars on this platform")) +pub unsafe fn setenv(key: &OsStr, val: &OsStr) -> io::Result<()> { + uefi_env::set(key, val) } -pub unsafe fn unsetenv(_: &OsStr) -> io::Result<()> { - Err(io::const_io_error!(io::ErrorKind::Unsupported, "cannot unset env vars on this platform")) +pub unsafe fn unsetenv(key: &OsStr) -> io::Result<()> { + uefi_env::unset(key) } pub fn temp_dir() -> PathBuf { @@ -239,3 +274,85 @@ pub fn exit(code: i32) -> ! { pub fn getpid() -> u32 { panic!("no pids on this platform") } + +mod uefi_env { + use crate::ffi::{OsStr, OsString}; + use crate::io; + use crate::os::uefi::ffi::OsStringExt; + use crate::ptr::NonNull; + use crate::sys::{helpers, unsupported_err}; + + pub(crate) fn get(key: &OsStr) -> Option { + let shell = helpers::open_shell()?; + let mut key_ptr = helpers::os_string_to_raw(key)?; + unsafe { get_raw(shell, key_ptr.as_mut_ptr()) } + } + + pub(crate) fn set(key: &OsStr, val: &OsStr) -> io::Result<()> { + let mut key_ptr = helpers::os_string_to_raw(key) + .ok_or(io::const_io_error!(io::ErrorKind::InvalidInput, "Invalid Key"))?; + let mut val_ptr = helpers::os_string_to_raw(val) + .ok_or(io::const_io_error!(io::ErrorKind::InvalidInput, "Invalid Value"))?; + unsafe { set_raw(key_ptr.as_mut_ptr(), val_ptr.as_mut_ptr()) } + } + + pub(crate) fn unset(key: &OsStr) -> io::Result<()> { + let mut key_ptr = helpers::os_string_to_raw(key) + .ok_or(io::const_io_error!(io::ErrorKind::InvalidInput, "Invalid Key"))?; + unsafe { set_raw(key_ptr.as_mut_ptr(), crate::ptr::null_mut()) } + } + + pub(crate) fn get_all() -> io::Result> { + let shell = helpers::open_shell().ok_or(unsupported_err())?; + + let mut vars = Vec::new(); + let val = unsafe { ((*shell.as_ptr()).get_env)(crate::ptr::null_mut()) }; + + if val.is_null() { + return Ok(vars); + } + + let mut start = 0; + + // UEFI Shell returns all keys seperated by NULL. + // End of string is denoted by two NULLs + for i in 0.. { + if unsafe { *val.add(i) } == 0 { + // Two NULL signal end of string + if i == start { + break; + } + + let key = OsString::from_wide(unsafe { + crate::slice::from_raw_parts(val.add(start), i - start) + }); + // SAFETY: val.add(start) is always NULL terminated + let val = unsafe { get_raw(shell, val.add(start)) } + .ok_or(io::const_io_error!(io::ErrorKind::InvalidInput, "Invalid Value"))?; + + vars.push((key, val)); + start = i + 1; + } + } + + Ok(vars) + } + + unsafe fn get_raw( + shell: NonNull, + key_ptr: *mut r_efi::efi::Char16, + ) -> Option { + let val = unsafe { ((*shell.as_ptr()).get_env)(key_ptr) }; + helpers::os_string_from_raw(val) + } + + unsafe fn set_raw( + key_ptr: *mut r_efi::efi::Char16, + val_ptr: *mut r_efi::efi::Char16, + ) -> io::Result<()> { + let shell = helpers::open_shell().ok_or(unsupported_err())?; + let r = + unsafe { ((*shell.as_ptr()).set_env)(key_ptr, val_ptr, r_efi::efi::Boolean::FALSE) }; + if r.is_error() { Err(io::Error::from_raw_os_error(r.as_usize())) } else { Ok(()) } + } +} diff --git a/library/std/src/sys/pal/unix/args.rs b/library/std/src/sys/pal/unix/args.rs index a943e3a581a83..8438a61e90feb 100644 --- a/library/std/src/sys/pal/unix/args.rs +++ b/library/std/src/sys/pal/unix/args.rs @@ -113,6 +113,7 @@ impl DoubleEndedIterator for Args { target_os = "nto", target_os = "hurd", target_os = "rtems", + target_os = "nuttx", ))] mod imp { use crate::ffi::c_char; diff --git a/library/std/src/sys/pal/unix/env.rs b/library/std/src/sys/pal/unix/env.rs index b2d399b8791b5..2aee0b5d46056 100644 --- a/library/std/src/sys/pal/unix/env.rs +++ b/library/std/src/sys/pal/unix/env.rs @@ -283,3 +283,14 @@ pub mod os { pub const EXE_SUFFIX: &str = ""; pub const EXE_EXTENSION: &str = ""; } + +#[cfg(target_os = "nuttx")] +pub mod os { + pub const FAMILY: &str = "unix"; + pub const OS: &str = "nuttx"; + pub const DLL_PREFIX: &str = "lib"; + pub const DLL_SUFFIX: &str = ".so"; + pub const DLL_EXTENSION: &str = "so"; + pub const EXE_SUFFIX: &str = ""; + pub const EXE_EXTENSION: &str = ""; +} diff --git a/library/std/src/sys/pal/unix/fd.rs b/library/std/src/sys/pal/unix/fd.rs index d8e239ee23ed5..6a28799ca55eb 100644 --- a/library/std/src/sys/pal/unix/fd.rs +++ b/library/std/src/sys/pal/unix/fd.rs @@ -3,14 +3,6 @@ #[cfg(test)] mod tests; -#[cfg(any( - target_os = "android", - target_os = "linux", - target_os = "emscripten", - target_os = "l4re", - target_os = "hurd", -))] -use libc::off64_t; #[cfg(not(any( target_os = "linux", target_os = "emscripten", @@ -19,6 +11,14 @@ use libc::off64_t; target_os = "hurd", )))] use libc::off_t as off64_t; +#[cfg(any( + target_os = "android", + target_os = "linux", + target_os = "emscripten", + target_os = "l4re", + target_os = "hurd", +))] +use libc::off64_t; use crate::cmp; use crate::io::{self, BorrowedCursor, IoSlice, IoSliceMut, Read}; @@ -98,7 +98,12 @@ impl FileDesc { Ok(ret as usize) } - #[cfg(not(any(target_os = "espidf", target_os = "horizon", target_os = "vita")))] + #[cfg(not(any( + target_os = "espidf", + target_os = "horizon", + target_os = "vita", + target_os = "nuttx" + )))] pub fn read_vectored(&self, bufs: &mut [IoSliceMut<'_>]) -> io::Result { let ret = cvt(unsafe { libc::readv( @@ -110,14 +115,24 @@ impl FileDesc { Ok(ret as usize) } - #[cfg(any(target_os = "espidf", target_os = "horizon", target_os = "vita"))] + #[cfg(any( + target_os = "espidf", + target_os = "horizon", + target_os = "vita", + target_os = "nuttx" + ))] pub fn read_vectored(&self, bufs: &mut [IoSliceMut<'_>]) -> io::Result { io::default_read_vectored(|b| self.read(b), bufs) } #[inline] pub fn is_read_vectored(&self) -> bool { - cfg!(not(any(target_os = "espidf", target_os = "horizon", target_os = "vita"))) + cfg!(not(any( + target_os = "espidf", + target_os = "horizon", + target_os = "vita", + target_os = "nuttx" + ))) } pub fn read_to_end(&self, buf: &mut Vec) -> io::Result { @@ -297,7 +312,12 @@ impl FileDesc { Ok(ret as usize) } - #[cfg(not(any(target_os = "espidf", target_os = "horizon", target_os = "vita")))] + #[cfg(not(any( + target_os = "espidf", + target_os = "horizon", + target_os = "vita", + target_os = "nuttx" + )))] pub fn write_vectored(&self, bufs: &[IoSlice<'_>]) -> io::Result { let ret = cvt(unsafe { libc::writev( @@ -309,14 +329,24 @@ impl FileDesc { Ok(ret as usize) } - #[cfg(any(target_os = "espidf", target_os = "horizon", target_os = "vita"))] + #[cfg(any( + target_os = "espidf", + target_os = "horizon", + target_os = "vita", + target_os = "nuttx" + ))] pub fn write_vectored(&self, bufs: &[IoSlice<'_>]) -> io::Result { io::default_write_vectored(|b| self.write(b), bufs) } #[inline] pub fn is_write_vectored(&self) -> bool { - cfg!(not(any(target_os = "espidf", target_os = "horizon", target_os = "vita"))) + cfg!(not(any( + target_os = "espidf", + target_os = "horizon", + target_os = "vita", + target_os = "nuttx" + ))) } #[cfg_attr(target_os = "vxworks", allow(unused_unsafe))] diff --git a/library/std/src/sys/pal/unix/fs.rs b/library/std/src/sys/pal/unix/fs.rs index 4ec577a0a01d0..f1f843a5f7ae7 100644 --- a/library/std/src/sys/pal/unix/fs.rs +++ b/library/std/src/sys/pal/unix/fs.rs @@ -31,10 +31,6 @@ use libc::fstatat64; all(target_os = "linux", target_env = "musl"), ))] use libc::readdir as readdir64; -#[cfg(any(all(target_os = "linux", not(target_env = "musl")), target_os = "hurd"))] -use libc::readdir64; -#[cfg(any(target_os = "emscripten", target_os = "l4re"))] -use libc::readdir64_r; #[cfg(not(any( target_os = "android", target_os = "linux", @@ -50,6 +46,10 @@ use libc::readdir64_r; target_os = "hurd", )))] use libc::readdir_r as readdir64_r; +#[cfg(any(all(target_os = "linux", not(target_env = "musl")), target_os = "hurd"))] +use libc::readdir64; +#[cfg(any(target_os = "emscripten", target_os = "l4re"))] +use libc::readdir64_r; use libc::{c_int, mode_t}; #[cfg(target_os = "android")] use libc::{ @@ -189,7 +189,7 @@ cfg_has_statx! {{ // See: https://github.com/rust-lang/rust/issues/65662 // // FIXME what about transient conditions like `ENOMEM`? - let err2 = cvt(statx(0, ptr::null(), 0, libc::STATX_ALL, ptr::null_mut())) + let err2 = cvt(statx(0, ptr::null(), 0, libc::STATX_BASIC_STATS | libc::STATX_BTIME, ptr::null_mut())) .err() .and_then(|e| e.raw_os_error()); if err2 == Some(libc::EFAULT) { @@ -479,6 +479,7 @@ impl FileAttr { target_os = "vita", target_os = "hurd", target_os = "rtems", + target_os = "nuttx", )))] pub fn modified(&self) -> io::Result { #[cfg(target_pointer_width = "32")] @@ -501,7 +502,7 @@ impl FileAttr { SystemTime::new(self.stat.st_mtime as i64, 0) } - #[cfg(any(target_os = "horizon", target_os = "hurd"))] + #[cfg(any(target_os = "horizon", target_os = "hurd", target_os = "nuttx"))] pub fn modified(&self) -> io::Result { SystemTime::new(self.stat.st_mtim.tv_sec as i64, self.stat.st_mtim.tv_nsec as i64) } @@ -513,6 +514,7 @@ impl FileAttr { target_os = "vita", target_os = "hurd", target_os = "rtems", + target_os = "nuttx", )))] pub fn accessed(&self) -> io::Result { #[cfg(target_pointer_width = "32")] @@ -535,7 +537,7 @@ impl FileAttr { SystemTime::new(self.stat.st_atime as i64, 0) } - #[cfg(any(target_os = "horizon", target_os = "hurd"))] + #[cfg(any(target_os = "horizon", target_os = "hurd", target_os = "nuttx"))] pub fn accessed(&self) -> io::Result { SystemTime::new(self.stat.st_atim.tv_sec as i64, self.stat.st_atim.tv_nsec as i64) } @@ -738,7 +740,7 @@ impl Iterator for ReadDir { // // Like for uninitialized contents, converting entry_ptr to `&dirent64` // would not be legal. However, unique to dirent64 is that we don't even - // get to use `addr_of!((*entry_ptr).d_name)` because that operation + // get to use `&raw const (*entry_ptr).d_name` because that operation // requires the full extent of *entry_ptr to be in bounds of the same // allocation, which is not necessarily the case here. // @@ -752,7 +754,7 @@ impl Iterator for ReadDir { } else { #[allow(deref_nullptr)] { - ptr::addr_of!((*ptr::null::()).$field) + &raw const (*ptr::null::()).$field } } }}; @@ -866,6 +868,7 @@ impl Drop for Dir { target_os = "horizon", target_os = "vxworks", target_os = "rtems", + target_os = "nuttx", )))] { let fd = unsafe { libc::dirfd(self.0) }; @@ -896,7 +899,7 @@ impl DirEntry { target_os = "android", target_os = "hurd" ), - not(miri) + not(miri) // no dirfd on Miri ))] pub fn metadata(&self) -> io::Result { let fd = cvt(unsafe { dirfd(self.dir.dirp.0) })?; @@ -907,7 +910,7 @@ impl DirEntry { fd, name, libc::AT_SYMLINK_NOFOLLOW | libc::AT_STATX_SYNC_AS_STAT, - libc::STATX_ALL, + libc::STATX_BASIC_STATS | libc::STATX_BTIME, ) } { return ret; } @@ -1000,6 +1003,13 @@ impl DirEntry { self.entry.d_fileno as u64 } + #[cfg(target_os = "nuttx")] + pub fn ino(&self) -> u64 { + // Leave this 0 for now, as NuttX does not provide an inode number + // in its directory entries. + 0 + } + #[cfg(any( target_os = "netbsd", target_os = "openbsd", @@ -1184,7 +1194,7 @@ impl File { fd, c"".as_ptr() as *const c_char, libc::AT_EMPTY_PATH | libc::AT_STATX_SYNC_AS_STAT, - libc::STATX_ALL, + libc::STATX_BASIC_STATS | libc::STATX_BTIME, ) } { return ret; } @@ -1327,7 +1337,8 @@ impl File { target_os = "redox", target_os = "espidf", target_os = "horizon", - target_os = "vxworks" + target_os = "vxworks", + target_os = "nuttx", )))] let to_timespec = |time: Option| match time { Some(time) if let Some(ts) = time.t.to_timespec() => Ok(ts), @@ -1342,7 +1353,7 @@ impl File { None => Ok(libc::timespec { tv_sec: 0, tv_nsec: libc::UTIME_OMIT as _ }), }; cfg_if::cfg_if! { - if #[cfg(any(target_os = "redox", target_os = "espidf", target_os = "horizon", target_os = "vxworks"))] { + if #[cfg(any(target_os = "redox", target_os = "espidf", target_os = "horizon", target_os = "vxworks", target_os = "nuttx"))] { // Redox doesn't appear to support `UTIME_OMIT`. // ESP-IDF and HorizonOS do not support `futimens` at all and the behavior for those OS is therefore // the same as for Redox. @@ -1374,7 +1385,7 @@ impl File { } cvt(unsafe { libc::fsetattrlist( self.as_raw_fd(), - core::ptr::addr_of!(attrlist).cast::().cast_mut(), + (&raw const attrlist).cast::().cast_mut(), buf.as_ptr().cast::().cast_mut(), num_times * mem::size_of::(), 0 @@ -1527,7 +1538,7 @@ impl fmt::Debug for File { Some(PathBuf::from(OsString::from_vec(buf))) } - #[cfg(all(target_os = "freebsd", target_arch = "x86_64"))] + #[cfg(target_os = "freebsd")] fn get_path(fd: c_int) -> Option { let info = Box::::new_zeroed(); let mut info = unsafe { info.assume_init() }; @@ -1555,7 +1566,7 @@ impl fmt::Debug for File { #[cfg(not(any( target_os = "linux", target_os = "vxworks", - all(target_os = "freebsd", target_arch = "x86_64"), + target_os = "freebsd", target_os = "netbsd", target_os = "illumos", target_os = "solaris", @@ -1731,7 +1742,7 @@ pub fn link(original: &Path, link: &Path) -> io::Result<()> { run_path_with_cstr(original, &|original| { run_path_with_cstr(link, &|link| { cfg_if::cfg_if! { - if #[cfg(any(target_os = "vxworks", target_os = "redox", target_os = "android", target_os = "espidf", target_os = "horizon", target_os = "vita", target_os = "nto"))] { + if #[cfg(any(target_os = "vxworks", target_os = "redox", target_os = "android", target_os = "espidf", target_os = "horizon", target_os = "vita", target_env = "nto70"))] { // VxWorks, Redox and ESP-IDF lack `linkat`, so use `link` instead. POSIX leaves // it implementation-defined whether `link` follows symlinks, so rely on the // `symlink_hard_link` test in library/std/src/fs/tests.rs to check the behavior. @@ -1756,7 +1767,7 @@ pub fn stat(p: &Path) -> io::Result { libc::AT_FDCWD, p.as_ptr(), libc::AT_STATX_SYNC_AS_STAT, - libc::STATX_ALL, + libc::STATX_BASIC_STATS | libc::STATX_BTIME, ) } { return ret; } @@ -1775,7 +1786,7 @@ pub fn lstat(p: &Path) -> io::Result { libc::AT_FDCWD, p.as_ptr(), libc::AT_SYMLINK_NOFOLLOW | libc::AT_STATX_SYNC_AS_STAT, - libc::STATX_ALL, + libc::STATX_BASIC_STATS | libc::STATX_BTIME, ) } { return ret; } @@ -1866,7 +1877,7 @@ pub fn copy(from: &Path, to: &Path) -> io::Result { let max_len = u64::MAX; let (mut writer, _) = open_to_and_set_permissions(to, reader_metadata)?; - use super::kernel_copy::{copy_regular_files, CopyResult}; + use super::kernel_copy::{CopyResult, copy_regular_files}; match copy_regular_files(reader.as_raw_fd(), writer.as_raw_fd(), max_len) { CopyResult::Ended(bytes) => Ok(bytes), @@ -1933,7 +1944,7 @@ pub fn copy(from: &Path, to: &Path) -> io::Result { libc::copyfile_state_get( state.0, libc::COPYFILE_STATE_COPIED as u32, - core::ptr::addr_of_mut!(bytes_copied) as *mut libc::c_void, + (&raw mut bytes_copied) as *mut libc::c_void, ) })?; Ok(bytes_copied as u64) @@ -2008,7 +2019,7 @@ mod remove_dir_impl { #[cfg(all(target_os = "linux", target_env = "gnu"))] use libc::{fdopendir, openat64 as openat, unlinkat}; - use super::{lstat, Dir, DirEntry, InnerReadDir, ReadDir}; + use super::{Dir, DirEntry, InnerReadDir, ReadDir, lstat}; use crate::ffi::CStr; use crate::io; use crate::os::unix::io::{AsRawFd, FromRawFd, IntoRawFd}; diff --git a/library/std/src/sys/pal/unix/futex.rs b/library/std/src/sys/pal/unix/futex.rs index cc725045c4810..0fc765dc87a5d 100644 --- a/library/std/src/sys/pal/unix/futex.rs +++ b/library/std/src/sys/pal/unix/futex.rs @@ -11,9 +11,14 @@ use crate::sync::atomic::AtomicU32; use crate::time::Duration; +/// An atomic for use as a futex that is at least 32-bits but may be larger +pub type Futex = AtomicU32; +/// Must be the underlying type of Futex +pub type Primitive = u32; + /// An atomic for use as a futex that is at least 8-bits but may be larger. -pub type SmallAtomic = AtomicU32; -/// Must be the underlying type of SmallAtomic +pub type SmallFutex = AtomicU32; +/// Must be the underlying type of SmallFutex pub type SmallPrimitive = u32; /// Waits for a `futex_wake` operation to wake us. diff --git a/library/std/src/sys/pal/unix/linux/pidfd.rs b/library/std/src/sys/pal/unix/linux/pidfd.rs index 7474f80e94f9d..78744430f3b51 100644 --- a/library/std/src/sys/pal/unix/linux/pidfd.rs +++ b/library/std/src/sys/pal/unix/linux/pidfd.rs @@ -13,7 +13,7 @@ pub(crate) struct PidFd(FileDesc); impl PidFd { pub fn kill(&self) -> io::Result<()> { - return cvt(unsafe { + cvt(unsafe { libc::syscall( libc::SYS_pidfd_send_signal, self.0.as_raw_fd(), @@ -22,7 +22,7 @@ impl PidFd { 0, ) }) - .map(drop); + .map(drop) } pub fn wait(&self) -> io::Result { @@ -30,7 +30,7 @@ impl PidFd { cvt(unsafe { libc::waitid(libc::P_PIDFD, self.0.as_raw_fd() as u32, &mut siginfo, libc::WEXITED) })?; - return Ok(ExitStatus::from_waitid_siginfo(siginfo)); + Ok(ExitStatus::from_waitid_siginfo(siginfo)) } pub fn try_wait(&self) -> io::Result> { @@ -45,9 +45,10 @@ impl PidFd { ) })?; if unsafe { siginfo.si_pid() } == 0 { - return Ok(None); + Ok(None) + } else { + Ok(Some(ExitStatus::from_waitid_siginfo(siginfo))) } - return Ok(Some(ExitStatus::from_waitid_siginfo(siginfo))); } } diff --git a/library/std/src/sys/pal/unix/mod.rs b/library/std/src/sys/pal/unix/mod.rs index e8428eccb1691..4fe18daa2040f 100644 --- a/library/std/src/sys/pal/unix/mod.rs +++ b/library/std/src/sys/pal/unix/mod.rs @@ -1,6 +1,5 @@ #![allow(missing_docs, nonstandard_style)] -pub use self::rand::hashmap_random_keys; use crate::io::ErrorKind; #[cfg(not(target_os = "espidf"))] @@ -26,7 +25,6 @@ pub use self::l4re::net; pub mod os; pub mod pipe; pub mod process; -pub mod rand; pub mod stack_overflow; pub mod stdio; pub mod thread; @@ -224,6 +222,7 @@ static ON_BROKEN_PIPE_FLAG_USED: crate::sync::atomic::AtomicBool = target_os = "horizon", target_os = "vxworks", target_os = "vita", + target_os = "nuttx", )))] pub(crate) fn on_broken_pipe_flag_used() -> bool { ON_BROKEN_PIPE_FLAG_USED.load(crate::sync::atomic::Ordering::Relaxed) @@ -281,6 +280,7 @@ pub fn decode_error_kind(errno: i32) -> ErrorKind { libc::ETIMEDOUT => TimedOut, libc::ETXTBSY => ExecutableFileBusy, libc::EXDEV => CrossesDevices, + libc::EINPROGRESS => InProgress, libc::EACCES | libc::EPERM => PermissionDenied, @@ -426,7 +426,7 @@ cfg_if::cfg_if! { } } -#[cfg(any(target_os = "espidf", target_os = "horizon", target_os = "vita"))] +#[cfg(any(target_os = "espidf", target_os = "horizon", target_os = "vita", target_os = "nuttx"))] mod unsupported { use crate::io; diff --git a/library/std/src/sys/pal/unix/net.rs b/library/std/src/sys/pal/unix/net.rs index d75a666d350ff..6a67bb0a101e9 100644 --- a/library/std/src/sys/pal/unix/net.rs +++ b/library/std/src/sys/pal/unix/net.rs @@ -1,4 +1,4 @@ -use libc::{c_int, c_void, size_t, sockaddr, socklen_t, MSG_PEEK}; +use libc::{MSG_PEEK, c_int, c_void, size_t, sockaddr, socklen_t}; use crate::ffi::CStr; use crate::io::{self, BorrowedBuf, BorrowedCursor, IoSlice, IoSliceMut}; @@ -38,19 +38,19 @@ pub fn cvt_gai(err: c_int) -> io::Result<()> { // We may need to trigger a glibc workaround. See on_resolver_failure() for details. on_resolver_failure(); - #[cfg(not(target_os = "espidf"))] + #[cfg(not(any(target_os = "espidf", target_os = "nuttx")))] if err == libc::EAI_SYSTEM { return Err(io::Error::last_os_error()); } - #[cfg(not(target_os = "espidf"))] + #[cfg(not(any(target_os = "espidf", target_os = "nuttx")))] let detail = unsafe { // We can't always expect a UTF-8 environment. When we don't get that luxury, // it's better to give a low-quality error message than none at all. CStr::from_ptr(libc::gai_strerror(err)).to_string_lossy() }; - #[cfg(target_os = "espidf")] + #[cfg(any(target_os = "espidf", target_os = "nuttx"))] let detail = ""; Err(io::Error::new( @@ -329,7 +329,7 @@ impl Socket { buf.as_mut_ptr() as *mut c_void, buf.len(), flags, - core::ptr::addr_of_mut!(storage) as *mut _, + (&raw mut storage) as *mut _, &mut addrlen, ) })?; diff --git a/library/std/src/sys/pal/unix/os.rs b/library/std/src/sys/pal/unix/os.rs index 503f8915256ee..f983d174ed61c 100644 --- a/library/std/src/sys/pal/unix/os.rs +++ b/library/std/src/sys/pal/unix/os.rs @@ -48,6 +48,7 @@ extern "C" { target_os = "openbsd", target_os = "android", target_os = "redox", + target_os = "nuttx", target_env = "newlib" ), link_name = "__errno" @@ -399,6 +400,7 @@ pub fn current_exe() -> io::Result { target_os = "linux", target_os = "hurd", target_os = "android", + target_os = "nuttx", target_os = "emscripten" ))] pub fn current_exe() -> io::Result { @@ -610,7 +612,7 @@ pub unsafe fn environ() -> *mut *const *const c_char { extern "C" { static mut environ: *const *const c_char; } - ptr::addr_of_mut!(environ) + &raw mut environ } static ENV_LOCK: RwLock<()> = RwLock::new(()); @@ -717,6 +719,7 @@ pub fn home_dir() -> Option { target_os = "espidf", target_os = "horizon", target_os = "vita", + target_os = "nuttx", all(target_vendor = "apple", not(target_os = "macos")), ))] unsafe fn fallback() -> Option { @@ -730,6 +733,7 @@ pub fn home_dir() -> Option { target_os = "espidf", target_os = "horizon", target_os = "vita", + target_os = "nuttx", all(target_vendor = "apple", not(target_os = "macos")), )))] unsafe fn fallback() -> Option { diff --git a/library/std/src/sys/pal/unix/process/mod.rs b/library/std/src/sys/pal/unix/process/mod.rs index 074f0a105e329..2751d51c44d2a 100644 --- a/library/std/src/sys/pal/unix/process/mod.rs +++ b/library/std/src/sys/pal/unix/process/mod.rs @@ -2,10 +2,10 @@ pub use self::process_common::{Command, CommandArgs, ExitCode, Stdio, StdioPipes pub use self::process_inner::{ExitStatus, ExitStatusError, Process}; pub use crate::ffi::OsString as EnvKey; -#[cfg_attr(any(target_os = "espidf", target_os = "horizon"), allow(unused))] +#[cfg_attr(any(target_os = "espidf", target_os = "horizon", target_os = "nuttx"), allow(unused))] mod process_common; -#[cfg(any(target_os = "espidf", target_os = "horizon", target_os = "vita"))] +#[cfg(any(target_os = "espidf", target_os = "horizon", target_os = "vita", target_os = "nuttx"))] mod process_unsupported; cfg_if::cfg_if! { @@ -16,7 +16,7 @@ cfg_if::cfg_if! { } else if #[cfg(target_os = "vxworks")] { #[path = "process_vxworks.rs"] mod process_inner; - } else if #[cfg(any(target_os = "espidf", target_os = "horizon", target_os = "vita"))] { + } else if #[cfg(any(target_os = "espidf", target_os = "horizon", target_os = "vita", target_os = "nuttx"))] { mod process_inner { pub use super::process_unsupported::*; } diff --git a/library/std/src/sys/pal/unix/process/process_common.rs b/library/std/src/sys/pal/unix/process/process_common.rs index fec825054195a..13290fed762ae 100644 --- a/library/std/src/sys/pal/unix/process/process_common.rs +++ b/library/std/src/sys/pal/unix/process/process_common.rs @@ -1,7 +1,7 @@ #[cfg(all(test, not(target_os = "emscripten")))] mod tests; -use libc::{c_char, c_int, gid_t, pid_t, uid_t, EXIT_FAILURE, EXIT_SUCCESS}; +use libc::{EXIT_FAILURE, EXIT_SUCCESS, c_char, c_int, gid_t, pid_t, uid_t}; use crate::collections::BTreeMap; use crate::ffi::{CStr, CString, OsStr, OsString}; @@ -312,8 +312,8 @@ impl Command { } #[allow(dead_code)] - pub fn get_cwd(&self) -> &Option { - &self.cwd + pub fn get_cwd(&self) -> Option<&CStr> { + self.cwd.as_deref() } #[allow(dead_code)] pub fn get_uid(&self) -> Option { diff --git a/library/std/src/sys/pal/unix/process/process_fuchsia.rs b/library/std/src/sys/pal/unix/process/process_fuchsia.rs index f3d5fdec4c291..8f7d786e32fcd 100644 --- a/library/std/src/sys/pal/unix/process/process_fuchsia.rs +++ b/library/std/src/sys/pal/unix/process/process_fuchsia.rs @@ -2,7 +2,7 @@ use libc::{c_int, size_t}; use crate::num::NonZero; use crate::sys::process::process_common::*; -use crate::sys::process::zircon::{zx_handle_t, Handle}; +use crate::sys::process::zircon::{Handle, zx_handle_t}; use crate::{fmt, io, mem, ptr}; //////////////////////////////////////////////////////////////////////////////// @@ -178,7 +178,7 @@ impl Process { zx_cvt(zx_object_get_info( self.handle.raw(), ZX_INFO_PROCESS, - core::ptr::addr_of_mut!(proc_info) as *mut libc::c_void, + (&raw mut proc_info) as *mut libc::c_void, mem::size_of::(), &mut actual, &mut avail, @@ -215,7 +215,7 @@ impl Process { zx_cvt(zx_object_get_info( self.handle.raw(), ZX_INFO_PROCESS, - core::ptr::addr_of_mut!(proc_info) as *mut libc::c_void, + (&raw mut proc_info) as *mut libc::c_void, mem::size_of::(), &mut actual, &mut avail, @@ -273,7 +273,7 @@ impl ExitStatus { // We don't know what someone who calls into_raw() will do with this value, but it should // have the conventional Unix representation. Despite the fact that this is not // standardised in SuS or POSIX, all Unix systems encode the signal and exit status the - // same way. (Ie the WIFEXITED, WEXITSTATUS etc. macros have identical behaviour on every + // same way. (Ie the WIFEXITED, WEXITSTATUS etc. macros have identical behavior on every // Unix.) // // The caller of `std::os::unix::into_raw` is probably wanting a Unix exit status, and may diff --git a/library/std/src/sys/pal/unix/process/process_unix.rs b/library/std/src/sys/pal/unix/process/process_unix.rs index 4bb22f3670978..8faf1fda5464d 100644 --- a/library/std/src/sys/pal/unix/process/process_unix.rs +++ b/library/std/src/sys/pal/unix/process/process_unix.rs @@ -335,7 +335,7 @@ impl Command { cvt(libc::setuid(u as uid_t))?; } } - if let Some(ref cwd) = *self.get_cwd() { + if let Some(cwd) = self.get_cwd() { cvt(libc::chdir(cwd.as_ptr()))?; } @@ -448,7 +448,6 @@ impl Command { use core::sync::atomic::{AtomicU8, Ordering}; use crate::mem::MaybeUninit; - use crate::sys::weak::weak; use crate::sys::{self, cvt_nz, on_broken_pipe_flag_used}; if self.get_gid().is_some() @@ -462,6 +461,8 @@ impl Command { cfg_if::cfg_if! { if #[cfg(target_os = "linux")] { + use crate::sys::weak::weak; + weak! { fn pidfd_spawnp( *mut libc::c_int, @@ -575,16 +576,44 @@ impl Command { } } - // Solaris, glibc 2.29+, and musl 1.24+ can set a new working directory, - // and maybe others will gain this non-POSIX function too. We'll check - // for this weak symbol as soon as it's needed, so we can return early - // otherwise to do a manual chdir before exec. - weak! { - fn posix_spawn_file_actions_addchdir_np( - *mut libc::posix_spawn_file_actions_t, - *const libc::c_char - ) -> libc::c_int + type PosixSpawnAddChdirFn = unsafe extern "C" fn( + *mut libc::posix_spawn_file_actions_t, + *const libc::c_char, + ) -> libc::c_int; + + /// Get the function pointer for adding a chdir action to a + /// `posix_spawn_file_actions_t`, if available, assuming a dynamic libc. + /// + /// Some platforms can set a new working directory for a spawned process in the + /// `posix_spawn` path. This function looks up the function pointer for adding + /// such an action to a `posix_spawn_file_actions_t` struct. + #[cfg(not(all(target_os = "linux", target_env = "musl")))] + fn get_posix_spawn_addchdir() -> Option { + use crate::sys::weak::weak; + + weak! { + fn posix_spawn_file_actions_addchdir_np( + *mut libc::posix_spawn_file_actions_t, + *const libc::c_char + ) -> libc::c_int + } + + posix_spawn_file_actions_addchdir_np.get() + } + + /// Get the function pointer for adding a chdir action to a + /// `posix_spawn_file_actions_t`, if available, on platforms where the function + /// is known to exist. + /// + /// Weak symbol lookup doesn't work with statically linked libcs, so in cases + /// where static linking is possible we need to either check for the presence + /// of the symbol at compile time or know about it upfront. + #[cfg(all(target_os = "linux", target_env = "musl"))] + fn get_posix_spawn_addchdir() -> Option { + // Our minimum required musl supports this function, so we can just use it. + Some(libc::posix_spawn_file_actions_addchdir_np) } + let addchdir = match self.get_cwd() { Some(cwd) => { if cfg!(target_vendor = "apple") { @@ -597,7 +626,10 @@ impl Command { return Ok(None); } } - match posix_spawn_file_actions_addchdir_np.get() { + // Check for the availability of the posix_spawn addchdir + // function now. If it isn't available, bail and use the + // fork/exec path. + match get_posix_spawn_addchdir() { Some(f) => Some((f, cwd)), None => return Ok(None), } @@ -788,15 +820,15 @@ impl Command { let mut iov = [IoSlice::new(b"")]; let mut msg: libc::msghdr = mem::zeroed(); - msg.msg_iov = core::ptr::addr_of_mut!(iov) as *mut _; + msg.msg_iov = (&raw mut iov) as *mut _; msg.msg_iovlen = 1; // only attach cmsg if we successfully acquired the pidfd if pidfd >= 0 { msg.msg_controllen = mem::size_of_val(&cmsg.buf) as _; - msg.msg_control = core::ptr::addr_of_mut!(cmsg.buf) as *mut _; + msg.msg_control = (&raw mut cmsg.buf) as *mut _; - let hdr = CMSG_FIRSTHDR(core::ptr::addr_of_mut!(msg) as *mut _); + let hdr = CMSG_FIRSTHDR((&raw mut msg) as *mut _); (*hdr).cmsg_level = SOL_SOCKET; (*hdr).cmsg_type = SCM_RIGHTS; (*hdr).cmsg_len = CMSG_LEN(SCM_MSG_LEN as _) as _; @@ -838,17 +870,17 @@ impl Command { let mut msg: libc::msghdr = mem::zeroed(); - msg.msg_iov = core::ptr::addr_of_mut!(iov) as *mut _; + msg.msg_iov = (&raw mut iov) as *mut _; msg.msg_iovlen = 1; msg.msg_controllen = mem::size_of::() as _; - msg.msg_control = core::ptr::addr_of_mut!(cmsg) as *mut _; + msg.msg_control = (&raw mut cmsg) as *mut _; match cvt_r(|| libc::recvmsg(sock.as_raw(), &mut msg, libc::MSG_CMSG_CLOEXEC)) { Err(_) => return -1, Ok(_) => {} } - let hdr = CMSG_FIRSTHDR(core::ptr::addr_of_mut!(msg) as *mut _); + let hdr = CMSG_FIRSTHDR((&raw mut msg) as *mut _); if hdr.is_null() || (*hdr).cmsg_level != SOL_SOCKET || (*hdr).cmsg_type != SCM_RIGHTS @@ -1190,8 +1222,8 @@ impl ExitStatusError { mod linux_child_ext { use crate::os::linux::process as os; - use crate::sys::pal::unix::linux::pidfd as imp; use crate::sys::pal::unix::ErrorKind; + use crate::sys::pal::unix::linux::pidfd as imp; use crate::sys_common::FromInner; use crate::{io, mem}; diff --git a/library/std/src/sys/pal/unix/process/process_vxworks.rs b/library/std/src/sys/pal/unix/process/process_vxworks.rs index 0477b3d9a70da..38daf6af91808 100644 --- a/library/std/src/sys/pal/unix/process/process_vxworks.rs +++ b/library/std/src/sys/pal/unix/process/process_vxworks.rs @@ -1,5 +1,5 @@ #![forbid(unsafe_op_in_unsafe_fn)] -use libc::{self, c_char, c_int, RTP_ID}; +use libc::{self, RTP_ID, c_char, c_int}; use crate::io::{self, ErrorKind}; use crate::num::NonZero; @@ -57,7 +57,7 @@ impl Command { t!(cvt_r(|| libc::dup2(fd, libc::STDERR_FILENO))); } - if let Some(ref cwd) = *self.get_cwd() { + if let Some(cwd) = self.get_cwd() { t!(cvt(libc::chdir(cwd.as_ptr()))); } diff --git a/library/std/src/sys/pal/unix/rand.rs b/library/std/src/sys/pal/unix/rand.rs deleted file mode 100644 index cc0852aab4396..0000000000000 --- a/library/std/src/sys/pal/unix/rand.rs +++ /dev/null @@ -1,302 +0,0 @@ -pub fn hashmap_random_keys() -> (u64, u64) { - const KEY_LEN: usize = core::mem::size_of::(); - - let mut v = [0u8; KEY_LEN * 2]; - if let Err(err) = read(&mut v) { - panic!("failed to retrieve random hash map seed: {err}"); - } - - let key1 = v[0..KEY_LEN].try_into().unwrap(); - let key2 = v[KEY_LEN..].try_into().unwrap(); - - (u64::from_ne_bytes(key1), u64::from_ne_bytes(key2)) -} - -cfg_if::cfg_if! { - if #[cfg(any( - target_vendor = "apple", - target_os = "openbsd", - target_os = "emscripten", - target_os = "vita", - all(target_os = "netbsd", not(netbsd10)), - target_os = "fuchsia", - target_os = "vxworks", - ))] { - // Some systems have a syscall that directly retrieves random data. - // If that is guaranteed to be available, use it. - use imp::syscall as read; - } else { - // Otherwise, try the syscall to see if it exists only on some systems - // and fall back to reading from the random device otherwise. - fn read(bytes: &mut [u8]) -> crate::io::Result<()> { - use crate::fs::File; - use crate::io::Read; - use crate::sync::OnceLock; - - #[cfg(any( - target_os = "linux", - target_os = "android", - target_os = "espidf", - target_os = "horizon", - target_os = "freebsd", - target_os = "dragonfly", - target_os = "solaris", - target_os = "illumos", - netbsd10, - ))] - if let Some(res) = imp::syscall(bytes) { - return res; - } - - const PATH: &'static str = if cfg!(target_os = "redox") { - "/scheme/rand" - } else { - "/dev/urandom" - }; - - static FILE: OnceLock = OnceLock::new(); - - FILE.get_or_try_init(|| File::open(PATH))?.read_exact(bytes) - } - } -} - -// All these systems a `getrandom` syscall. -// -// It is not guaranteed to be available, so return None to fallback to the file -// implementation. -#[cfg(any( - target_os = "linux", - target_os = "android", - target_os = "espidf", - target_os = "horizon", - target_os = "freebsd", - target_os = "dragonfly", - target_os = "solaris", - target_os = "illumos", - netbsd10, -))] -mod imp { - use crate::io::{Error, Result}; - use crate::sync::atomic::{AtomicBool, Ordering}; - use crate::sys::os::errno; - - #[cfg(any(target_os = "linux", target_os = "android"))] - fn getrandom(buf: &mut [u8]) -> libc::ssize_t { - use crate::sys::weak::syscall; - - // A weak symbol allows interposition, e.g. for perf measurements that want to - // disable randomness for consistency. Otherwise, we'll try a raw syscall. - // (`getrandom` was added in glibc 2.25, musl 1.1.20, android API level 28) - syscall! { - fn getrandom( - buffer: *mut libc::c_void, - length: libc::size_t, - flags: libc::c_uint - ) -> libc::ssize_t - } - - // This provides the best quality random numbers available at the given moment - // without ever blocking, and is preferable to falling back to /dev/urandom. - static GRND_INSECURE_AVAILABLE: AtomicBool = AtomicBool::new(true); - if GRND_INSECURE_AVAILABLE.load(Ordering::Relaxed) { - let ret = unsafe { getrandom(buf.as_mut_ptr().cast(), buf.len(), libc::GRND_INSECURE) }; - if ret == -1 && errno() as libc::c_int == libc::EINVAL { - GRND_INSECURE_AVAILABLE.store(false, Ordering::Relaxed); - } else { - return ret; - } - } - - unsafe { getrandom(buf.as_mut_ptr().cast(), buf.len(), libc::GRND_NONBLOCK) } - } - - #[cfg(any( - target_os = "dragonfly", - target_os = "espidf", - target_os = "horizon", - target_os = "freebsd", - netbsd10, - target_os = "illumos", - target_os = "solaris" - ))] - fn getrandom(buf: &mut [u8]) -> libc::ssize_t { - unsafe { libc::getrandom(buf.as_mut_ptr().cast(), buf.len(), 0) } - } - - pub fn syscall(v: &mut [u8]) -> Option> { - static GETRANDOM_UNAVAILABLE: AtomicBool = AtomicBool::new(false); - - if GETRANDOM_UNAVAILABLE.load(Ordering::Relaxed) { - return None; - } - - let mut read = 0; - while read < v.len() { - let result = getrandom(&mut v[read..]); - if result == -1 { - let err = errno() as libc::c_int; - if err == libc::EINTR { - continue; - } else if err == libc::ENOSYS || err == libc::EPERM { - // `getrandom` is not supported on the current system. - // - // Also fall back in case it is disabled by something like - // seccomp or inside of docker. - // - // If the `getrandom` syscall is not implemented in the current kernel version it should return an - // `ENOSYS` error. Docker also blocks the whole syscall inside unprivileged containers, and - // returns `EPERM` (instead of `ENOSYS`) when a program tries to invoke the syscall. Because of - // that we need to check for *both* `ENOSYS` and `EPERM`. - // - // Note that Docker's behavior is breaking other projects (notably glibc), so they're planning - // to update their filtering to return `ENOSYS` in a future release: - // - // https://github.com/moby/moby/issues/42680 - // - GETRANDOM_UNAVAILABLE.store(true, Ordering::Relaxed); - return None; - } else if err == libc::EAGAIN { - // getrandom has failed because it would have blocked as the - // non-blocking pool (urandom) has not been initialized in - // the kernel yet due to a lack of entropy. Fallback to - // reading from `/dev/urandom` which will return potentially - // insecure random data to avoid blocking applications which - // could depend on this call without ever knowing they do and - // don't have a work around. - return None; - } else { - return Some(Err(Error::from_raw_os_error(err))); - } - } else { - read += result as usize; - } - } - - Some(Ok(())) - } -} - -#[cfg(any( - target_os = "macos", // Supported since macOS 10.12+. - target_os = "openbsd", - target_os = "emscripten", - target_os = "vita", -))] -mod imp { - use crate::io::{Error, Result}; - - pub fn syscall(v: &mut [u8]) -> Result<()> { - // getentropy(2) permits a maximum buffer size of 256 bytes - for s in v.chunks_mut(256) { - let ret = unsafe { libc::getentropy(s.as_mut_ptr().cast(), s.len()) }; - if ret == -1 { - return Err(Error::last_os_error()); - } - } - - Ok(()) - } -} - -// On Apple platforms, `CCRandomGenerateBytes` and `SecRandomCopyBytes` simply -// call into `CCRandomCopyBytes` with `kCCRandomDefault`. `CCRandomCopyBytes` -// manages a CSPRNG which is seeded from the kernel's CSPRNG and which runs on -// its own thread accessed via GCD. This seems needlessly heavyweight for our purposes -// so we only use it when `getentropy` is blocked, which appears to be the case -// on all platforms except macOS (see #102643). -// -// `CCRandomGenerateBytes` is used instead of `SecRandomCopyBytes` because the former is accessible -// via `libSystem` (libc) while the other needs to link to `Security.framework`. -#[cfg(all(target_vendor = "apple", not(target_os = "macos")))] -mod imp { - use libc::size_t; - - use crate::ffi::{c_int, c_void}; - use crate::io::{Error, Result}; - - pub fn syscall(v: &mut [u8]) -> Result<()> { - extern "C" { - fn CCRandomGenerateBytes(bytes: *mut c_void, count: size_t) -> c_int; - } - - let ret = unsafe { CCRandomGenerateBytes(v.as_mut_ptr().cast(), v.len()) }; - if ret != -1 { Ok(()) } else { Err(Error::last_os_error()) } - } -} - -// FIXME: once the 10.x release becomes the minimum, this can be dropped for simplification. -#[cfg(all(target_os = "netbsd", not(netbsd10)))] -mod imp { - use crate::io::{Error, Result}; - use crate::ptr; - - pub fn syscall(v: &mut [u8]) -> Result<()> { - let mib = [libc::CTL_KERN, libc::KERN_ARND]; - // kern.arandom permits a maximum buffer size of 256 bytes - for s in v.chunks_mut(256) { - let mut s_len = s.len(); - let ret = unsafe { - libc::sysctl( - mib.as_ptr(), - mib.len() as libc::c_uint, - s.as_mut_ptr() as *mut _, - &mut s_len, - ptr::null(), - 0, - ) - }; - if ret == -1 { - return Err(Error::last_os_error()); - } else if s_len != s.len() { - // FIXME(joboet): this can't actually happen, can it? - panic!("read less bytes than requested from kern.arandom"); - } - } - - Ok(()) - } -} - -#[cfg(target_os = "fuchsia")] -mod imp { - use crate::io::Result; - - #[link(name = "zircon")] - extern "C" { - fn zx_cprng_draw(buffer: *mut u8, len: usize); - } - - pub fn syscall(v: &mut [u8]) -> Result<()> { - unsafe { zx_cprng_draw(v.as_mut_ptr(), v.len()) }; - Ok(()) - } -} - -#[cfg(target_os = "vxworks")] -mod imp { - use core::sync::atomic::AtomicBool; - use core::sync::atomic::Ordering::Relaxed; - - use crate::io::{Error, Result}; - - pub fn syscall(v: &mut [u8]) -> Result<()> { - static RNG_INIT: AtomicBool = AtomicBool::new(false); - while !RNG_INIT.load(Relaxed) { - let ret = unsafe { libc::randSecure() }; - if ret < 0 { - return Err(Error::last_os_error()); - } else if ret > 0 { - RNG_INIT.store(true, Relaxed); - break; - } - - unsafe { libc::usleep(10) }; - } - - let ret = unsafe { - libc::randABytes(v.as_mut_ptr() as *mut libc::c_uchar, v.len() as libc::c_int) - }; - if ret >= 0 { Ok(()) } else { Err(Error::last_os_error()) } - } -} diff --git a/library/std/src/sys/pal/unix/stack_overflow.rs b/library/std/src/sys/pal/unix/stack_overflow.rs index 9ff44b54c41a1..69b31da427fcb 100644 --- a/library/std/src/sys/pal/unix/stack_overflow.rs +++ b/library/std/src/sys/pal/unix/stack_overflow.rs @@ -32,24 +32,24 @@ impl Drop for Handler { target_os = "macos", target_os = "netbsd", target_os = "openbsd", - target_os = "solaris" + target_os = "solaris", + target_os = "illumos", ))] mod imp { + use libc::{ + MAP_ANON, MAP_FAILED, MAP_FIXED, MAP_PRIVATE, PROT_NONE, PROT_READ, PROT_WRITE, SA_ONSTACK, + SA_SIGINFO, SIG_DFL, SIGBUS, SIGSEGV, SS_DISABLE, sigaction, sigaltstack, sighandler_t, + }; #[cfg(not(all(target_os = "linux", target_env = "gnu")))] use libc::{mmap as mmap64, mprotect, munmap}; #[cfg(all(target_os = "linux", target_env = "gnu"))] use libc::{mmap64, mprotect, munmap}; - use libc::{ - sigaction, sigaltstack, sighandler_t, MAP_ANON, MAP_FAILED, MAP_FIXED, MAP_PRIVATE, - PROT_NONE, PROT_READ, PROT_WRITE, SA_ONSTACK, SA_SIGINFO, SIGBUS, SIGSEGV, SIG_DFL, - SS_DISABLE, - }; use super::Handler; use crate::cell::Cell; use crate::ops::Range; - use crate::sync::atomic::{AtomicBool, AtomicPtr, AtomicUsize, Ordering}; use crate::sync::OnceLock; + use crate::sync::atomic::{AtomicBool, AtomicPtr, AtomicUsize, Ordering}; use crate::sys::pal::unix::os; use crate::{io, mem, ptr, thread}; @@ -265,9 +265,7 @@ mod imp { /// Modern kernels on modern hardware can have dynamic signal stack sizes. #[cfg(any(target_os = "linux", target_os = "android"))] fn sigstack_size() -> usize { - // FIXME: reuse const from libc when available? - const AT_MINSIGSTKSZ: crate::ffi::c_ulong = 51; - let dynamic_sigstksz = unsafe { libc::getauxval(AT_MINSIGSTKSZ) }; + let dynamic_sigstksz = unsafe { libc::getauxval(libc::AT_MINSIGSTKSZ) }; // If getauxval couldn't find the entry, it returns 0, // so take the higher of the "constant" and auxval. // This transparently supports older kernels which don't provide AT_MINSIGSTKSZ @@ -280,7 +278,7 @@ mod imp { libc::SIGSTKSZ } - #[cfg(target_os = "solaris")] + #[cfg(any(target_os = "solaris", target_os = "illumos"))] unsafe fn get_stack_start() -> Option<*mut libc::c_void> { let mut current_stack: libc::stack_t = crate::mem::zeroed(); assert_eq!(libc::stack_getbounds(&mut current_stack), 0); @@ -426,8 +424,8 @@ mod imp { match sysctlbyname.get() { Some(fcn) if unsafe { fcn(oid.as_ptr(), - ptr::addr_of_mut!(guard).cast(), - ptr::addr_of_mut!(size), + (&raw mut guard).cast(), + &raw mut size, ptr::null_mut(), 0) == 0 } => guard, @@ -486,7 +484,12 @@ mod imp { Some(guardaddr..guardaddr + page_size) } - #[cfg(any(target_os = "macos", target_os = "openbsd", target_os = "solaris"))] + #[cfg(any( + target_os = "macos", + target_os = "openbsd", + target_os = "solaris", + target_os = "illumos", + ))] // FIXME: I am probably not unsafe. unsafe fn current_guard() -> Option> { let stackptr = get_stack_start()?; @@ -569,7 +572,8 @@ mod imp { target_os = "macos", target_os = "netbsd", target_os = "openbsd", - target_os = "solaris" + target_os = "solaris", + target_os = "illumos", )))] mod imp { pub unsafe fn init() {} diff --git a/library/std/src/sys/pal/unix/thread.rs b/library/std/src/sys/pal/unix/thread.rs index c9dcc5ad97a50..040246618360f 100644 --- a/library/std/src/sys/pal/unix/thread.rs +++ b/library/std/src/sys/pal/unix/thread.rs @@ -117,13 +117,15 @@ impl Thread { pub fn set_name(name: &CStr) { const PR_SET_NAME: libc::c_int = 15; unsafe { - libc::prctl( + let res = libc::prctl( PR_SET_NAME, name.as_ptr(), 0 as libc::c_ulong, 0 as libc::c_ulong, 0 as libc::c_ulong, ); + // We have no good way of propagating errors here, but in debug-builds let's check that this actually worked. + debug_assert_eq!(res, 0); } } @@ -140,7 +142,12 @@ impl Thread { } } - #[cfg(any(target_os = "freebsd", target_os = "dragonfly", target_os = "openbsd"))] + #[cfg(any( + target_os = "freebsd", + target_os = "dragonfly", + target_os = "openbsd", + target_os = "nuttx" + ))] pub fn set_name(name: &CStr) { unsafe { libc::pthread_set_name_np(libc::pthread_self(), name.as_ptr()); @@ -253,7 +260,7 @@ impl Thread { tv_nsec: nsecs, }; secs -= ts.tv_sec as u64; - let ts_ptr = core::ptr::addr_of_mut!(ts); + let ts_ptr = &raw mut ts; if libc::nanosleep(ts_ptr, ts_ptr) == -1 { assert_eq!(os::errno(), libc::EINTR); secs += ts.tv_sec as u64; @@ -442,8 +449,8 @@ pub fn available_parallelism() -> io::Result> { libc::sysctl( mib.as_mut_ptr(), 2, - core::ptr::addr_of_mut!(cpus) as *mut _, - core::ptr::addr_of_mut!(cpus_size) as *mut _, + (&raw mut cpus) as *mut _, + (&raw mut cpus_size) as *mut _, ptr::null_mut(), 0, ) @@ -516,8 +523,8 @@ mod cgroups { use crate::borrow::Cow; use crate::ffi::OsString; - use crate::fs::{exists, File}; - use crate::io::{BufRead, BufReader, Read}; + use crate::fs::{File, exists}; + use crate::io::{BufRead, Read}; use crate::os::unix::ffi::OsStringExt; use crate::path::{Path, PathBuf}; use crate::str::from_utf8; @@ -690,7 +697,7 @@ mod cgroups { /// If the cgroupfs is a bind mount then `group_path` is adjusted to skip /// over the already-included prefix fn find_mountpoint(group_path: &Path) -> Option<(Cow<'static, str>, &Path)> { - let mut reader = BufReader::new(File::open("/proc/self/mountinfo").ok()?); + let mut reader = File::open_buffered("/proc/self/mountinfo").ok()?; let mut line = String::with_capacity(256); loop { line.clear(); @@ -747,12 +754,15 @@ unsafe fn min_stack_size(attr: *const libc::pthread_attr_t) -> usize { } // No point in looking up __pthread_get_minstack() on non-glibc platforms. -#[cfg(all(not(all(target_os = "linux", target_env = "gnu")), not(target_os = "netbsd")))] +#[cfg(all( + not(all(target_os = "linux", target_env = "gnu")), + not(any(target_os = "netbsd", target_os = "nuttx")) +))] unsafe fn min_stack_size(_: *const libc::pthread_attr_t) -> usize { libc::PTHREAD_STACK_MIN } -#[cfg(target_os = "netbsd")] +#[cfg(any(target_os = "netbsd", target_os = "nuttx"))] unsafe fn min_stack_size(_: *const libc::pthread_attr_t) -> usize { static STACK: crate::sync::OnceLock = crate::sync::OnceLock::new(); diff --git a/library/std/src/sys/pal/unix/thread_parking.rs b/library/std/src/sys/pal/unix/thread_parking.rs index 1da5fce3cd30f..72dd2031479ee 100644 --- a/library/std/src/sys/pal/unix/thread_parking.rs +++ b/library/std/src/sys/pal/unix/thread_parking.rs @@ -2,7 +2,7 @@ // separate modules for each platform. #![cfg(target_os = "netbsd")] -use libc::{_lwp_self, c_long, clockid_t, lwpid_t, time_t, timespec, CLOCK_MONOTONIC}; +use libc::{_lwp_self, CLOCK_MONOTONIC, c_long, clockid_t, lwpid_t, time_t, timespec}; use crate::ffi::{c_int, c_void}; use crate::ptr; diff --git a/library/std/src/sys/pal/unsupported/common.rs b/library/std/src/sys/pal/unsupported/common.rs index 76f80291f0ea8..34a766683830d 100644 --- a/library/std/src/sys/pal/unsupported/common.rs +++ b/library/std/src/sys/pal/unsupported/common.rs @@ -27,7 +27,3 @@ pub fn decode_error_kind(_code: i32) -> crate::io::ErrorKind { pub fn abort_internal() -> ! { core::intrinsics::abort(); } - -pub fn hashmap_random_keys() -> (u64, u64) { - (1, 2) -} diff --git a/library/std/src/sys/pal/unsupported/process.rs b/library/std/src/sys/pal/unsupported/process.rs index 40231bfc90b1e..fee81744f09ec 100644 --- a/library/std/src/sys/pal/unsupported/process.rs +++ b/library/std/src/sys/pal/unsupported/process.rs @@ -255,11 +255,11 @@ impl ExitStatusError { } #[derive(PartialEq, Eq, Clone, Copy, Debug)] -pub struct ExitCode(bool); +pub struct ExitCode(u8); impl ExitCode { - pub const SUCCESS: ExitCode = ExitCode(false); - pub const FAILURE: ExitCode = ExitCode(true); + pub const SUCCESS: ExitCode = ExitCode(0); + pub const FAILURE: ExitCode = ExitCode(1); pub fn as_i32(&self) -> i32 { self.0 as i32 @@ -268,10 +268,7 @@ impl ExitCode { impl From for ExitCode { fn from(code: u8) -> Self { - match code { - 0 => Self::SUCCESS, - 1..=255 => Self::FAILURE, - } + Self(code) } } diff --git a/library/std/src/sys/pal/wasi/fs.rs b/library/std/src/sys/pal/wasi/fs.rs index 88b1e543ec7c2..59ecc45b06a1f 100644 --- a/library/std/src/sys/pal/wasi/fs.rs +++ b/library/std/src/sys/pal/wasi/fs.rs @@ -13,7 +13,7 @@ use crate::sys::common::small_c_string::run_path_with_cstr; use crate::sys::time::SystemTime; use crate::sys::unsupported; pub use crate::sys_common::fs::exists; -use crate::sys_common::{ignore_notfound, AsInner, FromInner, IntoInner}; +use crate::sys_common::{AsInner, FromInner, IntoInner, ignore_notfound}; use crate::{fmt, iter, ptr}; pub struct File { @@ -265,7 +265,7 @@ impl OpenOptions { pub fn new() -> OpenOptions { let mut base = OpenOptions::default(); base.dirflags = wasi::LOOKUPFLAGS_SYMLINK_FOLLOW; - return base; + base } pub fn read(&mut self, read: bool) { @@ -382,7 +382,7 @@ impl OpenOptions { base |= wasi::RIGHTS_PATH_UNLINK_FILE; base |= wasi::RIGHTS_POLL_FD_READWRITE; - return base; + base } fn rights_inheriting(&self) -> wasi::Rights { diff --git a/library/std/src/sys/pal/wasi/helpers.rs b/library/std/src/sys/pal/wasi/helpers.rs index d047bf2fce857..404747f0dc756 100644 --- a/library/std/src/sys/pal/wasi/helpers.rs +++ b/library/std/src/sys/pal/wasi/helpers.rs @@ -1,6 +1,6 @@ #![forbid(unsafe_op_in_unsafe_fn)] -use crate::{io as std_io, mem}; +use crate::io as std_io; #[inline] pub fn is_interrupted(errno: i32) -> bool { @@ -108,16 +108,6 @@ pub fn abort_internal() -> ! { unsafe { libc::abort() } } -pub fn hashmap_random_keys() -> (u64, u64) { - let mut ret = (0u64, 0u64); - unsafe { - let base = &mut ret as *mut (u64, u64) as *mut u8; - let len = mem::size_of_val(&ret); - wasi::random_get(base, len).expect("random_get failure"); - } - return ret; -} - #[inline] pub(crate) fn err2io(err: wasi::Errno) -> std_io::Error { std_io::Error::from_raw_os_error(err.raw().into()) diff --git a/library/std/src/sys/pal/wasi/mod.rs b/library/std/src/sys/pal/wasi/mod.rs index 8051021a58897..5d54c7903065c 100644 --- a/library/std/src/sys/pal/wasi/mod.rs +++ b/library/std/src/sys/pal/wasi/mod.rs @@ -47,4 +47,4 @@ mod helpers; // then the compiler complains about conflicts. use helpers::err2io; -pub use helpers::{abort_internal, decode_error_kind, hashmap_random_keys, is_interrupted}; +pub use helpers::{abort_internal, decode_error_kind, is_interrupted}; diff --git a/library/std/src/sys/pal/wasip2/mod.rs b/library/std/src/sys/pal/wasip2/mod.rs index 546fadbe5011c..320712fdcc9fe 100644 --- a/library/std/src/sys/pal/wasip2/mod.rs +++ b/library/std/src/sys/pal/wasip2/mod.rs @@ -20,7 +20,6 @@ pub mod futex; #[path = "../wasi/io.rs"] pub mod io; -#[path = "../wasi/net.rs"] pub mod net; #[path = "../wasi/os.rs"] pub mod os; @@ -50,6 +49,6 @@ mod helpers; // then the compiler complains about conflicts. use helpers::err2io; -pub use helpers::{abort_internal, decode_error_kind, hashmap_random_keys, is_interrupted}; +pub use helpers::{abort_internal, decode_error_kind, is_interrupted}; mod cabi_realloc; diff --git a/library/std/src/sys/pal/wasip2/net.rs b/library/std/src/sys/pal/wasip2/net.rs new file mode 100644 index 0000000000000..06e623df8438e --- /dev/null +++ b/library/std/src/sys/pal/wasip2/net.rs @@ -0,0 +1,417 @@ +#![deny(unsafe_op_in_unsafe_fn)] + +use libc::{c_int, c_void, size_t}; + +use crate::ffi::CStr; +use crate::io::{self, BorrowedBuf, BorrowedCursor, IoSlice, IoSliceMut}; +use crate::net::{Shutdown, SocketAddr}; +use crate::os::wasi::io::{AsFd, AsRawFd, BorrowedFd, FromRawFd, IntoRawFd, OwnedFd, RawFd}; +use crate::sys::unsupported; +use crate::sys_common::net::{getsockopt, setsockopt, sockaddr_to_addr}; +use crate::sys_common::{AsInner, FromInner, IntoInner}; +use crate::time::{Duration, Instant}; +use crate::{cmp, mem, str}; + +pub extern crate libc as netc; + +#[allow(non_camel_case_types)] +pub type wrlen_t = size_t; + +#[doc(hidden)] +pub trait IsMinusOne { + fn is_minus_one(&self) -> bool; +} + +macro_rules! impl_is_minus_one { + ($($t:ident)*) => ($(impl IsMinusOne for $t { + fn is_minus_one(&self) -> bool { + *self == -1 + } + })*) +} + +impl_is_minus_one! { i8 i16 i32 i64 isize } + +pub fn cvt(t: T) -> crate::io::Result { + if t.is_minus_one() { Err(crate::io::Error::last_os_error()) } else { Ok(t) } +} + +pub fn cvt_r(mut f: F) -> crate::io::Result +where + T: IsMinusOne, + F: FnMut() -> T, +{ + loop { + match cvt(f()) { + Err(ref e) if e.is_interrupted() => {} + other => return other, + } + } +} + +pub fn cvt_gai(err: c_int) -> io::Result<()> { + if err == 0 { + return Ok(()); + } + + if err == netc::EAI_SYSTEM { + return Err(io::Error::last_os_error()); + } + + let detail = unsafe { + str::from_utf8(CStr::from_ptr(netc::gai_strerror(err)).to_bytes()).unwrap().to_owned() + }; + + Err(io::Error::new( + io::ErrorKind::Uncategorized, + &format!("failed to lookup address information: {detail}")[..], + )) +} + +pub fn init() {} + +pub struct WasiSocket(OwnedFd); + +pub struct Socket(WasiSocket); + +impl Socket { + pub fn new(addr: &SocketAddr, ty: c_int) -> io::Result { + let fam = match *addr { + SocketAddr::V4(..) => netc::AF_INET, + SocketAddr::V6(..) => netc::AF_INET6, + }; + Socket::new_raw(fam, ty) + } + + pub fn new_raw(fam: c_int, ty: c_int) -> io::Result { + let fd = cvt(unsafe { netc::socket(fam, ty, 0) })?; + Ok(unsafe { Self::from_raw_fd(fd) }) + } + + pub fn connect(&self, addr: &SocketAddr) -> io::Result<()> { + let (addr, len) = addr.into_inner(); + cvt_r(|| unsafe { netc::connect(self.as_raw_fd(), addr.as_ptr(), len) })?; + Ok(()) + } + + pub fn connect_timeout(&self, addr: &SocketAddr, timeout: Duration) -> io::Result<()> { + self.set_nonblocking(true)?; + let r = self.connect(addr); + self.set_nonblocking(false)?; + + match r { + Ok(_) => return Ok(()), + // there's no ErrorKind for EINPROGRESS + Err(ref e) if e.raw_os_error() == Some(netc::EINPROGRESS) => {} + Err(e) => return Err(e), + } + + let mut pollfd = netc::pollfd { fd: self.as_raw_fd(), events: netc::POLLOUT, revents: 0 }; + + if timeout.as_secs() == 0 && timeout.subsec_nanos() == 0 { + return Err(io::Error::ZERO_TIMEOUT); + } + + let start = Instant::now(); + + loop { + let elapsed = start.elapsed(); + if elapsed >= timeout { + return Err(io::const_io_error!(io::ErrorKind::TimedOut, "connection timed out")); + } + + let timeout = timeout - elapsed; + let mut timeout = timeout + .as_secs() + .saturating_mul(1_000) + .saturating_add(timeout.subsec_nanos() as u64 / 1_000_000); + if timeout == 0 { + timeout = 1; + } + + let timeout = cmp::min(timeout, c_int::MAX as u64) as c_int; + + match unsafe { netc::poll(&mut pollfd, 1, timeout) } { + -1 => { + let err = io::Error::last_os_error(); + if !err.is_interrupted() { + return Err(err); + } + } + 0 => {} + _ => { + // WASI poll does not return POLLHUP or POLLERR in revents. Check if the + // connnection actually succeeded and return ok only when the socket is + // ready and no errors were found. + if let Some(e) = self.take_error()? { + return Err(e); + } + + return Ok(()); + } + } + } + } + + pub fn accept( + &self, + storage: *mut netc::sockaddr, + len: *mut netc::socklen_t, + ) -> io::Result { + let fd = cvt_r(|| unsafe { netc::accept(self.as_raw_fd(), storage, len) })?; + Ok(unsafe { Self::from_raw_fd(fd) }) + } + + pub fn duplicate(&self) -> io::Result { + unsupported() + } + + fn recv_with_flags(&self, mut buf: BorrowedCursor<'_>, flags: c_int) -> io::Result<()> { + let ret = cvt(unsafe { + netc::recv( + self.as_raw_fd(), + buf.as_mut().as_mut_ptr() as *mut c_void, + buf.capacity(), + flags, + ) + })?; + unsafe { + buf.advance_unchecked(ret as usize); + } + Ok(()) + } + + pub fn read(&self, buf: &mut [u8]) -> io::Result { + let mut buf = BorrowedBuf::from(buf); + self.recv_with_flags(buf.unfilled(), 0)?; + Ok(buf.len()) + } + + pub fn peek(&self, buf: &mut [u8]) -> io::Result { + let mut buf = BorrowedBuf::from(buf); + self.recv_with_flags(buf.unfilled(), netc::MSG_PEEK)?; + Ok(buf.len()) + } + + pub fn read_buf(&self, buf: BorrowedCursor<'_>) -> io::Result<()> { + self.recv_with_flags(buf, 0) + } + + pub fn read_vectored(&self, bufs: &mut [IoSliceMut<'_>]) -> io::Result { + io::default_read_vectored(|b| self.read(b), bufs) + } + + #[inline] + pub fn is_read_vectored(&self) -> bool { + false + } + + fn recv_from_with_flags( + &self, + buf: &mut [u8], + flags: c_int, + ) -> io::Result<(usize, SocketAddr)> { + let mut storage: netc::sockaddr_storage = unsafe { mem::zeroed() }; + let mut addrlen = mem::size_of_val(&storage) as netc::socklen_t; + + let n = cvt(unsafe { + netc::recvfrom( + self.as_raw_fd(), + buf.as_mut_ptr() as *mut c_void, + buf.len(), + flags, + core::ptr::addr_of_mut!(storage) as *mut _, + &mut addrlen, + ) + })?; + Ok((n as usize, sockaddr_to_addr(&storage, addrlen as usize)?)) + } + + pub fn recv_from(&self, buf: &mut [u8]) -> io::Result<(usize, SocketAddr)> { + self.recv_from_with_flags(buf, 0) + } + + pub fn peek_from(&self, buf: &mut [u8]) -> io::Result<(usize, SocketAddr)> { + self.recv_from_with_flags(buf, netc::MSG_PEEK) + } + + fn write(&self, buf: &[u8]) -> io::Result { + let len = cmp::min(buf.len(), ::MAX as usize) as wrlen_t; + let ret = cvt(unsafe { + netc::send(self.as_raw(), buf.as_ptr() as *const c_void, len, netc::MSG_NOSIGNAL) + })?; + Ok(ret as usize) + } + + pub fn write_vectored(&self, bufs: &[IoSlice<'_>]) -> io::Result { + io::default_write_vectored(|b| self.write(b), bufs) + } + + #[inline] + pub fn is_write_vectored(&self) -> bool { + false + } + + pub fn set_timeout(&self, dur: Option, kind: c_int) -> io::Result<()> { + let timeout = match dur { + Some(dur) => { + if dur.as_secs() == 0 && dur.subsec_nanos() == 0 { + return Err(io::Error::ZERO_TIMEOUT); + } + + let secs = dur.as_secs().try_into().unwrap_or(netc::time_t::MAX); + let mut timeout = netc::timeval { + tv_sec: secs, + tv_usec: dur.subsec_micros() as netc::suseconds_t, + }; + if timeout.tv_sec == 0 && timeout.tv_usec == 0 { + timeout.tv_usec = 1; + } + timeout + } + None => netc::timeval { tv_sec: 0, tv_usec: 0 }, + }; + setsockopt(self, netc::SOL_SOCKET, kind, timeout) + } + + pub fn timeout(&self, kind: c_int) -> io::Result> { + let raw: netc::timeval = getsockopt(self, netc::SOL_SOCKET, kind)?; + if raw.tv_sec == 0 && raw.tv_usec == 0 { + Ok(None) + } else { + let sec = raw.tv_sec as u64; + let nsec = (raw.tv_usec as u32) * 1000; + Ok(Some(Duration::new(sec, nsec))) + } + } + + pub fn shutdown(&self, how: Shutdown) -> io::Result<()> { + let how = match how { + Shutdown::Write => netc::SHUT_WR, + Shutdown::Read => netc::SHUT_RD, + Shutdown::Both => netc::SHUT_RDWR, + }; + cvt(unsafe { netc::shutdown(self.as_raw_fd(), how) })?; + Ok(()) + } + + pub fn set_linger(&self, _linger: Option) -> io::Result<()> { + unsupported() + } + + pub fn linger(&self) -> io::Result> { + unsupported() + } + + pub fn set_nodelay(&self, nodelay: bool) -> io::Result<()> { + setsockopt(self, netc::IPPROTO_TCP, netc::TCP_NODELAY, nodelay as c_int) + } + + pub fn nodelay(&self) -> io::Result { + let raw: c_int = getsockopt(self, netc::IPPROTO_TCP, netc::TCP_NODELAY)?; + Ok(raw != 0) + } + + pub fn set_nonblocking(&self, nonblocking: bool) -> io::Result<()> { + let mut nonblocking = nonblocking as c_int; + cvt(unsafe { netc::ioctl(self.as_raw_fd(), netc::FIONBIO, &mut nonblocking) }).map(drop) + } + + pub fn take_error(&self) -> io::Result> { + let raw: c_int = getsockopt(self, netc::SOL_SOCKET, netc::SO_ERROR)?; + if raw == 0 { Ok(None) } else { Ok(Some(io::Error::from_raw_os_error(raw as i32))) } + } + + // This is used by sys_common code to abstract over Windows and Unix. + pub fn as_raw(&self) -> RawFd { + self.as_raw_fd() + } +} + +impl AsInner for WasiSocket { + #[inline] + fn as_inner(&self) -> &OwnedFd { + &self.0 + } +} + +impl IntoInner for WasiSocket { + fn into_inner(self) -> OwnedFd { + self.0 + } +} + +impl FromInner for WasiSocket { + fn from_inner(owned_fd: OwnedFd) -> Self { + Self(owned_fd) + } +} + +impl AsFd for WasiSocket { + fn as_fd(&self) -> BorrowedFd<'_> { + self.0.as_fd() + } +} + +impl AsRawFd for WasiSocket { + #[inline] + fn as_raw_fd(&self) -> RawFd { + self.0.as_raw_fd() + } +} + +impl IntoRawFd for WasiSocket { + fn into_raw_fd(self) -> RawFd { + self.0.into_raw_fd() + } +} + +impl FromRawFd for WasiSocket { + unsafe fn from_raw_fd(raw_fd: RawFd) -> Self { + unsafe { Self(FromRawFd::from_raw_fd(raw_fd)) } + } +} + +impl AsInner for Socket { + #[inline] + fn as_inner(&self) -> &WasiSocket { + &self.0 + } +} + +impl IntoInner for Socket { + fn into_inner(self) -> WasiSocket { + self.0 + } +} + +impl FromInner for Socket { + fn from_inner(sock: WasiSocket) -> Socket { + Socket(sock) + } +} + +impl AsFd for Socket { + fn as_fd(&self) -> BorrowedFd<'_> { + self.0.as_fd() + } +} + +impl AsRawFd for Socket { + #[inline] + fn as_raw_fd(&self) -> RawFd { + self.0.as_raw_fd() + } +} + +impl IntoRawFd for Socket { + fn into_raw_fd(self) -> RawFd { + self.0.into_raw_fd() + } +} + +impl FromRawFd for Socket { + unsafe fn from_raw_fd(raw_fd: RawFd) -> Self { + unsafe { Self(FromRawFd::from_raw_fd(raw_fd)) } + } +} diff --git a/library/std/src/sys/pal/wasm/atomics/futex.rs b/library/std/src/sys/pal/wasm/atomics/futex.rs index 42913a99ee9d6..bdad0da73f0a5 100644 --- a/library/std/src/sys/pal/wasm/atomics/futex.rs +++ b/library/std/src/sys/pal/wasm/atomics/futex.rs @@ -6,9 +6,14 @@ use core::arch::wasm64 as wasm; use crate::sync::atomic::AtomicU32; use crate::time::Duration; +/// An atomic for use as a futex that is at least 32-bits but may be larger +pub type Futex = AtomicU32; +/// Must be the underlying type of Futex +pub type Primitive = u32; + /// An atomic for use as a futex that is at least 8-bits but may be larger. -pub type SmallAtomic = AtomicU32; -/// Must be the underlying type of SmallAtomic +pub type SmallFutex = AtomicU32; +/// Must be the underlying type of SmallFutex pub type SmallPrimitive = u32; /// Wait for a futex_wake operation to wake us. diff --git a/library/std/src/sys/pal/windows/api.rs b/library/std/src/sys/pal/windows/api.rs index 9e336ff2d473d..ebe207fde935c 100644 --- a/library/std/src/sys/pal/windows/api.rs +++ b/library/std/src/sys/pal/windows/api.rs @@ -30,7 +30,6 @@ //! should go in sys/pal/windows/mod.rs rather than here. See `IoResult` as an example. use core::ffi::c_void; -use core::ptr::addr_of; use super::c; @@ -186,7 +185,7 @@ unsafe trait SizedSetFileInformation: Sized { unsafe impl SetFileInformation for T { const CLASS: i32 = T::CLASS; fn as_ptr(&self) -> *const c_void { - addr_of!(*self).cast::() + (&raw const *self).cast::() } fn size(&self) -> u32 { win32_size_of::() diff --git a/library/std/src/sys/pal/windows/args.rs b/library/std/src/sys/pal/windows/args.rs index 66e75a8357149..e9fc19bcb99c1 100644 --- a/library/std/src/sys/pal/windows/args.rs +++ b/library/std/src/sys/pal/windows/args.rs @@ -14,21 +14,10 @@ use crate::path::{Path, PathBuf}; use crate::sys::path::get_long_path; use crate::sys::process::ensure_no_nuls; use crate::sys::{c, to_u16s}; -use crate::sys_common::wstr::WStrUnits; use crate::sys_common::AsInner; +use crate::sys_common::wstr::WStrUnits; use crate::{fmt, io, iter, vec}; -/// This is the const equivalent to `NonZero::new(n).unwrap()` -/// -/// FIXME: This can be removed once `Option::unwrap` is stably const. -/// See the `const_option` feature (#67441). -const fn non_zero_u16(n: u16) -> NonZero { - match NonZero::new(n) { - Some(n) => n, - None => panic!("called `unwrap` on a `None` value"), - } -} - pub fn args() -> Args { // SAFETY: `GetCommandLineW` returns a pointer to a null terminated UTF-16 // string so it's safe for `WStrUnits` to use. @@ -66,10 +55,10 @@ fn parse_lp_cmd_line<'a, F: Fn() -> OsString>( lp_cmd_line: Option>, exe_name: F, ) -> Vec { - const BACKSLASH: NonZero = non_zero_u16(b'\\' as u16); - const QUOTE: NonZero = non_zero_u16(b'"' as u16); - const TAB: NonZero = non_zero_u16(b'\t' as u16); - const SPACE: NonZero = non_zero_u16(b' ' as u16); + const BACKSLASH: NonZero = NonZero::new(b'\\' as u16).unwrap(); + const QUOTE: NonZero = NonZero::new(b'"' as u16).unwrap(); + const TAB: NonZero = NonZero::new(b'\t' as u16).unwrap(); + const SPACE: NonZero = NonZero::new(b' ' as u16).unwrap(); let mut ret_val = Vec::new(); // If the cmd line pointer is null or it points to an empty string then diff --git a/library/std/src/sys/pal/windows/args/tests.rs b/library/std/src/sys/pal/windows/args/tests.rs index 484a90ab056df..6d5c953cbd5c6 100644 --- a/library/std/src/sys/pal/windows/args/tests.rs +++ b/library/std/src/sys/pal/windows/args/tests.rs @@ -47,10 +47,10 @@ fn whitespace_behavior() { fn genius_quotes() { chk(r#"EXE "" """#, &["EXE", "", ""]); chk(r#"EXE "" """"#, &["EXE", "", r#"""#]); - chk( - r#"EXE "this is """all""" in the same argument""#, - &["EXE", r#"this is "all" in the same argument"#], - ); + chk(r#"EXE "this is """all""" in the same argument""#, &[ + "EXE", + r#"this is "all" in the same argument"#, + ]); chk(r#"EXE "a"""#, &["EXE", r#"a""#]); chk(r#"EXE "a"" a"#, &["EXE", r#"a" a"#]); // quotes cannot be escaped in command names diff --git a/library/std/src/sys/pal/windows/c.rs b/library/std/src/sys/pal/windows/c.rs index b888eb7d95ca3..9ce3e912caf1b 100644 --- a/library/std/src/sys/pal/windows/c.rs +++ b/library/std/src/sys/pal/windows/c.rs @@ -5,7 +5,7 @@ #![unstable(issue = "none", feature = "windows_c")] #![allow(clippy::style)] -use core::ffi::{c_uint, c_ulong, c_ushort, c_void, CStr}; +use core::ffi::{CStr, c_uint, c_ulong, c_ushort, c_void}; use core::{mem, ptr}; mod windows_sys; @@ -175,9 +175,9 @@ extern "system" { pub fn WakeByAddressAll(address: *const c_void); } +// These are loaded by `load_synch_functions`. #[cfg(target_vendor = "win7")] compat_fn_optional! { - crate::sys::compat::load_synch_functions(); pub fn WaitOnAddress( address: *const c_void, compareaddress: *const c_void, diff --git a/library/std/src/sys/pal/windows/c/bindings.txt b/library/std/src/sys/pal/windows/c/bindings.txt index 9c2e4500da068..192c95fd203c6 100644 --- a/library/std/src/sys/pal/windows/c/bindings.txt +++ b/library/std/src/sys/pal/windows/c/bindings.txt @@ -2337,7 +2337,9 @@ Windows.Win32.Storage.FileSystem.FileStandardInfo Windows.Win32.Storage.FileSystem.FileStorageInfo Windows.Win32.Storage.FileSystem.FileStreamInfo Windows.Win32.Storage.FileSystem.FindClose -Windows.Win32.Storage.FileSystem.FindFirstFileW +Windows.Win32.Storage.FileSystem.FindExInfoBasic +Windows.Win32.Storage.FileSystem.FindExSearchNameMatch +Windows.Win32.Storage.FileSystem.FindFirstFileExW Windows.Win32.Storage.FileSystem.FindNextFileW Windows.Win32.Storage.FileSystem.FlushFileBuffers Windows.Win32.Storage.FileSystem.GetFileAttributesW diff --git a/library/std/src/sys/pal/windows/c/windows_sys.rs b/library/std/src/sys/pal/windows/c/windows_sys.rs index ab5f8919d7af6..52444c2c009ed 100644 --- a/library/std/src/sys/pal/windows/c/windows_sys.rs +++ b/library/std/src/sys/pal/windows/c/windows_sys.rs @@ -26,7 +26,7 @@ windows_targets::link!("kernel32.dll" "system" fn DeviceIoControl(hdevice : HAND windows_targets::link!("kernel32.dll" "system" fn DuplicateHandle(hsourceprocesshandle : HANDLE, hsourcehandle : HANDLE, htargetprocesshandle : HANDLE, lptargethandle : *mut HANDLE, dwdesiredaccess : u32, binherithandle : BOOL, dwoptions : DUPLICATE_HANDLE_OPTIONS) -> BOOL); windows_targets::link!("kernel32.dll" "system" fn ExitProcess(uexitcode : u32) -> !); windows_targets::link!("kernel32.dll" "system" fn FindClose(hfindfile : HANDLE) -> BOOL); -windows_targets::link!("kernel32.dll" "system" fn FindFirstFileW(lpfilename : PCWSTR, lpfindfiledata : *mut WIN32_FIND_DATAW) -> HANDLE); +windows_targets::link!("kernel32.dll" "system" fn FindFirstFileExW(lpfilename : PCWSTR, finfolevelid : FINDEX_INFO_LEVELS, lpfindfiledata : *mut core::ffi::c_void, fsearchop : FINDEX_SEARCH_OPS, lpsearchfilter : *const core::ffi::c_void, dwadditionalflags : FIND_FIRST_EX_FLAGS) -> HANDLE); windows_targets::link!("kernel32.dll" "system" fn FindNextFileW(hfindfile : HANDLE, lpfindfiledata : *mut WIN32_FIND_DATAW) -> BOOL); windows_targets::link!("kernel32.dll" "system" fn FlushFileBuffers(hfile : HANDLE) -> BOOL); windows_targets::link!("kernel32.dll" "system" fn FormatMessageW(dwflags : FORMAT_MESSAGE_OPTIONS, lpsource : *const core::ffi::c_void, dwmessageid : u32, dwlanguageid : u32, lpbuffer : PWSTR, nsize : u32, arguments : *const *const i8) -> u32); @@ -2501,6 +2501,9 @@ pub const FILE_WRITE_ATTRIBUTES: FILE_ACCESS_RIGHTS = 256u32; pub const FILE_WRITE_DATA: FILE_ACCESS_RIGHTS = 2u32; pub const FILE_WRITE_EA: FILE_ACCESS_RIGHTS = 16u32; pub const FILE_WRITE_THROUGH: NTCREATEFILE_CREATE_OPTIONS = 2u32; +pub type FINDEX_INFO_LEVELS = i32; +pub type FINDEX_SEARCH_OPS = i32; +pub type FIND_FIRST_EX_FLAGS = u32; pub const FIONBIO: i32 = -2147195266i32; #[repr(C)] #[cfg(any(target_arch = "aarch64", target_arch = "arm64ec", target_arch = "x86_64"))] @@ -2565,6 +2568,8 @@ pub const FileRenameInfoEx: FILE_INFO_BY_HANDLE_CLASS = 22i32; pub const FileStandardInfo: FILE_INFO_BY_HANDLE_CLASS = 1i32; pub const FileStorageInfo: FILE_INFO_BY_HANDLE_CLASS = 16i32; pub const FileStreamInfo: FILE_INFO_BY_HANDLE_CLASS = 7i32; +pub const FindExInfoBasic: FINDEX_INFO_LEVELS = 1i32; +pub const FindExSearchNameMatch: FINDEX_SEARCH_OPS = 0i32; pub type GENERIC_ACCESS_RIGHTS = u32; pub const GENERIC_ALL: GENERIC_ACCESS_RIGHTS = 268435456u32; pub const GENERIC_EXECUTE: GENERIC_ACCESS_RIGHTS = 536870912u32; @@ -3307,7 +3312,6 @@ pub struct XSAVE_FORMAT { pub XmmRegisters: [M128A; 8], pub Reserved4: [u8; 224], } - #[cfg(target_arch = "arm")] #[repr(C)] pub struct WSADATA { diff --git a/library/std/src/sys/pal/windows/compat.rs b/library/std/src/sys/pal/windows/compat.rs index 75232dfc0b0f1..42999da166451 100644 --- a/library/std/src/sys/pal/windows/compat.rs +++ b/library/std/src/sys/pal/windows/compat.rs @@ -19,7 +19,7 @@ //! function is called. In the worst case, multiple threads may all end up //! importing the same function unnecessarily. -use crate::ffi::{c_void, CStr}; +use crate::ffi::{CStr, c_void}; use crate::ptr::NonNull; use crate::sys::c; @@ -198,11 +198,10 @@ macro_rules! compat_fn_with_fallback { /// Optionally loaded functions. /// -/// Actual loading of the function defers to $load_functions. +/// Relies on the functions being pre-loaded elsewhere. #[cfg(target_vendor = "win7")] macro_rules! compat_fn_optional { - ($load_functions:expr; - $( + ($( $(#[$meta:meta])* $vis:vis fn $symbol:ident($($argname:ident: $argtype:ty),*) $(-> $rettype:ty)?; )+) => ( @@ -221,9 +220,6 @@ macro_rules! compat_fn_optional { #[inline(always)] pub fn option() -> Option { - // Miri does not understand the way we do preloading - // therefore load the function here instead. - #[cfg(miri)] $load_functions; NonNull::new(PTR.load(Ordering::Relaxed)).map(|f| unsafe { mem::transmute(f) }) } } diff --git a/library/std/src/sys/pal/windows/fs.rs b/library/std/src/sys/pal/windows/fs.rs index 5b360640c4e67..5a9bfccc1fabb 100644 --- a/library/std/src/sys/pal/windows/fs.rs +++ b/library/std/src/sys/pal/windows/fs.rs @@ -1,9 +1,7 @@ -use core::ptr::addr_of; - use super::api::{self, WinError}; -use super::{to_u16s, IoResult}; +use super::{IoResult, to_u16s}; use crate::borrow::Cow; -use crate::ffi::{c_void, OsStr, OsString}; +use crate::ffi::{OsStr, OsString, c_void}; use crate::io::{self, BorrowedCursor, Error, IoSlice, IoSliceMut, SeekFrom}; use crate::mem::{self, MaybeUninit}; use crate::os::windows::io::{AsHandle, BorrowedHandle}; @@ -13,7 +11,7 @@ use crate::sync::Arc; use crate::sys::handle::Handle; use crate::sys::path::maybe_verbatim; use crate::sys::time::SystemTime; -use crate::sys::{c, cvt, Align8}; +use crate::sys::{Align8, c, cvt}; use crate::sys_common::{AsInner, FromInner, IntoInner}; use crate::{fmt, ptr, slice}; @@ -116,7 +114,7 @@ impl Iterator for ReadDir { fn next(&mut self) -> Option> { if self.handle.0 == c::INVALID_HANDLE_VALUE { // This iterator was initialized with an `INVALID_HANDLE_VALUE` as its handle. - // Simply return `None` because this is only the case when `FindFirstFileW` in + // Simply return `None` because this is only the case when `FindFirstFileExW` in // the construction of this iterator returns `ERROR_FILE_NOT_FOUND` which means // no matchhing files can be found. return None; @@ -325,7 +323,7 @@ impl File { let result = c::SetFileInformationByHandle( handle.as_raw_handle(), c::FileEndOfFileInfo, - ptr::addr_of!(eof).cast::(), + (&raw const eof).cast::(), mem::size_of::() as u32, ); if result == 0 { @@ -364,7 +362,7 @@ impl File { cvt(c::GetFileInformationByHandleEx( self.handle.as_raw_handle(), c::FileAttributeTagInfo, - ptr::addr_of_mut!(attr_tag).cast(), + (&raw mut attr_tag).cast(), mem::size_of::().try_into().unwrap(), ))?; if attr_tag.FileAttributes & c::FILE_ATTRIBUTE_REPARSE_POINT != 0 { @@ -396,7 +394,7 @@ impl File { cvt(c::GetFileInformationByHandleEx( self.handle.as_raw_handle(), c::FileBasicInfo, - core::ptr::addr_of_mut!(info) as *mut c_void, + (&raw mut info) as *mut c_void, size as u32, ))?; let mut attr = FileAttr { @@ -428,7 +426,7 @@ impl File { cvt(c::GetFileInformationByHandleEx( self.handle.as_raw_handle(), c::FileStandardInfo, - core::ptr::addr_of_mut!(info) as *mut c_void, + (&raw mut info) as *mut c_void, size as u32, ))?; attr.file_size = info.AllocationSize as u64; @@ -438,7 +436,7 @@ impl File { cvt(c::GetFileInformationByHandleEx( self.handle.as_raw_handle(), c::FileAttributeTagInfo, - ptr::addr_of_mut!(attr_tag).cast(), + (&raw mut attr_tag).cast(), mem::size_of::().try_into().unwrap(), ))?; if attr_tag.FileAttributes & c::FILE_ATTRIBUTE_REPARSE_POINT != 0 { @@ -545,22 +543,20 @@ impl File { unsafe { let (path_buffer, subst_off, subst_len, relative) = match (*buf).ReparseTag { c::IO_REPARSE_TAG_SYMLINK => { - let info: *mut c::SYMBOLIC_LINK_REPARSE_BUFFER = - ptr::addr_of_mut!((*buf).rest).cast(); + let info: *mut c::SYMBOLIC_LINK_REPARSE_BUFFER = (&raw mut (*buf).rest).cast(); assert!(info.is_aligned()); ( - ptr::addr_of_mut!((*info).PathBuffer).cast::(), + (&raw mut (*info).PathBuffer).cast::(), (*info).SubstituteNameOffset / 2, (*info).SubstituteNameLength / 2, (*info).Flags & c::SYMLINK_FLAG_RELATIVE != 0, ) } c::IO_REPARSE_TAG_MOUNT_POINT => { - let info: *mut c::MOUNT_POINT_REPARSE_BUFFER = - ptr::addr_of_mut!((*buf).rest).cast(); + let info: *mut c::MOUNT_POINT_REPARSE_BUFFER = (&raw mut (*buf).rest).cast(); assert!(info.is_aligned()); ( - ptr::addr_of_mut!((*info).PathBuffer).cast::(), + (&raw mut (*info).PathBuffer).cast::(), (*info).SubstituteNameOffset / 2, (*info).SubstituteNameLength / 2, false, @@ -643,7 +639,7 @@ impl File { cvt(c::GetFileInformationByHandleEx( self.handle.as_raw_handle(), c::FileBasicInfo, - core::ptr::addr_of_mut!(info) as *mut c_void, + (&raw mut info) as *mut c_void, size as u32, ))?; Ok(info) @@ -790,11 +786,11 @@ impl<'a> Iterator for DirBuffIter<'a> { // it does not seem that reality is so kind, and assuming this // caused crashes in some cases (https://github.com/rust-lang/rust/issues/104530) // presumably, this can be blamed on buggy filesystem drivers, but who knows. - let next_entry = ptr::addr_of!((*info).NextEntryOffset).read_unaligned() as usize; - let length = ptr::addr_of!((*info).FileNameLength).read_unaligned() as usize; - let attrs = ptr::addr_of!((*info).FileAttributes).read_unaligned(); + let next_entry = (&raw const (*info).NextEntryOffset).read_unaligned() as usize; + let length = (&raw const (*info).FileNameLength).read_unaligned() as usize; + let attrs = (&raw const (*info).FileAttributes).read_unaligned(); let name = from_maybe_unaligned( - ptr::addr_of!((*info).FileName).cast::(), + (&raw const (*info).FileName).cast::(), length / size_of::(), ); let is_directory = (attrs & c::FILE_ATTRIBUTE_DIRECTORY) != 0; @@ -1051,8 +1047,22 @@ pub fn readdir(p: &Path) -> io::Result { let path = maybe_verbatim(&star)?; unsafe { - let mut wfd = mem::zeroed(); - let find_handle = c::FindFirstFileW(path.as_ptr(), &mut wfd); + let mut wfd: c::WIN32_FIND_DATAW = mem::zeroed(); + // this is like FindFirstFileW (see https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-findfirstfileexw), + // but with FindExInfoBasic it should skip filling WIN32_FIND_DATAW.cAlternateFileName + // (see https://learn.microsoft.com/en-us/windows/win32/api/minwinbase/ns-minwinbase-win32_find_dataw) + // (which will be always null string value and currently unused) and should be faster. + // + // We can pass FIND_FIRST_EX_LARGE_FETCH to dwAdditionalFlags to speed up things more, + // but as we don't know user's use profile of this function, lets be conservative. + let find_handle = c::FindFirstFileExW( + path.as_ptr(), + c::FindExInfoBasic, + &mut wfd as *mut _ as _, + c::FindExSearchNameMatch, + ptr::null(), + 0, + ); if find_handle != c::INVALID_HANDLE_VALUE { Ok(ReadDir { @@ -1061,7 +1071,7 @@ pub fn readdir(p: &Path) -> io::Result { first: Some(wfd), }) } else { - // The status `ERROR_FILE_NOT_FOUND` is returned by the `FindFirstFileW` function + // The status `ERROR_FILE_NOT_FOUND` is returned by the `FindFirstFileExW` function // if no matching files can be found, but not necessarily that the path to find the // files in does not exist. // @@ -1083,7 +1093,7 @@ pub fn readdir(p: &Path) -> io::Result { // Just return the error constructed from the raw OS error if the above is not the case. // - // Note: `ERROR_PATH_NOT_FOUND` would have been returned by the `FindFirstFileW` function + // Note: `ERROR_PATH_NOT_FOUND` would have been returned by the `FindFirstFileExW` function // when the path to search in does not exist in the first place. Err(Error::from_raw_os_error(last_error.code as i32)) } @@ -1149,7 +1159,7 @@ pub fn symlink_inner(original: &Path, link: &Path, dir: bool) -> io::Result<()> // Formerly, symlink creation required the SeCreateSymbolicLink privilege. For the Windows 10 // Creators Update, Microsoft loosened this to allow unprivileged symlink creation if the // computer is in Developer Mode, but SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE must be - // added to dwFlags to opt into this behaviour. + // added to dwFlags to opt into this behavior. let result = cvt(unsafe { c::CreateSymbolicLinkW( link.as_ptr(), @@ -1224,7 +1234,7 @@ fn metadata(path: &Path, reparse: ReparsePoint) -> io::Result { opts.custom_flags(c::FILE_FLAG_BACKUP_SEMANTICS | reparse.as_flag()); // Attempt to open the file normally. - // If that fails with `ERROR_SHARING_VIOLATION` then retry using `FindFirstFileW`. + // If that fails with `ERROR_SHARING_VIOLATION` then retry using `FindFirstFileExW`. // If the fallback fails for any reason we return the original error. match File::open(path, &opts) { Ok(file) => file.file_attr(), @@ -1241,13 +1251,20 @@ fn metadata(path: &Path, reparse: ReparsePoint) -> io::Result { unsafe { let path = maybe_verbatim(path)?; - // `FindFirstFileW` accepts wildcard file names. + // `FindFirstFileExW` accepts wildcard file names. // Fortunately wildcards are not valid file names and // `ERROR_SHARING_VIOLATION` means the file exists (but is locked) // therefore it's safe to assume the file name given does not // include wildcards. - let mut wfd = mem::zeroed(); - let handle = c::FindFirstFileW(path.as_ptr(), &mut wfd); + let mut wfd: c::WIN32_FIND_DATAW = mem::zeroed(); + let handle = c::FindFirstFileExW( + path.as_ptr(), + c::FindExInfoBasic, + &mut wfd as *mut _ as _, + c::FindExSearchNameMatch, + ptr::null(), + 0, + ); if handle == c::INVALID_HANDLE_VALUE { // This can fail if the user does not have read access to the @@ -1257,7 +1274,7 @@ fn metadata(path: &Path, reparse: ReparsePoint) -> io::Result { // We no longer need the find handle. c::FindClose(handle); - // `FindFirstFileW` reads the cached file information from the + // `FindFirstFileExW` reads the cached file information from the // directory. The downside is that this metadata may be outdated. let attrs = FileAttr::from(wfd); if reparse == ReparsePoint::Follow && attrs.file_type().is_symlink() { @@ -1326,7 +1343,7 @@ pub fn copy(from: &Path, to: &Path) -> io::Result { pfrom.as_ptr(), pto.as_ptr(), Some(callback), - core::ptr::addr_of_mut!(size) as *mut _, + (&raw mut size) as *mut _, ptr::null_mut(), 0, ) @@ -1405,7 +1422,7 @@ pub fn junction_point(original: &Path, link: &Path) -> io::Result<()> { cvt(c::DeviceIoControl( d.as_raw_handle(), c::FSCTL_SET_REPARSE_POINT, - addr_of!(header).cast::(), + (&raw const header).cast::(), data_len as u32 + 8, ptr::null_mut(), 0, diff --git a/library/std/src/sys/pal/windows/fs/remove_dir_all.rs b/library/std/src/sys/pal/windows/fs/remove_dir_all.rs index e7234ed8e5f56..9416049da78f8 100644 --- a/library/std/src/sys/pal/windows/fs/remove_dir_all.rs +++ b/library/std/src/sys/pal/windows/fs/remove_dir_all.rs @@ -71,10 +71,12 @@ unsafe fn nt_open_file( } /// Open the file `path` in the directory `parent`, requesting the given `access` rights. +/// `options` will be OR'd with `FILE_OPEN_REPARSE_POINT`. fn open_link_no_reparse( parent: &File, path: &[u16], access: u32, + options: u32, ) -> Result, WinError> { // This is implemented using the lower level `NtOpenFile` function as // unfortunately opening a file relative to a parent is not supported by @@ -96,7 +98,7 @@ fn open_link_no_reparse( ..c::OBJECT_ATTRIBUTES::default() }; let share = c::FILE_SHARE_DELETE | c::FILE_SHARE_READ | c::FILE_SHARE_WRITE; - let options = c::FILE_OPEN_REPARSE_POINT; + let options = c::FILE_OPEN_REPARSE_POINT | options; let result = nt_open_file(access, &object, share, options); // Retry without OBJ_DONT_REPARSE if it's not supported. @@ -128,13 +130,20 @@ fn open_link_no_reparse( } fn open_dir(parent: &File, name: &[u16]) -> Result, WinError> { - open_link_no_reparse(parent, name, c::SYNCHRONIZE | c::FILE_LIST_DIRECTORY) + // Open the directory for synchronous directory listing. + open_link_no_reparse( + parent, + name, + c::SYNCHRONIZE | c::FILE_LIST_DIRECTORY, + // "_IO_NONALERT" means that a synchronous call won't be interrupted. + c::FILE_SYNCHRONOUS_IO_NONALERT, + ) } fn delete(parent: &File, name: &[u16]) -> Result<(), WinError> { // Note that the `delete` function consumes the opened file to ensure it's // dropped immediately. See module comments for why this is important. - match open_link_no_reparse(parent, name, c::SYNCHRONIZE | c::DELETE) { + match open_link_no_reparse(parent, name, c::DELETE, 0) { Ok(Some(f)) => f.delete(), Ok(None) => Ok(()), Err(e) => Err(e), diff --git a/library/std/src/sys/pal/windows/futex.rs b/library/std/src/sys/pal/windows/futex.rs index 8c5081a607aa3..38afb8c043b3b 100644 --- a/library/std/src/sys/pal/windows/futex.rs +++ b/library/std/src/sys/pal/windows/futex.rs @@ -1,7 +1,7 @@ use core::ffi::c_void; use core::sync::atomic::{ - AtomicBool, AtomicI16, AtomicI32, AtomicI64, AtomicI8, AtomicIsize, AtomicPtr, AtomicU16, - AtomicU32, AtomicU64, AtomicU8, AtomicUsize, + AtomicBool, AtomicI8, AtomicI16, AtomicI32, AtomicI64, AtomicIsize, AtomicPtr, AtomicU8, + AtomicU16, AtomicU32, AtomicU64, AtomicUsize, }; use core::time::Duration; use core::{mem, ptr}; @@ -9,22 +9,27 @@ use core::{mem, ptr}; use super::api::{self, WinError}; use crate::sys::{c, dur2timeout}; +/// An atomic for use as a futex that is at least 32-bits but may be larger +pub type Futex = AtomicU32; +/// Must be the underlying type of Futex +pub type Primitive = u32; + /// An atomic for use as a futex that is at least 8-bits but may be larger. -pub type SmallAtomic = AtomicU8; -/// Must be the underlying type of SmallAtomic +pub type SmallFutex = AtomicU8; +/// Must be the underlying type of SmallFutex pub type SmallPrimitive = u8; -pub unsafe trait Futex {} +pub unsafe trait Futexable {} pub unsafe trait Waitable { - type Atomic; + type Futex; } macro_rules! unsafe_waitable_int { ($(($int:ty, $atomic:ty)),*$(,)?) => { $( unsafe impl Waitable for $int { - type Atomic = $atomic; + type Futex = $atomic; } - unsafe impl Futex for $atomic {} + unsafe impl Futexable for $atomic {} )* }; } @@ -42,51 +47,51 @@ unsafe_waitable_int! { (usize, AtomicUsize), } unsafe impl Waitable for *const T { - type Atomic = AtomicPtr; + type Futex = AtomicPtr; } unsafe impl Waitable for *mut T { - type Atomic = AtomicPtr; + type Futex = AtomicPtr; } -unsafe impl Futex for AtomicPtr {} +unsafe impl Futexable for AtomicPtr {} pub fn wait_on_address( - address: &W::Atomic, + address: &W::Futex, compare: W, timeout: Option, ) -> bool { unsafe { let addr = ptr::from_ref(address).cast::(); let size = mem::size_of::(); - let compare_addr = ptr::addr_of!(compare).cast::(); + let compare_addr = (&raw const compare).cast::(); let timeout = timeout.map(dur2timeout).unwrap_or(c::INFINITE); c::WaitOnAddress(addr, compare_addr, size, timeout) == c::TRUE } } -pub fn wake_by_address_single(address: &T) { +pub fn wake_by_address_single(address: &T) { unsafe { let addr = ptr::from_ref(address).cast::(); c::WakeByAddressSingle(addr); } } -pub fn wake_by_address_all(address: &T) { +pub fn wake_by_address_all(address: &T) { unsafe { let addr = ptr::from_ref(address).cast::(); c::WakeByAddressAll(addr); } } -pub fn futex_wait(futex: &W::Atomic, expected: W, timeout: Option) -> bool { +pub fn futex_wait(futex: &W::Futex, expected: W, timeout: Option) -> bool { // return false only on timeout wait_on_address(futex, expected, timeout) || api::get_last_error() != WinError::TIMEOUT } -pub fn futex_wake(futex: &T) -> bool { +pub fn futex_wake(futex: &T) -> bool { wake_by_address_single(futex); false } -pub fn futex_wake_all(futex: &T) { +pub fn futex_wake_all(futex: &T) { wake_by_address_all(futex) } diff --git a/library/std/src/sys/pal/windows/handle/tests.rs b/library/std/src/sys/pal/windows/handle/tests.rs index d836dae4c305b..0c976ed84e6ba 100644 --- a/library/std/src/sys/pal/windows/handle/tests.rs +++ b/library/std/src/sys/pal/windows/handle/tests.rs @@ -1,4 +1,4 @@ -use crate::sys::pipe::{anon_pipe, Pipes}; +use crate::sys::pipe::{Pipes, anon_pipe}; use crate::{thread, time}; /// Test the synchronous fallback for overlapped I/O. diff --git a/library/std/src/sys/pal/windows/io.rs b/library/std/src/sys/pal/windows/io.rs index 785a3f6768b70..1e7d02908f63d 100644 --- a/library/std/src/sys/pal/windows/io.rs +++ b/library/std/src/sys/pal/windows/io.rs @@ -122,7 +122,7 @@ fn msys_tty_on(handle: BorrowedHandle<'_>) -> bool { c::GetFileInformationByHandleEx( handle.as_raw_handle(), c::FileNameInfo, - core::ptr::addr_of_mut!(name_info) as *mut c_void, + (&raw mut name_info) as *mut c_void, size_of::() as u32, ) }; diff --git a/library/std/src/sys/pal/windows/mod.rs b/library/std/src/sys/pal/windows/mod.rs index 1cc9a2b7ffa98..aca69490d7a1a 100644 --- a/library/std/src/sys/pal/windows/mod.rs +++ b/library/std/src/sys/pal/windows/mod.rs @@ -1,7 +1,6 @@ #![allow(missing_docs, nonstandard_style)] #![forbid(unsafe_op_in_unsafe_fn)] -pub use self::rand::hashmap_random_keys; use crate::ffi::{OsStr, OsString}; use crate::io::ErrorKind; use crate::mem::MaybeUninit; @@ -27,7 +26,6 @@ pub mod net; pub mod os; pub mod pipe; pub mod process; -pub mod rand; pub mod stdio; pub mod thread; pub mod time; @@ -40,7 +38,7 @@ cfg_if::cfg_if! { } } -/// Map a Result to io::Result. +/// Map a [`Result`] to [`io::Result`](crate::io::Result). trait IoResult { fn io_result(self) -> crate::io::Result; } @@ -122,6 +120,7 @@ pub fn decode_error_kind(errno: i32) -> ErrorKind { c::ERROR_NOT_SAME_DEVICE => return CrossesDevices, c::ERROR_TOO_MANY_LINKS => return TooManyLinks, c::ERROR_FILENAME_EXCED_RANGE => return InvalidFilename, + c::ERROR_CANT_RESOLVE_FILENAME => return FilesystemLoop, _ => {} } @@ -139,6 +138,7 @@ pub fn decode_error_kind(errno: i32) -> ErrorKind { c::WSAEHOSTUNREACH => HostUnreachable, c::WSAENETDOWN => NetworkDown, c::WSAENETUNREACH => NetworkUnreachable, + c::WSAEDQUOT => FilesystemQuotaExceeded, _ => Uncategorized, } @@ -346,7 +346,6 @@ pub fn abort_internal() -> ! { } } -// miri is sensitive to changes here so check that miri is happy if touching this #[cfg(miri)] pub fn abort_internal() -> ! { crate::intrinsics::abort(); diff --git a/library/std/src/sys/pal/windows/net.rs b/library/std/src/sys/pal/windows/net.rs index ce995f5ed5af7..fd62d1f407c27 100644 --- a/library/std/src/sys/pal/windows/net.rs +++ b/library/std/src/sys/pal/windows/net.rs @@ -9,7 +9,7 @@ use crate::os::windows::io::{ }; use crate::sync::OnceLock; use crate::sys::c; -use crate::sys_common::{net, AsInner, FromInner, IntoInner}; +use crate::sys_common::{AsInner, FromInner, IntoInner, net}; use crate::time::Duration; use crate::{cmp, mem, ptr, sys}; @@ -27,12 +27,12 @@ pub mod netc { use crate::sys::c::{self, ADDRESS_FAMILY, ADDRINFOA, SOCKADDR, SOCKET}; // re-exports from Windows API bindings. pub use crate::sys::c::{ - bind, connect, freeaddrinfo, getpeername, getsockname, getsockopt, listen, setsockopt, - ADDRESS_FAMILY as sa_family_t, ADDRINFOA as addrinfo, IPPROTO_IP, IPPROTO_IPV6, - IPV6_ADD_MEMBERSHIP, IPV6_DROP_MEMBERSHIP, IPV6_MULTICAST_LOOP, IPV6_V6ONLY, - IP_ADD_MEMBERSHIP, IP_DROP_MEMBERSHIP, IP_MULTICAST_LOOP, IP_MULTICAST_TTL, IP_TTL, - SOCKADDR as sockaddr, SOCKADDR_STORAGE as sockaddr_storage, SOCK_DGRAM, SOCK_STREAM, - SOL_SOCKET, SO_BROADCAST, SO_RCVTIMEO, SO_SNDTIMEO, + ADDRESS_FAMILY as sa_family_t, ADDRINFOA as addrinfo, IP_ADD_MEMBERSHIP, + IP_DROP_MEMBERSHIP, IP_MULTICAST_LOOP, IP_MULTICAST_TTL, IP_TTL, IPPROTO_IP, IPPROTO_IPV6, + IPV6_ADD_MEMBERSHIP, IPV6_DROP_MEMBERSHIP, IPV6_MULTICAST_LOOP, IPV6_V6ONLY, SO_BROADCAST, + SO_RCVTIMEO, SO_SNDTIMEO, SOCK_DGRAM, SOCK_STREAM, SOCKADDR as sockaddr, + SOCKADDR_STORAGE as sockaddr_storage, SOL_SOCKET, bind, connect, freeaddrinfo, getpeername, + getsockname, getsockopt, listen, setsockopt, }; #[allow(non_camel_case_types)] @@ -390,7 +390,7 @@ impl Socket { buf.as_mut_ptr() as *mut _, length, flags, - core::ptr::addr_of_mut!(storage) as *mut _, + (&raw mut storage) as *mut _, &mut addrlen, ) }; diff --git a/library/std/src/sys/pal/windows/pipe.rs b/library/std/src/sys/pal/windows/pipe.rs index 7d1b5aca1d5fe..a8f6617c9dc8f 100644 --- a/library/std/src/sys/pal/windows/pipe.rs +++ b/library/std/src/sys/pal/windows/pipe.rs @@ -2,12 +2,13 @@ use crate::ffi::OsStr; use crate::io::{self, BorrowedCursor, IoSlice, IoSliceMut}; use crate::os::windows::prelude::*; use crate::path::Path; +use crate::random::{DefaultRandomSource, Random}; use crate::sync::atomic::AtomicUsize; use crate::sync::atomic::Ordering::Relaxed; +use crate::sys::c; use crate::sys::fs::{File, OpenOptions}; use crate::sys::handle::Handle; use crate::sys::pal::windows::api::{self, WinError}; -use crate::sys::{c, hashmap_random_keys}; use crate::sys_common::{FromInner, IntoInner}; use crate::{mem, ptr}; @@ -79,7 +80,7 @@ pub fn anon_pipe(ours_readable: bool, their_handle_inheritable: bool) -> io::Res name = format!( r"\\.\pipe\__rust_anonymous_pipe1__.{}.{}", c::GetCurrentProcessId(), - random_number() + random_number(), ); let wide_name = OsStr::new(&name).encode_wide().chain(Some(0)).collect::>(); let mut flags = c::FILE_FLAG_FIRST_PIPE_INSTANCE | c::FILE_FLAG_OVERLAPPED; @@ -214,7 +215,7 @@ fn random_number() -> usize { return N.fetch_add(1, Relaxed); } - N.store(hashmap_random_keys().0 as usize, Relaxed); + N.store(usize::random(&mut DefaultRandomSource), Relaxed); } } @@ -374,7 +375,7 @@ impl AnonPipe { let mut overlapped: c::OVERLAPPED = unsafe { crate::mem::zeroed() }; // `hEvent` is unused by `ReadFileEx` and `WriteFileEx`. // Therefore the documentation suggests using it to smuggle a pointer to the callback. - overlapped.hEvent = core::ptr::addr_of_mut!(async_result) as *mut _; + overlapped.hEvent = (&raw mut async_result) as *mut _; // Asynchronous read of the pipe. // If successful, `callback` will be called once it completes. diff --git a/library/std/src/sys/pal/windows/process.rs b/library/std/src/sys/pal/windows/process.rs index d40a537e3594a..17bb03fe7af04 100644 --- a/library/std/src/sys/pal/windows/process.rs +++ b/library/std/src/sys/pal/windows/process.rs @@ -22,8 +22,8 @@ use crate::sys::fs::{File, OpenOptions}; use crate::sys::handle::Handle; use crate::sys::pipe::{self, AnonPipe}; use crate::sys::{cvt, path, stdio}; -use crate::sys_common::process::{CommandEnv, CommandEnvs}; use crate::sys_common::IntoInner; +use crate::sys_common::process::{CommandEnv, CommandEnvs}; use crate::{cmp, env, fmt, mem, ptr}; //////////////////////////////////////////////////////////////////////////////// @@ -47,7 +47,7 @@ impl EnvKey { } } -// Comparing Windows environment variable keys[1] are behaviourally the +// Comparing Windows environment variable keys[1] are behaviorally the // composition of two operations[2]: // // 1. Case-fold both strings. This is done using a language-independent @@ -253,10 +253,10 @@ impl Command { attribute: usize, value: T, ) { - self.proc_thread_attributes.insert( - attribute, - ProcThreadAttributeValue { size: mem::size_of::(), data: Box::new(value) }, - ); + self.proc_thread_attributes.insert(attribute, ProcThreadAttributeValue { + size: mem::size_of::(), + data: Box::new(value), + }); } pub fn spawn( @@ -338,8 +338,8 @@ impl Command { // If at least one of stdin, stdout or stderr are set (i.e. are non null) // then set the `hStd` fields in `STARTUPINFO`. - // Otherwise skip this and allow the OS to apply its default behaviour. - // This provides more consistent behaviour between Win7 and Win8+. + // Otherwise skip this and allow the OS to apply its default behavior. + // This provides more consistent behavior between Win7 and Win8+. let is_set = |stdio: &Handle| !stdio.as_raw_handle().is_null(); if is_set(&stderr) || is_set(&stdout) || is_set(&stdin) { si.dwFlags |= c::STARTF_USESTDHANDLES; @@ -368,10 +368,10 @@ impl Command { StartupInfo: si, lpAttributeList: proc_thread_attribute_list.0.as_mut_ptr() as _, }; - si_ptr = core::ptr::addr_of_mut!(si_ex) as _; + si_ptr = (&raw mut si_ex) as _; } else { si.cb = mem::size_of::() as u32; - si_ptr = core::ptr::addr_of_mut!(si) as _; + si_ptr = (&raw mut si) as _; } unsafe { @@ -507,7 +507,7 @@ where Exists: FnMut(PathBuf) -> Option>, { // 1. Child paths - // This is for consistency with Rust's historic behaviour. + // This is for consistency with Rust's historic behavior. if let Some(paths) = child_paths { for path in env::split_paths(paths).filter(|p| !p.as_os_str().is_empty()) { if let Some(path) = exists(path) { @@ -953,7 +953,7 @@ fn make_proc_thread_attribute_list( // It's theoretically possible for the attribute count to exceed a u32 value. // Therefore, we ensure that we don't add more attributes than the buffer was initialized for. for (&attribute, value) in attributes.iter().take(attribute_count as usize) { - let value_ptr = core::ptr::addr_of!(*value.data) as _; + let value_ptr = (&raw const *value.data) as _; cvt(unsafe { c::UpdateProcThreadAttribute( proc_thread_attribute_list.0.as_mut_ptr() as _, diff --git a/library/std/src/sys/pal/windows/process/tests.rs b/library/std/src/sys/pal/windows/process/tests.rs index 65325fa64a077..1bcc5fa6b2048 100644 --- a/library/std/src/sys/pal/windows/process/tests.rs +++ b/library/std/src/sys/pal/windows/process/tests.rs @@ -1,4 +1,4 @@ -use super::{make_command_line, Arg}; +use super::{Arg, make_command_line}; use crate::env; use crate::ffi::{OsStr, OsString}; use crate::process::Command; @@ -191,7 +191,7 @@ fn windows_exe_resolver() { /* Some of the following tests may need to be changed if you are deliberately - changing the behaviour of `resolve_exe`. + changing the behavior of `resolve_exe`. */ let empty_paths = || None; diff --git a/library/std/src/sys/pal/windows/rand.rs b/library/std/src/sys/pal/windows/rand.rs deleted file mode 100644 index e366bb995626a..0000000000000 --- a/library/std/src/sys/pal/windows/rand.rs +++ /dev/null @@ -1,27 +0,0 @@ -use core::{mem, ptr}; - -use crate::sys::c; - -#[cfg(not(target_vendor = "win7"))] -#[inline] -pub fn hashmap_random_keys() -> (u64, u64) { - let mut v = (0, 0); - let ret = unsafe { c::ProcessPrng(ptr::addr_of_mut!(v).cast::(), mem::size_of_val(&v)) }; - // ProcessPrng is documented as always returning `TRUE`. - // https://learn.microsoft.com/en-us/windows/win32/seccng/processprng#return-value - debug_assert_eq!(ret, c::TRUE); - v -} - -#[cfg(target_vendor = "win7")] -pub fn hashmap_random_keys() -> (u64, u64) { - use crate::ffi::c_void; - use crate::io; - - let mut v = (0, 0); - let ret = unsafe { - c::RtlGenRandom(ptr::addr_of_mut!(v).cast::(), mem::size_of_val(&v) as u32) - }; - - if ret != 0 { v } else { panic!("RNG broken: {}", io::Error::last_os_error()) } -} diff --git a/library/std/src/sys/pal/windows/thread.rs b/library/std/src/sys/pal/windows/thread.rs index 28bce529cd991..2c8ce42f4148b 100644 --- a/library/std/src/sys/pal/windows/thread.rs +++ b/library/std/src/sys/pal/windows/thread.rs @@ -99,7 +99,7 @@ impl Thread { } // Attempt to use high-precision sleep (Windows 10, version 1803+). // On error fallback to the standard `Sleep` function. - // Also preserves the zero duration behaviour of `Sleep`. + // Also preserves the zero duration behavior of `Sleep`. if dur.is_zero() || high_precision_sleep(dur).is_err() { unsafe { c::Sleep(super::dur2timeout(dur)) } } diff --git a/library/std/src/sys/pal/xous/args.rs b/library/std/src/sys/pal/xous/args.rs new file mode 100644 index 0000000000000..00c44ca220a9e --- /dev/null +++ b/library/std/src/sys/pal/xous/args.rs @@ -0,0 +1,53 @@ +use crate::ffi::OsString; +use crate::sys::pal::xous::os::get_application_parameters; +use crate::sys::pal::xous::os::params::ArgumentList; +use crate::{fmt, vec}; + +pub struct Args { + parsed_args_list: vec::IntoIter, +} + +pub fn args() -> Args { + let Some(params) = get_application_parameters() else { + return Args { parsed_args_list: vec![].into_iter() }; + }; + + for param in params { + if let Ok(args) = ArgumentList::try_from(¶m) { + let mut parsed_args = vec![]; + for arg in args { + parsed_args.push(arg.into()); + } + return Args { parsed_args_list: parsed_args.into_iter() }; + } + } + Args { parsed_args_list: vec![].into_iter() } +} + +impl fmt::Debug for Args { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + self.parsed_args_list.as_slice().fmt(f) + } +} + +impl Iterator for Args { + type Item = OsString; + fn next(&mut self) -> Option { + self.parsed_args_list.next() + } + fn size_hint(&self) -> (usize, Option) { + self.parsed_args_list.size_hint() + } +} + +impl DoubleEndedIterator for Args { + fn next_back(&mut self) -> Option { + self.parsed_args_list.next_back() + } +} + +impl ExactSizeIterator for Args { + fn len(&self) -> usize { + self.parsed_args_list.len() + } +} diff --git a/library/std/src/sys/pal/xous/mod.rs b/library/std/src/sys/pal/xous/mod.rs index b211e94db65d6..a64cd06856006 100644 --- a/library/std/src/sys/pal/xous/mod.rs +++ b/library/std/src/sys/pal/xous/mod.rs @@ -1,6 +1,5 @@ #![forbid(unsafe_op_in_unsafe_fn)] -#[path = "../unsupported/args.rs"] pub mod args; #[path = "../unsupported/env.rs"] pub mod env; diff --git a/library/std/src/sys/pal/xous/net/dns.rs b/library/std/src/sys/pal/xous/net/dns.rs index 50efe978c4a83..1a2b56b4da5d3 100644 --- a/library/std/src/sys/pal/xous/net/dns.rs +++ b/library/std/src/sys/pal/xous/net/dns.rs @@ -3,9 +3,10 @@ use core::convert::{TryFrom, TryInto}; use crate::io; use crate::net::{Ipv4Addr, SocketAddr, SocketAddrV4, SocketAddrV6}; use crate::os::xous::ffi::lend_mut; -use crate::os::xous::services::{dns_server, DnsLendMut}; +use crate::os::xous::services::{DnsLendMut, dns_server}; pub struct DnsError { + #[allow(dead_code)] pub code: u8, } diff --git a/library/std/src/sys/pal/xous/net/mod.rs b/library/std/src/sys/pal/xous/net/mod.rs index dd8b765aa74ae..3e18ed24208d3 100644 --- a/library/std/src/sys/pal/xous/net/mod.rs +++ b/library/std/src/sys/pal/xous/net/mod.rs @@ -60,6 +60,7 @@ pub mod netc { #[derive(Copy, Clone)] pub struct sockaddr_in { + #[allow(dead_code)] pub sin_family: sa_family_t, pub sin_port: u16, pub sin_addr: in_addr, @@ -72,6 +73,7 @@ pub mod netc { #[derive(Copy, Clone)] pub struct sockaddr_in6 { + #[allow(dead_code)] pub sin6_family: sa_family_t, pub sin6_port: u16, pub sin6_addr: in6_addr, diff --git a/library/std/src/sys/pal/xous/os.rs b/library/std/src/sys/pal/xous/os.rs index 8f8f35428c487..b0ab01a6383d2 100644 --- a/library/std/src/sys/pal/xous/os.rs +++ b/library/std/src/sys/pal/xous/os.rs @@ -1,29 +1,35 @@ use super::unsupported; +use crate::collections::HashMap; use crate::error::Error as StdError; use crate::ffi::{OsStr, OsString}; use crate::marker::PhantomData; use crate::os::xous::ffi::Error as XousError; use crate::path::{self, PathBuf}; -use crate::{fmt, io}; +use crate::sync::atomic::{AtomicPtr, AtomicUsize, Ordering}; +use crate::sync::{Mutex, Once}; +use crate::{fmt, io, vec}; + +pub(crate) mod params; + +static PARAMS_ADDRESS: AtomicPtr = AtomicPtr::new(core::ptr::null_mut()); #[cfg(not(test))] #[cfg(feature = "panic_unwind")] mod eh_unwinding { - pub(crate) struct EhFrameFinder(usize /* eh_frame */); - pub(crate) static mut EH_FRAME_SETTINGS: EhFrameFinder = EhFrameFinder(0); - impl EhFrameFinder { - pub(crate) unsafe fn init(&mut self, eh_frame: usize) { - unsafe { - EH_FRAME_SETTINGS.0 = eh_frame; - } - } - } + pub(crate) struct EhFrameFinder; + pub(crate) static mut EH_FRAME_ADDRESS: usize = 0; + pub(crate) static EH_FRAME_SETTINGS: EhFrameFinder = EhFrameFinder; + unsafe impl unwind::EhFrameFinder for EhFrameFinder { fn find(&self, _pc: usize) -> Option { - Some(unwind::FrameInfo { - text_base: None, - kind: unwind::FrameInfoKind::EhFrame(self.0), - }) + if unsafe { EH_FRAME_ADDRESS == 0 } { + None + } else { + Some(unwind::FrameInfo { + text_base: None, + kind: unwind::FrameInfoKind::EhFrame(unsafe { EH_FRAME_ADDRESS }), + }) + } } } } @@ -41,12 +47,21 @@ mod c_compat { } #[no_mangle] - pub extern "C" fn _start(eh_frame: usize) { + pub extern "C" fn _start(eh_frame: usize, params_address: usize) { #[cfg(feature = "panic_unwind")] - unsafe { - super::eh_unwinding::EH_FRAME_SETTINGS.init(eh_frame); + { + unsafe { super::eh_unwinding::EH_FRAME_ADDRESS = eh_frame }; unwind::set_custom_eh_frame_finder(&super::eh_unwinding::EH_FRAME_SETTINGS).ok(); } + + if params_address != 0 { + let params_address = crate::ptr::with_exposed_provenance_mut::(params_address); + if unsafe { + super::params::ApplicationParameters::new_from_ptr(params_address).is_some() + } { + super::PARAMS_ADDRESS.store(params_address, core::sync::atomic::Ordering::Relaxed); + } + } exit(unsafe { main() }); } @@ -116,44 +131,103 @@ pub fn current_exe() -> io::Result { unsupported() } -pub struct Env(!); +pub(crate) fn get_application_parameters() -> Option { + let params_address = PARAMS_ADDRESS.load(Ordering::Relaxed); + unsafe { params::ApplicationParameters::new_from_ptr(params_address) } +} + +// ---------- Environment handling ---------- // +static ENV: AtomicUsize = AtomicUsize::new(0); +static ENV_INIT: Once = Once::new(); +type EnvStore = Mutex>; + +fn get_env_store() -> &'static EnvStore { + ENV_INIT.call_once(|| { + let env_store = EnvStore::default(); + if let Some(params) = get_application_parameters() { + for param in params { + if let Ok(envs) = params::EnvironmentBlock::try_from(¶m) { + let mut env_store = env_store.lock().unwrap(); + for env in envs { + env_store.insert(env.key.into(), env.value.into()); + } + break; + } + } + } + ENV.store(Box::into_raw(Box::new(env_store)) as _, Ordering::Relaxed) + }); + unsafe { &*core::ptr::with_exposed_provenance::(ENV.load(Ordering::Relaxed)) } +} + +pub struct Env { + iter: vec::IntoIter<(OsString, OsString)>, +} + +// FIXME(https://github.com/rust-lang/rust/issues/114583): Remove this when ::fmt matches ::fmt. +pub struct EnvStrDebug<'a> { + slice: &'a [(OsString, OsString)], +} + +impl fmt::Debug for EnvStrDebug<'_> { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + let Self { slice } = self; + f.debug_list() + .entries(slice.iter().map(|(a, b)| (a.to_str().unwrap(), b.to_str().unwrap()))) + .finish() + } +} impl Env { // FIXME(https://github.com/rust-lang/rust/issues/114583): Remove this when ::fmt matches ::fmt. pub fn str_debug(&self) -> impl fmt::Debug + '_ { - let Self(inner) = self; - match *inner {} + let Self { iter } = self; + EnvStrDebug { slice: iter.as_slice() } } } impl fmt::Debug for Env { - fn fmt(&self, _: &mut fmt::Formatter<'_>) -> fmt::Result { - let Self(inner) = self; - match *inner {} + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + let Self { iter } = self; + f.debug_list().entries(iter.as_slice()).finish() } } +impl !Send for Env {} +impl !Sync for Env {} + impl Iterator for Env { type Item = (OsString, OsString); fn next(&mut self) -> Option<(OsString, OsString)> { - self.0 + self.iter.next() + } + fn size_hint(&self) -> (usize, Option) { + self.iter.size_hint() } } pub fn env() -> Env { - panic!("not supported on this platform") + let clone_to_vec = |map: &HashMap| -> Vec<_> { + map.iter().map(|(k, v)| (k.clone(), v.clone())).collect() + }; + + let iter = clone_to_vec(&*get_env_store().lock().unwrap()).into_iter(); + Env { iter } } -pub fn getenv(_: &OsStr) -> Option { - None +pub fn getenv(k: &OsStr) -> Option { + get_env_store().lock().unwrap().get(k).cloned() } -pub unsafe fn setenv(_: &OsStr, _: &OsStr) -> io::Result<()> { - Err(io::const_io_error!(io::ErrorKind::Unsupported, "cannot set env vars on this platform")) +pub unsafe fn setenv(k: &OsStr, v: &OsStr) -> io::Result<()> { + let (k, v) = (k.to_owned(), v.to_owned()); + get_env_store().lock().unwrap().insert(k, v); + Ok(()) } -pub unsafe fn unsetenv(_: &OsStr) -> io::Result<()> { - Err(io::const_io_error!(io::ErrorKind::Unsupported, "cannot unset env vars on this platform")) +pub unsafe fn unsetenv(k: &OsStr) -> io::Result<()> { + get_env_store().lock().unwrap().remove(k); + Ok(()) } pub fn temp_dir() -> PathBuf { diff --git a/library/std/src/sys/pal/xous/os/params.rs b/library/std/src/sys/pal/xous/os/params.rs new file mode 100644 index 0000000000000..0d02cf35477f9 --- /dev/null +++ b/library/std/src/sys/pal/xous/os/params.rs @@ -0,0 +1,271 @@ +/// Xous passes a pointer to the parameter block as the second argument. +/// This is used for passing flags such as environment variables. The +/// format of the argument block is: +/// +/// #[repr(C)] +/// struct BlockHeader { +/// /// Magic number that identifies this block. Must be printable ASCII. +/// magic: [u8; 4], +/// +/// /// The size of the data block. Does not include this header. May be 0. +/// size: u32, +/// +/// /// The contents of this block. Varies depending on the block type. +/// data: [u8; 0], +/// } +/// +/// There is a BlockHeader at the start that has magic `AppP`, and the data +/// that follows is the number of blocks present: +/// +/// #[repr(C)] +/// struct ApplicationParameters { +/// magic: b"AppP", +/// size: 4u32, +/// +/// /// The size of the entire application slice, in bytes, including all headers +/// length: u32, +/// +/// /// Number of application parameters present. Must be at least 1 (this block) +/// entries: (parameter_count as u32).to_bytes_le(), +/// } +/// +/// #[repr(C)] +/// struct EnvironmentBlock { +/// magic: b"EnvB", +/// +/// /// Total number of bytes, excluding this header +/// size: 2+data.len(), +/// +/// /// The number of environment variables +/// count: u16, +/// +/// /// Environment variable iteration +/// data: [u8; 0], +/// } +/// +/// Environment variables are present in an `EnvB` block. The `data` section is +/// a sequence of bytes of the form: +/// +/// (u16 /* key_len */; [0u8; key_len as usize] /* key */, +/// u16 /* val_len */ [0u8; val_len as usize]) +/// +/// #[repr(C)] +/// struct ArgumentList { +/// magic: b"ArgL", +/// +/// /// Total number of bytes, excluding this header +/// size: 2+data.len(), +/// +/// /// The number of arguments variables +/// count: u16, +/// +/// /// Argument variable iteration +/// data: [u8; 0], +/// } +/// +/// Args are just an array of strings that represent command line arguments. +/// They are a sequence of the form: +/// +/// (u16 /* val_len */ [0u8; val_len as usize]) +use core::slice; + +use crate::ffi::OsString; + +/// Magic number indicating we have an environment block +const ENV_MAGIC: [u8; 4] = *b"EnvB"; + +/// Command line arguments list +const ARGS_MAGIC: [u8; 4] = *b"ArgL"; + +/// Magic number indicating the loader has passed application parameters +const PARAMS_MAGIC: [u8; 4] = *b"AppP"; + +#[cfg(test)] +mod tests; + +pub(crate) struct ApplicationParameters { + data: &'static [u8], + offset: usize, + _entries: usize, +} + +impl ApplicationParameters { + pub(crate) unsafe fn new_from_ptr(data: *const u8) -> Option { + if data.is_null() { + return None; + } + + let magic = unsafe { core::slice::from_raw_parts(data, 4) }; + let block_length = unsafe { + u32::from_le_bytes(slice::from_raw_parts(data.add(4), 4).try_into().ok()?) as usize + }; + let data_length = unsafe { + u32::from_le_bytes(slice::from_raw_parts(data.add(8), 4).try_into().ok()?) as usize + }; + let entries = unsafe { + u32::from_le_bytes(slice::from_raw_parts(data.add(12), 4).try_into().ok()?) as usize + }; + + // Check for the main header + if data_length < 16 || magic != PARAMS_MAGIC || block_length != 8 { + return None; + } + + let data = unsafe { slice::from_raw_parts(data, data_length) }; + + Some(ApplicationParameters { data, offset: 0, _entries: entries }) + } +} + +impl Iterator for ApplicationParameters { + type Item = ApplicationParameter; + + fn next(&mut self) -> Option { + // Fetch magic, ensuring we don't run off the end + if self.offset + 4 > self.data.len() { + return None; + } + let magic = &self.data[self.offset..self.offset + 4]; + self.offset += 4; + + // Fetch header size + if self.offset + 4 > self.data.len() { + return None; + } + let size = u32::from_le_bytes(self.data[self.offset..self.offset + 4].try_into().unwrap()) + as usize; + self.offset += 4; + + // Fetch data contents + if self.offset + size > self.data.len() { + return None; + } + let data = &self.data[self.offset..self.offset + size]; + self.offset += size; + + Some(ApplicationParameter { data, magic: magic.try_into().unwrap() }) + } +} + +pub(crate) struct ApplicationParameter { + data: &'static [u8], + magic: [u8; 4], +} + +pub(crate) struct ApplicationParameterError; + +pub(crate) struct EnvironmentBlock { + _count: usize, + data: &'static [u8], + offset: usize, +} + +impl TryFrom<&ApplicationParameter> for EnvironmentBlock { + type Error = ApplicationParameterError; + + fn try_from(value: &ApplicationParameter) -> Result { + if value.data.len() < 2 || value.magic != ENV_MAGIC { + return Err(ApplicationParameterError); + } + + let count = u16::from_le_bytes(value.data[0..2].try_into().unwrap()) as usize; + + Ok(EnvironmentBlock { data: &value.data[2..], offset: 0, _count: count }) + } +} + +pub(crate) struct EnvironmentEntry { + pub key: &'static str, + pub value: &'static str, +} + +impl Iterator for EnvironmentBlock { + type Item = EnvironmentEntry; + + fn next(&mut self) -> Option { + if self.offset + 2 > self.data.len() { + return None; + } + let key_len = + u16::from_le_bytes(self.data[self.offset..self.offset + 2].try_into().ok()?) as usize; + self.offset += 2; + + if self.offset + key_len > self.data.len() { + return None; + } + let key = core::str::from_utf8(&self.data[self.offset..self.offset + key_len]).ok()?; + self.offset += key_len; + + if self.offset + 2 > self.data.len() { + return None; + } + let value_len = + u16::from_le_bytes(self.data[self.offset..self.offset + 2].try_into().ok()?) as usize; + self.offset += 2; + + if self.offset + value_len > self.data.len() { + return None; + } + let value = core::str::from_utf8(&self.data[self.offset..self.offset + value_len]).ok()?; + self.offset += value_len; + + Some(EnvironmentEntry { key, value }) + } +} + +pub(crate) struct ArgumentList { + data: &'static [u8], + _count: usize, + offset: usize, +} + +impl TryFrom<&ApplicationParameter> for ArgumentList { + type Error = ApplicationParameterError; + + fn try_from(value: &ApplicationParameter) -> Result { + if value.data.len() < 2 || value.magic != ARGS_MAGIC { + return Err(ApplicationParameterError); + } + let count = + u16::from_le_bytes(value.data[0..2].try_into().or(Err(ApplicationParameterError))?) + as usize; + Ok(ArgumentList { data: &value.data[2..], _count: count, offset: 0 }) + } +} + +pub(crate) struct ArgumentEntry { + value: &'static str, +} + +impl Into<&str> for ArgumentEntry { + fn into(self) -> &'static str { + self.value + } +} + +impl Into for ArgumentEntry { + fn into(self) -> OsString { + self.value.into() + } +} + +impl Iterator for ArgumentList { + type Item = ArgumentEntry; + + fn next(&mut self) -> Option { + if self.offset + 2 > self.data.len() { + return None; + } + let value_len = + u16::from_le_bytes(self.data[self.offset..self.offset + 2].try_into().ok()?) as usize; + self.offset += 2; + + if self.offset + value_len > self.data.len() { + return None; + } + let value = core::str::from_utf8(&self.data[self.offset..self.offset + value_len]).ok()?; + self.offset += value_len; + + Some(ArgumentEntry { value }) + } +} diff --git a/library/std/src/sys/pal/xous/os/params/tests.rs b/library/std/src/sys/pal/xous/os/params/tests.rs new file mode 100644 index 0000000000000..0ef04ee30919f --- /dev/null +++ b/library/std/src/sys/pal/xous/os/params/tests.rs @@ -0,0 +1,75 @@ +use super::*; +use crate::collections::HashMap; +use crate::io::Write; + +fn create_args_test() -> std::io::Result> { + let mut sample_data = vec![]; + let mut h = HashMap::new(); + + h.insert("foo", "bar"); + h.insert("baz", "qux"); + h.insert("some", "val"); + + // Magic number + sample_data.write_all(&PARAMS_MAGIC)?; + // Size of the AppP block + sample_data.write_all(&4u32.to_le_bytes())?; + // Number of blocks + sample_data.write_all(&2u32.to_le_bytes())?; + + // Magic number + sample_data.write_all(&ENV_MAGIC)?; + let mut data = vec![]; + for (key, value) in h.iter() { + data.extend_from_slice(&(key.len() as u16).to_le_bytes()); + data.extend_from_slice(key.as_bytes()); + data.extend_from_slice(&(value.len() as u16).to_le_bytes()); + data.extend_from_slice(value.as_bytes()); + } + // Size of the EnvB block + sample_data.write_all(&(data.len() as u32 + 2).to_le_bytes())?; + + // Number of environment variables + sample_data.write_all(&(h.len() as u16).to_le_bytes())?; + + // Environment variables + sample_data.write_all(&data)?; + + // Write command line arguments + let args = vec!["some", "command", "line variable", "entries"]; + sample_data.write_all(&ARGS_MAGIC)?; + let mut args_size = 0; + for entry in args.iter() { + args_size += entry.len() + 2; + } + sample_data.write_all(&(args_size as u32 + 2).to_le_bytes())?; + sample_data.write_all(&(args.len() as u16).to_le_bytes())?; + for entry in args { + sample_data.write_all(&(entry.len() as u16).to_le_bytes())?; + sample_data.write_all(entry.as_bytes())?; + } + + Ok(sample_data) +} + +#[test] +fn basic_arg_parsing() { + let arg_data = create_args_test().expect("couldn't create test data"); + for byte in &arg_data { + print!("{:02x} ", byte); + } + println!(); + + let args = ApplicationParameters::new(&arg_data).expect("Unable to parse arguments"); + for arg in args { + if let Ok(env) = EnvironmentBlock::try_from(&arg) { + for env in env { + println!("{}={}", env.key, env.value); + } + } else if let Ok(args) = ArgumentList::try_from(&arg) { + for arg in args { + println!("Arg: {}", arg.value); + } + } + } +} diff --git a/library/std/src/sys/pal/xous/stdio.rs b/library/std/src/sys/pal/xous/stdio.rs index 11608964b52e6..dfd47a1775ae2 100644 --- a/library/std/src/sys/pal/xous/stdio.rs +++ b/library/std/src/sys/pal/xous/stdio.rs @@ -4,8 +4,8 @@ pub struct Stdin; pub struct Stdout {} pub struct Stderr; -use crate::os::xous::ffi::{lend, try_lend, try_scalar, Connection}; -use crate::os::xous::services::{log_server, try_connect, LogLend, LogScalar}; +use crate::os::xous::ffi::{Connection, lend, try_lend, try_scalar}; +use crate::os::xous::services::{LogLend, LogScalar, log_server, try_connect}; impl Stdin { pub const fn new() -> Stdin { diff --git a/library/std/src/sys/pal/xous/thread.rs b/library/std/src/sys/pal/xous/thread.rs index a95b0aa14d255..0ebb46dc19faa 100644 --- a/library/std/src/sys/pal/xous/thread.rs +++ b/library/std/src/sys/pal/xous/thread.rs @@ -4,10 +4,10 @@ use crate::ffi::CStr; use crate::io; use crate::num::NonZero; use crate::os::xous::ffi::{ - blocking_scalar, create_thread, do_yield, join_thread, map_memory, update_memory_flags, - MemoryFlags, Syscall, ThreadId, + MemoryFlags, Syscall, ThreadId, blocking_scalar, create_thread, do_yield, join_thread, + map_memory, update_memory_flags, }; -use crate::os::xous::services::{ticktimer_server, TicktimerScalar}; +use crate::os::xous::services::{TicktimerScalar, ticktimer_server}; use crate::time::Duration; pub struct Thread { diff --git a/library/std/src/sys/pal/zkvm/args.rs b/library/std/src/sys/pal/zkvm/args.rs index 583c16e3a4721..47857f6c448bc 100644 --- a/library/std/src/sys/pal/zkvm/args.rs +++ b/library/std/src/sys/pal/zkvm/args.rs @@ -1,4 +1,4 @@ -use super::{abi, WORD_SIZE}; +use super::{WORD_SIZE, abi}; use crate::ffi::OsString; use crate::fmt; use crate::sys::os_str; diff --git a/library/std/src/sys/pal/zkvm/mod.rs b/library/std/src/sys/pal/zkvm/mod.rs index 20fdb7468a40d..6ea057720296d 100644 --- a/library/std/src/sys/pal/zkvm/mod.rs +++ b/library/std/src/sys/pal/zkvm/mod.rs @@ -60,11 +60,3 @@ pub fn decode_error_kind(_code: i32) -> crate::io::ErrorKind { pub fn abort_internal() -> ! { core::intrinsics::abort(); } - -pub fn hashmap_random_keys() -> (u64, u64) { - let mut buf = [0u32; 4]; - unsafe { - abi::sys_rand(buf.as_mut_ptr(), 4); - }; - ((buf[0] as u64) << 32 + buf[1] as u64, (buf[2] as u64) << 32 + buf[3] as u64) -} diff --git a/library/std/src/sys/pal/zkvm/os.rs b/library/std/src/sys/pal/zkvm/os.rs index 68d91a123acd4..5d224ffd1ba5a 100644 --- a/library/std/src/sys/pal/zkvm/os.rs +++ b/library/std/src/sys/pal/zkvm/os.rs @@ -1,4 +1,4 @@ -use super::{abi, unsupported, WORD_SIZE}; +use super::{WORD_SIZE, abi, unsupported}; use crate::error::Error as StdError; use crate::ffi::{OsStr, OsString}; use crate::marker::PhantomData; diff --git a/library/std/src/sys/path/windows.rs b/library/std/src/sys/path/windows.rs index 2ae9a0a91996f..9267602cb9715 100644 --- a/library/std/src/sys/path/windows.rs +++ b/library/std/src/sys/path/windows.rs @@ -99,7 +99,7 @@ impl<'a> PrefixParserSlice<'a, '_> { } pub fn parse_prefix(path: &OsStr) -> Option> { - use Prefix::{DeviceNS, Disk, Verbatim, VerbatimDisk, VerbatimUNC, UNC}; + use Prefix::{DeviceNS, Disk, UNC, Verbatim, VerbatimDisk, VerbatimUNC}; let parser = PrefixParser::<8>::new(path); let parser = parser.as_slice(); diff --git a/library/std/src/sys/path/windows/tests.rs b/library/std/src/sys/path/windows/tests.rs index 623c6236166da..f2a60e30bc610 100644 --- a/library/std/src/sys/path/windows/tests.rs +++ b/library/std/src/sys/path/windows/tests.rs @@ -119,7 +119,7 @@ fn test_windows_prefix_components() { /// See #101358. /// -/// Note that the exact behaviour here may change in the future. +/// Note that the exact behavior here may change in the future. /// In which case this test will need to adjusted. #[test] fn broken_unc_path() { diff --git a/library/std/src/sys/personality/dwarf/eh.rs b/library/std/src/sys/personality/dwarf/eh.rs index c37c3e442aea6..778d8686f023e 100644 --- a/library/std/src/sys/personality/dwarf/eh.rs +++ b/library/std/src/sys/personality/dwarf/eh.rs @@ -54,10 +54,10 @@ pub enum EHAction { Terminate, } -/// 32-bit Apple ARM uses SjLj exceptions, except for watchOS. +/// 32-bit ARM Darwin platforms uses SjLj exceptions. /// -/// I.e. iOS and tvOS, as those are the only Apple OSes that used 32-bit ARM -/// devices. +/// The exception is watchOS armv7k (specifically that subarchitecture), which +/// instead uses DWARF Call Frame Information (CFI) unwinding. /// /// pub const USING_SJLJ_EXCEPTIONS: bool = diff --git a/library/std/src/sys/personality/gcc.rs b/library/std/src/sys/personality/gcc.rs index f6b1844e153fd..ad596ecff65d5 100644 --- a/library/std/src/sys/personality/gcc.rs +++ b/library/std/src/sys/personality/gcc.rs @@ -95,14 +95,15 @@ const UNWIND_DATA_REG: (i32, i32) = (4, 5); // a0, a1 cfg_if::cfg_if! { if #[cfg(all( - target_arch = "arm", - not(all(target_vendor = "apple", not(target_os = "watchos"))), - not(target_os = "netbsd"), - ))] { + target_arch = "arm", + not(target_vendor = "apple"), + not(target_os = "netbsd"), + ))] { /// personality fn called by [ARM EHABI][armeabi-eh] /// - /// Apple 32-bit ARM (but not watchOS) uses the default routine instead - /// since it uses "setjmp-longjmp" unwinding. + /// 32-bit ARM on iOS/tvOS/watchOS does not use ARM EHABI, it uses + /// either "setjmp-longjmp" unwinding or DWARF CFI unwinding, which is + /// handled by the default routine. /// /// [armeabi-eh]: https://web.archive.org/web/20190728160938/https://infocenter.arm.com/help/topic/com.arm.doc.ihi0038b/IHI0038B_ehabi.pdf #[lang = "eh_personality"] diff --git a/library/std/src/sys/personality/mod.rs b/library/std/src/sys/personality/mod.rs index 68085d026c40a..9754e840d151a 100644 --- a/library/std/src/sys/personality/mod.rs +++ b/library/std/src/sys/personality/mod.rs @@ -31,7 +31,7 @@ cfg_if::cfg_if! { target_os = "psp", target_os = "xous", target_os = "solid_asp3", - all(target_family = "unix", not(target_os = "espidf"), not(target_os = "l4re"), not(target_os = "rtems")), + all(target_family = "unix", not(target_os = "espidf"), not(target_os = "l4re"), not(target_os = "rtems"), not(target_os = "nuttx")), all(target_vendor = "fortanix", target_env = "sgx"), ))] { mod gcc; diff --git a/library/std/src/sys/random/apple.rs b/library/std/src/sys/random/apple.rs new file mode 100644 index 0000000000000..417198c9d850a --- /dev/null +++ b/library/std/src/sys/random/apple.rs @@ -0,0 +1,15 @@ +//! Random data on Apple platforms. +//! +//! `CCRandomGenerateBytes` calls into `CCRandomCopyBytes` with `kCCRandomDefault`. +//! `CCRandomCopyBytes` manages a CSPRNG which is seeded from the kernel's CSPRNG. +//! We use `CCRandomGenerateBytes` instead of `SecCopyBytes` because it is accessible via +//! `libSystem` (libc) while the other needs to link to `Security.framework`. +//! +//! Note that technically, `arc4random_buf` is available as well, but that calls +//! into the same system service anyway, and `CCRandomGenerateBytes` has been +//! proven to be App Store-compatible. + +pub fn fill_bytes(bytes: &mut [u8]) { + let ret = unsafe { libc::CCRandomGenerateBytes(bytes.as_mut_ptr().cast(), bytes.len()) }; + assert_eq!(ret, libc::kCCSuccess, "failed to generate random data"); +} diff --git a/library/std/src/sys/random/arc4random.rs b/library/std/src/sys/random/arc4random.rs new file mode 100644 index 0000000000000..ffabaafbee803 --- /dev/null +++ b/library/std/src/sys/random/arc4random.rs @@ -0,0 +1,36 @@ +//! Random data generation with `arc4random_buf`. +//! +//! Contrary to its name, `arc4random` doesn't actually use the horribly-broken +//! RC4 cypher anymore, at least not on modern systems, but rather something +//! like ChaCha20 with continual reseeding from the OS. That makes it an ideal +//! source of large quantities of cryptographically secure data, which is exactly +//! what we need for `DefaultRandomSource`. Unfortunately, it's not available +//! on all UNIX systems, most notably Linux (until recently, but it's just a +//! wrapper for `getrandom`. Since we need to hook into `getrandom` directly +//! for `HashMap` keys anyway, we just keep our version). + +#[cfg(not(any( + target_os = "haiku", + target_os = "illumos", + target_os = "rtems", + target_os = "solaris", + target_os = "vita", +)))] +use libc::arc4random_buf; + +// FIXME: move these to libc (except Haiku, that one needs to link to libbsd.so). +#[cfg(any( + target_os = "haiku", // See https://git.haiku-os.org/haiku/tree/headers/compatibility/bsd/stdlib.h + target_os = "illumos", // See https://www.illumos.org/man/3C/arc4random + target_os = "rtems", // See https://docs.rtems.org/branches/master/bsp-howto/getentropy.html + target_os = "solaris", // See https://docs.oracle.com/cd/E88353_01/html/E37843/arc4random-3c.html + target_os = "vita", // See https://github.com/vitasdk/newlib/blob/b89e5bc183b516945f9ee07eef483ecb916e45ff/newlib/libc/include/stdlib.h#L74 +))] +#[cfg_attr(target_os = "haiku", link(name = "bsd"))] +extern "C" { + fn arc4random_buf(buf: *mut core::ffi::c_void, nbytes: libc::size_t); +} + +pub fn fill_bytes(bytes: &mut [u8]) { + unsafe { arc4random_buf(bytes.as_mut_ptr().cast(), bytes.len()) } +} diff --git a/library/std/src/sys/random/espidf.rs b/library/std/src/sys/random/espidf.rs new file mode 100644 index 0000000000000..fd52cb5559ce5 --- /dev/null +++ b/library/std/src/sys/random/espidf.rs @@ -0,0 +1,9 @@ +use crate::ffi::c_void; + +extern "C" { + fn esp_fill_random(buf: *mut c_void, len: usize); +} + +pub fn fill_bytes(bytes: &mut [u8]) { + unsafe { esp_fill_random(bytes.as_mut_ptr().cast(), bytes.len()) } +} diff --git a/library/std/src/sys/random/fuchsia.rs b/library/std/src/sys/random/fuchsia.rs new file mode 100644 index 0000000000000..77d72b3c5b784 --- /dev/null +++ b/library/std/src/sys/random/fuchsia.rs @@ -0,0 +1,13 @@ +//! Random data generation using the Zircon kernel. +//! +//! Fuchsia, as always, is quite nice and provides exactly the API we need: +//! . + +#[link(name = "zircon")] +extern "C" { + fn zx_cprng_draw(buffer: *mut u8, len: usize); +} + +pub fn fill_bytes(bytes: &mut [u8]) { + unsafe { zx_cprng_draw(bytes.as_mut_ptr(), bytes.len()) } +} diff --git a/library/std/src/sys/random/getentropy.rs b/library/std/src/sys/random/getentropy.rs new file mode 100644 index 0000000000000..110ac134c1f47 --- /dev/null +++ b/library/std/src/sys/random/getentropy.rs @@ -0,0 +1,17 @@ +//! Random data generation through `getentropy`. +//! +//! Since issue 8 (2024), the POSIX specification mandates the existence of the +//! `getentropy` function, which fills a slice of up to `GETENTROPY_MAX` bytes +//! (256 on all known platforms) with random data. Unfortunately, it's only +//! meant to be used to seed other CPRNGs, which we don't have, so we only use +//! it where `arc4random_buf` and friends aren't available or secure (currently +//! that's only the case on Emscripten). + +pub fn fill_bytes(bytes: &mut [u8]) { + // GETENTROPY_MAX isn't defined yet on most platforms, but it's mandated + // to be at least 256, so just use that as limit. + for chunk in bytes.chunks_mut(256) { + let r = unsafe { libc::getentropy(chunk.as_mut_ptr().cast(), chunk.len()) }; + assert_ne!(r, -1, "failed to generate random data"); + } +} diff --git a/library/std/src/sys/random/hermit.rs b/library/std/src/sys/random/hermit.rs new file mode 100644 index 0000000000000..92c0550d2d584 --- /dev/null +++ b/library/std/src/sys/random/hermit.rs @@ -0,0 +1,7 @@ +pub fn fill_bytes(mut bytes: &mut [u8]) { + while !bytes.is_empty() { + let res = unsafe { hermit_abi::read_entropy(bytes.as_mut_ptr(), bytes.len(), 0) }; + assert_ne!(res, -1, "failed to generate random data"); + bytes = &mut bytes[res as usize..]; + } +} diff --git a/library/std/src/sys/random/horizon.rs b/library/std/src/sys/random/horizon.rs new file mode 100644 index 0000000000000..0be2eae20a727 --- /dev/null +++ b/library/std/src/sys/random/horizon.rs @@ -0,0 +1,7 @@ +pub fn fill_bytes(mut bytes: &mut [u8]) { + while !bytes.is_empty() { + let r = unsafe { libc::getrandom(bytes.as_mut_ptr().cast(), bytes.len(), 0) }; + assert_ne!(r, -1, "failed to generate random data"); + bytes = &mut bytes[r as usize..]; + } +} diff --git a/library/std/src/sys/random/linux.rs b/library/std/src/sys/random/linux.rs new file mode 100644 index 0000000000000..e3cb79285cd15 --- /dev/null +++ b/library/std/src/sys/random/linux.rs @@ -0,0 +1,170 @@ +//! Random data generation with the Linux kernel. +//! +//! The first interface random data interface to be introduced on Linux were +//! the `/dev/random` and `/dev/urandom` special files. As paths can become +//! unreachable when inside a chroot and when the file descriptors are exhausted, +//! this was not enough to provide userspace with a reliable source of randomness, +//! so when the OpenBSD 5.6 introduced the `getentropy` syscall, Linux 3.17 got +//! its very own `getrandom` syscall to match.[^1] Unfortunately, even if our +//! minimum supported version were high enough, we still couldn't rely on the +//! syscall being available, as it is blocked in `seccomp` by default. +//! +//! The question is therefore which of the random sources to use. Historically, +//! the kernel contained two pools: the blocking and non-blocking pool. The +//! blocking pool used entropy estimation to limit the amount of available +//! bytes, while the non-blocking pool, once initialized using the blocking +//! pool, uses a CPRNG to return an unlimited number of random bytes. With a +//! strong enough CPRNG however, the entropy estimation didn't contribute that +//! much towards security while being an excellent vector for DoS attacs. Thus, +//! the blocking pool was removed in kernel version 5.6.[^2] That patch did not +//! magically increase the quality of the non-blocking pool, however, so we can +//! safely consider it strong enough even in older kernel versions and use it +//! unconditionally. +//! +//! One additional consideration to make is that the non-blocking pool is not +//! always initialized during early boot. We want the best quality of randomness +//! for the output of `DefaultRandomSource` so we simply wait until it is +//! initialized. When `HashMap` keys however, this represents a potential source +//! of deadlocks, as the additional entropy may only be generated once the +//! program makes forward progress. In that case, we just use the best random +//! data the system has available at the time. +//! +//! So in conclusion, we always want the output of the non-blocking pool, but +//! may need to wait until it is initalized. The default behavior of `getrandom` +//! is to wait until the non-blocking pool is initialized and then draw from there, +//! so if `getrandom` is available, we use its default to generate the bytes. For +//! `HashMap`, however, we need to specify the `GRND_INSECURE` flags, but that +//! is only available starting with kernel version 5.6. Thus, if we detect that +//! the flag is unsupported, we try `GRND_NONBLOCK` instead, which will only +//! succeed if the pool is initialized. If it isn't, we fall back to the file +//! access method. +//! +//! The behavior of `/dev/urandom` is inverse to that of `getrandom`: it always +//! yields data, even when the pool is not initialized. For generating `HashMap` +//! keys, this is not important, so we can use it directly. For secure data +//! however, we need to wait until initialization, which we can do by `poll`ing +//! `/dev/random`. +//! +//! TLDR: our fallback strategies are: +//! +//! Secure data | `HashMap` keys +//! --------------------------------------------|------------------ +//! getrandom(0) | getrandom(GRND_INSECURE) +//! poll("/dev/random") && read("/dev/urandom") | getrandom(GRND_NONBLOCK) +//! | read("/dev/urandom") +//! +//! [^1]: +//! [^2]: +//! +// FIXME(in 2040 or so): once the minimum kernel version is 5.6, remove the +// `GRND_NONBLOCK` fallback and use `/dev/random` instead of `/dev/urandom` +// when secure data is required. + +use crate::fs::File; +use crate::io::Read; +use crate::os::fd::AsRawFd; +use crate::sync::OnceLock; +use crate::sync::atomic::AtomicBool; +use crate::sync::atomic::Ordering::{Acquire, Relaxed, Release}; +use crate::sys::pal::os::errno; +use crate::sys::pal::weak::syscall; + +fn getrandom(mut bytes: &mut [u8], insecure: bool) { + // A weak symbol allows interposition, e.g. for perf measurements that want to + // disable randomness for consistency. Otherwise, we'll try a raw syscall. + // (`getrandom` was added in glibc 2.25, musl 1.1.20, android API level 28) + syscall! { + fn getrandom( + buffer: *mut libc::c_void, + length: libc::size_t, + flags: libc::c_uint + ) -> libc::ssize_t + } + + static GETRANDOM_AVAILABLE: AtomicBool = AtomicBool::new(true); + static GRND_INSECURE_AVAILABLE: AtomicBool = AtomicBool::new(true); + static URANDOM_READY: AtomicBool = AtomicBool::new(false); + static DEVICE: OnceLock = OnceLock::new(); + + if GETRANDOM_AVAILABLE.load(Relaxed) { + loop { + if bytes.is_empty() { + return; + } + + let flags = if insecure { + if GRND_INSECURE_AVAILABLE.load(Relaxed) { + libc::GRND_INSECURE + } else { + libc::GRND_NONBLOCK + } + } else { + 0 + }; + + let ret = unsafe { getrandom(bytes.as_mut_ptr().cast(), bytes.len(), flags) }; + if ret != -1 { + bytes = &mut bytes[ret as usize..]; + } else { + match errno() { + libc::EINTR => continue, + // `GRND_INSECURE` is not available, try + // `GRND_NONBLOCK`. + libc::EINVAL if flags == libc::GRND_INSECURE => { + GRND_INSECURE_AVAILABLE.store(false, Relaxed); + continue; + } + // The pool is not initialized yet, fall back to + // /dev/urandom for now. + libc::EAGAIN if flags == libc::GRND_NONBLOCK => break, + // `getrandom` is unavailable or blocked by seccomp. + // Don't try it again and fall back to /dev/urandom. + libc::ENOSYS | libc::EPERM => { + GETRANDOM_AVAILABLE.store(false, Relaxed); + break; + } + _ => panic!("failed to generate random data"), + } + } + } + } + + // When we want cryptographic strength, we need to wait for the CPRNG-pool + // to become initialized. Do this by polling `/dev/random` until it is ready. + if !insecure { + if !URANDOM_READY.load(Acquire) { + let random = File::open("/dev/random").expect("failed to open /dev/random"); + let mut fd = libc::pollfd { fd: random.as_raw_fd(), events: libc::POLLIN, revents: 0 }; + + while !URANDOM_READY.load(Acquire) { + let ret = unsafe { libc::poll(&mut fd, 1, -1) }; + match ret { + 1 => { + assert_eq!(fd.revents, libc::POLLIN); + URANDOM_READY.store(true, Release); + break; + } + -1 if errno() == libc::EINTR => continue, + _ => panic!("poll(\"/dev/random\") failed"), + } + } + } + } + + DEVICE + .get_or_try_init(|| File::open("/dev/urandom")) + .and_then(|mut dev| dev.read_exact(bytes)) + .expect("failed to generate random data"); +} + +pub fn fill_bytes(bytes: &mut [u8]) { + getrandom(bytes, false); +} + +pub fn hashmap_random_keys() -> (u64, u64) { + let mut bytes = [0; 16]; + getrandom(&mut bytes, true); + let k1 = u64::from_ne_bytes(bytes[..8].try_into().unwrap()); + let k2 = u64::from_ne_bytes(bytes[8..].try_into().unwrap()); + (k1, k2) +} diff --git a/library/std/src/sys/random/mod.rs b/library/std/src/sys/random/mod.rs new file mode 100644 index 0000000000000..f42351deb92c0 --- /dev/null +++ b/library/std/src/sys/random/mod.rs @@ -0,0 +1,98 @@ +cfg_if::cfg_if! { + // Tier 1 + if #[cfg(any(target_os = "linux", target_os = "android"))] { + mod linux; + pub use linux::{fill_bytes, hashmap_random_keys}; + } else if #[cfg(target_os = "windows")] { + mod windows; + pub use windows::fill_bytes; + } else if #[cfg(target_vendor = "apple")] { + mod apple; + pub use apple::fill_bytes; + // Others, in alphabetical ordering. + } else if #[cfg(any( + target_os = "dragonfly", + target_os = "freebsd", + target_os = "haiku", + target_os = "illumos", + target_os = "netbsd", + target_os = "openbsd", + target_os = "rtems", + target_os = "solaris", + target_os = "vita", + ))] { + mod arc4random; + pub use arc4random::fill_bytes; + } else if #[cfg(target_os = "emscripten")] { + mod getentropy; + pub use getentropy::fill_bytes; + } else if #[cfg(target_os = "espidf")] { + mod espidf; + pub use espidf::fill_bytes; + } else if #[cfg(target_os = "fuchsia")] { + mod fuchsia; + pub use fuchsia::fill_bytes; + } else if #[cfg(target_os = "hermit")] { + mod hermit; + pub use hermit::fill_bytes; + } else if #[cfg(target_os = "horizon")] { + // FIXME: add arc4random_buf to shim-3ds + mod horizon; + pub use horizon::fill_bytes; + } else if #[cfg(any( + target_os = "aix", + target_os = "hurd", + target_os = "l4re", + target_os = "nto", + target_os = "nuttx", + ))] { + mod unix_legacy; + pub use unix_legacy::fill_bytes; + } else if #[cfg(target_os = "redox")] { + mod redox; + pub use redox::fill_bytes; + } else if #[cfg(all(target_vendor = "fortanix", target_env = "sgx"))] { + mod sgx; + pub use sgx::fill_bytes; + } else if #[cfg(target_os = "solid_asp3")] { + mod solid; + pub use solid::fill_bytes; + } else if #[cfg(target_os = "teeos")] { + mod teeos; + pub use teeos::fill_bytes; + } else if #[cfg(target_os = "uefi")] { + mod uefi; + pub use uefi::fill_bytes; + } else if #[cfg(target_os = "vxworks")] { + mod vxworks; + pub use vxworks::fill_bytes; + } else if #[cfg(target_os = "wasi")] { + mod wasi; + pub use wasi::fill_bytes; + } else if #[cfg(target_os = "zkvm")] { + mod zkvm; + pub use zkvm::fill_bytes; + } else if #[cfg(any( + all(target_family = "wasm", target_os = "unknown"), + target_os = "xous", + ))] { + // FIXME: finally remove std support for wasm32-unknown-unknown + // FIXME: add random data generation to xous + mod unsupported; + pub use unsupported::{fill_bytes, hashmap_random_keys}; + } +} + +#[cfg(not(any( + target_os = "linux", + target_os = "android", + all(target_family = "wasm", target_os = "unknown"), + target_os = "xous", +)))] +pub fn hashmap_random_keys() -> (u64, u64) { + let mut buf = [0; 16]; + fill_bytes(&mut buf); + let k1 = u64::from_ne_bytes(buf[..8].try_into().unwrap()); + let k2 = u64::from_ne_bytes(buf[8..].try_into().unwrap()); + (k1, k2) +} diff --git a/library/std/src/sys/random/redox.rs b/library/std/src/sys/random/redox.rs new file mode 100644 index 0000000000000..b004335a35176 --- /dev/null +++ b/library/std/src/sys/random/redox.rs @@ -0,0 +1,12 @@ +use crate::fs::File; +use crate::io::Read; +use crate::sync::OnceLock; + +static SCHEME: OnceLock = OnceLock::new(); + +pub fn fill_bytes(bytes: &mut [u8]) { + SCHEME + .get_or_try_init(|| File::open("/scheme/rand")) + .and_then(|mut scheme| scheme.read_exact(bytes)) + .expect("failed to generate random data"); +} diff --git a/library/std/src/sys/random/sgx.rs b/library/std/src/sys/random/sgx.rs new file mode 100644 index 0000000000000..c3647a8df220e --- /dev/null +++ b/library/std/src/sys/random/sgx.rs @@ -0,0 +1,67 @@ +use crate::arch::x86_64::{_rdrand16_step, _rdrand32_step, _rdrand64_step}; + +const RETRIES: u32 = 10; + +fn fail() -> ! { + panic!("failed to generate random data"); +} + +fn rdrand64() -> u64 { + unsafe { + let mut ret: u64 = 0; + for _ in 0..RETRIES { + if _rdrand64_step(&mut ret) == 1 { + return ret; + } + } + + fail(); + } +} + +fn rdrand32() -> u32 { + unsafe { + let mut ret: u32 = 0; + for _ in 0..RETRIES { + if _rdrand32_step(&mut ret) == 1 { + return ret; + } + } + + fail(); + } +} + +fn rdrand16() -> u16 { + unsafe { + let mut ret: u16 = 0; + for _ in 0..RETRIES { + if _rdrand16_step(&mut ret) == 1 { + return ret; + } + } + + fail(); + } +} + +pub fn fill_bytes(bytes: &mut [u8]) { + let mut chunks = bytes.array_chunks_mut(); + for chunk in &mut chunks { + *chunk = rdrand64().to_ne_bytes(); + } + + let mut chunks = chunks.into_remainder().array_chunks_mut(); + for chunk in &mut chunks { + *chunk = rdrand32().to_ne_bytes(); + } + + let mut chunks = chunks.into_remainder().array_chunks_mut(); + for chunk in &mut chunks { + *chunk = rdrand16().to_ne_bytes(); + } + + if let [byte] = chunks.into_remainder() { + *byte = rdrand16() as u8; + } +} diff --git a/library/std/src/sys/random/solid.rs b/library/std/src/sys/random/solid.rs new file mode 100644 index 0000000000000..545771150e284 --- /dev/null +++ b/library/std/src/sys/random/solid.rs @@ -0,0 +1,8 @@ +use crate::sys::pal::abi; + +pub fn fill_bytes(bytes: &mut [u8]) { + unsafe { + let result = abi::SOLID_RNG_SampleRandomBytes(bytes.as_mut_ptr(), bytes.len()); + assert_eq!(result, 0, "failed to generate random data"); + } +} diff --git a/library/std/src/sys/random/teeos.rs b/library/std/src/sys/random/teeos.rs new file mode 100644 index 0000000000000..fd6b24e19e982 --- /dev/null +++ b/library/std/src/sys/random/teeos.rs @@ -0,0 +1,7 @@ +extern "C" { + fn TEE_GenerateRandom(randomBuffer: *mut core::ffi::c_void, randomBufferLen: libc::size_t); +} + +pub fn fill_bytes(bytes: &mut [u8]) { + unsafe { TEE_GenerateRandom(bytes.as_mut_ptr().cast(), bytes.len()) } +} diff --git a/library/std/src/sys/random/uefi.rs b/library/std/src/sys/random/uefi.rs new file mode 100644 index 0000000000000..a4d29e66f3875 --- /dev/null +++ b/library/std/src/sys/random/uefi.rs @@ -0,0 +1,27 @@ +use r_efi::protocols::rng; + +use crate::sys::pal::helpers; + +pub fn fill_bytes(bytes: &mut [u8]) { + let handles = + helpers::locate_handles(rng::PROTOCOL_GUID).expect("failed to generate random data"); + for handle in handles { + if let Ok(protocol) = helpers::open_protocol::(handle, rng::PROTOCOL_GUID) { + let r = unsafe { + ((*protocol.as_ptr()).get_rng)( + protocol.as_ptr(), + crate::ptr::null_mut(), + bytes.len(), + bytes.as_mut_ptr(), + ) + }; + if r.is_error() { + continue; + } else { + return; + } + } + } + + panic!("failed to generate random data"); +} diff --git a/library/std/src/sys/random/unix_legacy.rs b/library/std/src/sys/random/unix_legacy.rs new file mode 100644 index 0000000000000..587068b0d6641 --- /dev/null +++ b/library/std/src/sys/random/unix_legacy.rs @@ -0,0 +1,20 @@ +//! Random data from `/dev/urandom` +//! +//! Before `getentropy` was standardized in 2024, UNIX didn't have a standardized +//! way of getting random data, so systems just followed the precedent set by +//! Linux and exposed random devices at `/dev/random` and `/dev/urandom`. Thus, +//! for the few systems that support neither `arc4random_buf` nor `getentropy` +//! yet, we just read from the file. + +use crate::fs::File; +use crate::io::Read; +use crate::sync::OnceLock; + +static DEVICE: OnceLock = OnceLock::new(); + +pub fn fill_bytes(bytes: &mut [u8]) { + DEVICE + .get_or_try_init(|| File::open("/dev/urandom")) + .and_then(|mut dev| dev.read_exact(bytes)) + .expect("failed to generate random data"); +} diff --git a/library/std/src/sys/random/unsupported.rs b/library/std/src/sys/random/unsupported.rs new file mode 100644 index 0000000000000..d68ce4a9e8703 --- /dev/null +++ b/library/std/src/sys/random/unsupported.rs @@ -0,0 +1,15 @@ +use crate::ptr; + +pub fn fill_bytes(_: &mut [u8]) { + panic!("this target does not support random data generation"); +} + +pub fn hashmap_random_keys() -> (u64, u64) { + // Use allocation addresses for a bit of randomness. This isn't + // particularily secure, but there isn't really an alternative. + let stack = 0u8; + let heap = Box::new(0u8); + let k1 = ptr::from_ref(&stack).addr() as u64; + let k2 = ptr::from_ref(&*heap).addr() as u64; + (k1, k2) +} diff --git a/library/std/src/sys/random/vxworks.rs b/library/std/src/sys/random/vxworks.rs new file mode 100644 index 0000000000000..d549ccebdb2cd --- /dev/null +++ b/library/std/src/sys/random/vxworks.rs @@ -0,0 +1,25 @@ +use crate::sync::atomic::AtomicBool; +use crate::sync::atomic::Ordering::Relaxed; + +static RNG_INIT: AtomicBool = AtomicBool::new(false); + +pub fn fill_bytes(mut bytes: &mut [u8]) { + while !RNG_INIT.load(Relaxed) { + let ret = unsafe { libc::randSecure() }; + if ret < 0 { + panic!("failed to generate random data"); + } else if ret > 0 { + RNG_INIT.store(true, Relaxed); + break; + } + + unsafe { libc::usleep(10) }; + } + + while !bytes.is_empty() { + let len = bytes.len().try_into().unwrap_or(libc::c_int::MAX); + let ret = unsafe { libc::randABytes(bytes.as_mut_ptr(), len) }; + assert!(ret >= 0, "failed to generate random data"); + bytes = &mut bytes[len as usize..]; + } +} diff --git a/library/std/src/sys/random/wasi.rs b/library/std/src/sys/random/wasi.rs new file mode 100644 index 0000000000000..d41da3751fc04 --- /dev/null +++ b/library/std/src/sys/random/wasi.rs @@ -0,0 +1,5 @@ +pub fn fill_bytes(bytes: &mut [u8]) { + unsafe { + wasi::random_get(bytes.as_mut_ptr(), bytes.len()).expect("failed to generate random data") + } +} diff --git a/library/std/src/sys/random/windows.rs b/library/std/src/sys/random/windows.rs new file mode 100644 index 0000000000000..7566000f9e6ff --- /dev/null +++ b/library/std/src/sys/random/windows.rs @@ -0,0 +1,20 @@ +use crate::sys::c; + +#[cfg(not(target_vendor = "win7"))] +#[inline] +pub fn fill_bytes(bytes: &mut [u8]) { + let ret = unsafe { c::ProcessPrng(bytes.as_mut_ptr(), bytes.len()) }; + // ProcessPrng is documented as always returning `TRUE`. + // https://learn.microsoft.com/en-us/windows/win32/seccng/processprng#return-value + debug_assert_eq!(ret, c::TRUE); +} + +#[cfg(target_vendor = "win7")] +pub fn fill_bytes(mut bytes: &mut [u8]) { + while !bytes.is_empty() { + let len = bytes.len().try_into().unwrap_or(u32::MAX); + let ret = unsafe { c::RtlGenRandom(bytes.as_mut_ptr().cast(), len) }; + assert_ne!(ret, 0, "failed to generate random data"); + bytes = &mut bytes[len as usize..]; + } +} diff --git a/library/std/src/sys/random/zkvm.rs b/library/std/src/sys/random/zkvm.rs new file mode 100644 index 0000000000000..3011942f6b26b --- /dev/null +++ b/library/std/src/sys/random/zkvm.rs @@ -0,0 +1,21 @@ +use crate::sys::pal::abi; + +pub fn fill_bytes(bytes: &mut [u8]) { + let (pre, words, post) = unsafe { bytes.align_to_mut::() }; + if !words.is_empty() { + unsafe { + abi::sys_rand(words.as_mut_ptr(), words.len()); + } + } + + let mut buf = [0u32; 2]; + let len = (pre.len() + post.len() + size_of::() - 1) / size_of::(); + if len != 0 { + unsafe { abi::sys_rand(buf.as_mut_ptr(), len) }; + } + + let buf = buf.map(u32::to_ne_bytes); + let buf = buf.as_flattened(); + pre.copy_from_slice(&buf[..pre.len()]); + post.copy_from_slice(&buf[pre.len()..pre.len() + post.len()]); +} diff --git a/library/std/src/sys/sync/condvar/futex.rs b/library/std/src/sys/sync/condvar/futex.rs index 39cd97c01ea32..0d0c5f0dbe701 100644 --- a/library/std/src/sys/sync/condvar/futex.rs +++ b/library/std/src/sys/sync/condvar/futex.rs @@ -1,6 +1,5 @@ -use crate::sync::atomic::AtomicU32; use crate::sync::atomic::Ordering::Relaxed; -use crate::sys::futex::{futex_wait, futex_wake, futex_wake_all}; +use crate::sys::futex::{Futex, futex_wait, futex_wake, futex_wake_all}; use crate::sys::sync::Mutex; use crate::time::Duration; @@ -8,13 +7,13 @@ pub struct Condvar { // The value of this atomic is simply incremented on every notification. // This is used by `.wait()` to not miss any notifications after // unlocking the mutex and before waiting for notifications. - futex: AtomicU32, + futex: Futex, } impl Condvar { #[inline] pub const fn new() -> Self { - Self { futex: AtomicU32::new(0) } + Self { futex: Futex::new(0) } } // All the memory orderings here are `Relaxed`, diff --git a/library/std/src/sys/sync/condvar/mod.rs b/library/std/src/sys/sync/condvar/mod.rs index 6849cacf88e76..d0c998a559737 100644 --- a/library/std/src/sys/sync/condvar/mod.rs +++ b/library/std/src/sys/sync/condvar/mod.rs @@ -12,7 +12,10 @@ cfg_if::cfg_if! { ))] { mod futex; pub use futex::Condvar; - } else if #[cfg(target_family = "unix")] { + } else if #[cfg(any( + target_family = "unix", + target_os = "teeos", + ))] { mod pthread; pub use pthread::Condvar; } else if #[cfg(all(target_os = "windows", target_vendor = "win7"))] { @@ -24,9 +27,6 @@ cfg_if::cfg_if! { } else if #[cfg(target_os = "solid_asp3")] { mod itron; pub use itron::Condvar; - } else if #[cfg(target_os = "teeos")] { - mod teeos; - pub use teeos::Condvar; } else if #[cfg(target_os = "xous")] { mod xous; pub use xous::Condvar; diff --git a/library/std/src/sys/sync/condvar/no_threads.rs b/library/std/src/sys/sync/condvar/no_threads.rs index 36b89c5f5bef7..2a67ed766aa0c 100644 --- a/library/std/src/sys/sync/condvar/no_threads.rs +++ b/library/std/src/sys/sync/condvar/no_threads.rs @@ -5,7 +5,7 @@ pub struct Condvar {} impl Condvar { #[inline] - #[rustc_const_stable(feature = "const_locks", since = "1.63.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_locks", since = "1.63.0"))] pub const fn new() -> Condvar { Condvar {} } diff --git a/library/std/src/sys/sync/condvar/pthread.rs b/library/std/src/sys/sync/condvar/pthread.rs index 2b4bdfe415c80..cee728e35cdfc 100644 --- a/library/std/src/sys/sync/condvar/pthread.rs +++ b/library/std/src/sys/sync/condvar/pthread.rs @@ -2,31 +2,25 @@ use crate::cell::UnsafeCell; use crate::ptr; use crate::sync::atomic::AtomicPtr; use crate::sync::atomic::Ordering::Relaxed; -use crate::sys::sync::{mutex, Mutex}; +use crate::sys::sync::{Mutex, OnceBox}; #[cfg(not(target_os = "nto"))] use crate::sys::time::TIMESPEC_MAX; #[cfg(target_os = "nto")] use crate::sys::time::TIMESPEC_MAX_CAPPED; -use crate::sys_common::lazy_box::{LazyBox, LazyInit}; use crate::time::Duration; struct AllocatedCondvar(UnsafeCell); pub struct Condvar { - inner: LazyBox, + inner: OnceBox, mutex: AtomicPtr, } -#[inline] -fn raw(c: &Condvar) -> *mut libc::pthread_cond_t { - c.inner.0.get() -} - unsafe impl Send for AllocatedCondvar {} unsafe impl Sync for AllocatedCondvar {} -impl LazyInit for AllocatedCondvar { - fn init() -> Box { +impl AllocatedCondvar { + fn new() -> Box { let condvar = Box::new(AllocatedCondvar(UnsafeCell::new(libc::PTHREAD_COND_INITIALIZER))); cfg_if::cfg_if! { @@ -37,7 +31,7 @@ impl LazyInit for AllocatedCondvar { target_vendor = "apple", ))] { // `pthread_condattr_setclock` is unfortunately not supported on these platforms. - } else if #[cfg(any(target_os = "espidf", target_os = "horizon"))] { + } else if #[cfg(any(target_os = "espidf", target_os = "horizon", target_os = "teeos"))] { // NOTE: ESP-IDF's PTHREAD_COND_INITIALIZER support is not released yet // So on that platform, init() should always be called // Moreover, that platform does not have pthread_condattr_setclock support, @@ -72,7 +66,7 @@ impl Drop for AllocatedCondvar { // On DragonFly pthread_cond_destroy() returns EINVAL if called on // a condvar that was just initialized with // libc::PTHREAD_COND_INITIALIZER. Once it is used or - // pthread_cond_init() is called, this behaviour no longer occurs. + // pthread_cond_init() is called, this behavior no longer occurs. debug_assert!(r == 0 || r == libc::EINVAL); } else { debug_assert_eq!(r, 0); @@ -82,7 +76,11 @@ impl Drop for AllocatedCondvar { impl Condvar { pub const fn new() -> Condvar { - Condvar { inner: LazyBox::new(), mutex: AtomicPtr::new(ptr::null_mut()) } + Condvar { inner: OnceBox::new(), mutex: AtomicPtr::new(ptr::null_mut()) } + } + + fn get(&self) -> *mut libc::pthread_cond_t { + self.inner.get_or_init(AllocatedCondvar::new).0.get() } #[inline] @@ -98,21 +96,21 @@ impl Condvar { #[inline] pub fn notify_one(&self) { - let r = unsafe { libc::pthread_cond_signal(raw(self)) }; + let r = unsafe { libc::pthread_cond_signal(self.get()) }; debug_assert_eq!(r, 0); } #[inline] pub fn notify_all(&self) { - let r = unsafe { libc::pthread_cond_broadcast(raw(self)) }; + let r = unsafe { libc::pthread_cond_broadcast(self.get()) }; debug_assert_eq!(r, 0); } #[inline] pub unsafe fn wait(&self, mutex: &Mutex) { - let mutex = mutex::raw(mutex); + let mutex = mutex.get_assert_locked(); self.verify(mutex); - let r = libc::pthread_cond_wait(raw(self), mutex); + let r = libc::pthread_cond_wait(self.get(), mutex); debug_assert_eq!(r, 0); } @@ -129,7 +127,7 @@ impl Condvar { pub unsafe fn wait_timeout(&self, mutex: &Mutex, dur: Duration) -> bool { use crate::sys::time::Timespec; - let mutex = mutex::raw(mutex); + let mutex = mutex.get_assert_locked(); self.verify(mutex); #[cfg(not(target_os = "nto"))] @@ -144,7 +142,7 @@ impl Condvar { .and_then(|t| t.to_timespec_capped()) .unwrap_or(TIMESPEC_MAX_CAPPED); - let r = libc::pthread_cond_timedwait(raw(self), mutex, &timeout); + let r = libc::pthread_cond_timedwait(self.get(), mutex, &timeout); assert!(r == libc::ETIMEDOUT || r == 0); r == 0 } @@ -162,7 +160,7 @@ impl Condvar { use crate::sys::time::SystemTime; use crate::time::Instant; - let mutex = mutex::raw(mutex); + let mutex = mutex.get_assert_locked(); self.verify(mutex); // OSX implementation of `pthread_cond_timedwait` is buggy @@ -188,7 +186,7 @@ impl Condvar { .and_then(|t| t.to_timespec()) .unwrap_or(TIMESPEC_MAX); - let r = libc::pthread_cond_timedwait(raw(self), mutex, &timeout); + let r = libc::pthread_cond_timedwait(self.get(), mutex, &timeout); debug_assert!(r == libc::ETIMEDOUT || r == 0); // ETIMEDOUT is not a totally reliable method of determining timeout due diff --git a/library/std/src/sys/sync/condvar/sgx.rs b/library/std/src/sys/sync/condvar/sgx.rs index ecb5872f60d90..e60715e4b592e 100644 --- a/library/std/src/sys/sync/condvar/sgx.rs +++ b/library/std/src/sys/sync/condvar/sgx.rs @@ -1,44 +1,39 @@ use crate::sys::pal::waitqueue::{SpinMutex, WaitQueue, WaitVariable}; -use crate::sys::sync::Mutex; -use crate::sys_common::lazy_box::{LazyBox, LazyInit}; +use crate::sys::sync::{Mutex, OnceBox}; use crate::time::Duration; -/// FIXME: `UnsafeList` is not movable. -struct AllocatedCondvar(SpinMutex>); - pub struct Condvar { - inner: LazyBox, -} - -impl LazyInit for AllocatedCondvar { - fn init() -> Box { - Box::new(AllocatedCondvar(SpinMutex::new(WaitVariable::new(())))) - } + // FIXME: `UnsafeList` is not movable. + inner: OnceBox>>, } impl Condvar { pub const fn new() -> Condvar { - Condvar { inner: LazyBox::new() } + Condvar { inner: OnceBox::new() } + } + + fn get(&self) -> &SpinMutex> { + self.inner.get_or_init(|| Box::new(SpinMutex::new(WaitVariable::new(())))) } #[inline] pub fn notify_one(&self) { - let _ = WaitQueue::notify_one(self.inner.0.lock()); + let _ = WaitQueue::notify_one(self.get().lock()); } #[inline] pub fn notify_all(&self) { - let _ = WaitQueue::notify_all(self.inner.0.lock()); + let _ = WaitQueue::notify_all(self.get().lock()); } pub unsafe fn wait(&self, mutex: &Mutex) { - let guard = self.inner.0.lock(); + let guard = self.get().lock(); WaitQueue::wait(guard, || unsafe { mutex.unlock() }); mutex.lock() } pub unsafe fn wait_timeout(&self, mutex: &Mutex, dur: Duration) -> bool { - let success = WaitQueue::wait_timeout(&self.inner.0, dur, || unsafe { mutex.unlock() }); + let success = WaitQueue::wait_timeout(self.get(), dur, || unsafe { mutex.unlock() }); mutex.lock(); success } diff --git a/library/std/src/sys/sync/condvar/teeos.rs b/library/std/src/sys/sync/condvar/teeos.rs deleted file mode 100644 index 943867cd76169..0000000000000 --- a/library/std/src/sys/sync/condvar/teeos.rs +++ /dev/null @@ -1,101 +0,0 @@ -use crate::cell::UnsafeCell; -use crate::ptr; -use crate::sync::atomic::AtomicPtr; -use crate::sync::atomic::Ordering::Relaxed; -use crate::sys::sync::mutex::{self, Mutex}; -use crate::sys::time::TIMESPEC_MAX; -use crate::sys_common::lazy_box::{LazyBox, LazyInit}; -use crate::time::Duration; - -extern "C" { - pub fn pthread_cond_timedwait( - cond: *mut libc::pthread_cond_t, - lock: *mut libc::pthread_mutex_t, - adstime: *const libc::timespec, - ) -> libc::c_int; -} - -struct AllocatedCondvar(UnsafeCell); - -pub struct Condvar { - inner: LazyBox, - mutex: AtomicPtr, -} - -#[inline] -fn raw(c: &Condvar) -> *mut libc::pthread_cond_t { - c.inner.0.get() -} - -unsafe impl Send for AllocatedCondvar {} -unsafe impl Sync for AllocatedCondvar {} - -impl LazyInit for AllocatedCondvar { - fn init() -> Box { - let condvar = Box::new(AllocatedCondvar(UnsafeCell::new(libc::PTHREAD_COND_INITIALIZER))); - - let r = unsafe { libc::pthread_cond_init(condvar.0.get(), crate::ptr::null()) }; - assert_eq!(r, 0); - - condvar - } -} - -impl Drop for AllocatedCondvar { - #[inline] - fn drop(&mut self) { - let r = unsafe { libc::pthread_cond_destroy(self.0.get()) }; - debug_assert_eq!(r, 0); - } -} - -impl Condvar { - pub const fn new() -> Condvar { - Condvar { inner: LazyBox::new(), mutex: AtomicPtr::new(ptr::null_mut()) } - } - - #[inline] - fn verify(&self, mutex: *mut libc::pthread_mutex_t) { - match self.mutex.compare_exchange(ptr::null_mut(), mutex, Relaxed, Relaxed) { - Ok(_) => {} // Stored the address - Err(n) if n == mutex => {} // Lost a race to store the same address - _ => panic!("attempted to use a condition variable with two mutexes"), - } - } - - #[inline] - pub fn notify_one(&self) { - let r = unsafe { libc::pthread_cond_signal(raw(self)) }; - debug_assert_eq!(r, 0); - } - - #[inline] - pub fn notify_all(&self) { - let r = unsafe { libc::pthread_cond_broadcast(raw(self)) }; - debug_assert_eq!(r, 0); - } - - #[inline] - pub unsafe fn wait(&self, mutex: &Mutex) { - let mutex = unsafe { mutex::raw(mutex) }; - self.verify(mutex); - let r = unsafe { libc::pthread_cond_wait(raw(self), mutex) }; - debug_assert_eq!(r, 0); - } - - pub unsafe fn wait_timeout(&self, mutex: &Mutex, dur: Duration) -> bool { - use crate::sys::time::Timespec; - - let mutex = unsafe { mutex::raw(mutex) }; - self.verify(mutex); - - let timeout = Timespec::now(libc::CLOCK_MONOTONIC) - .checked_add_duration(&dur) - .and_then(|t| t.to_timespec()) - .unwrap_or(TIMESPEC_MAX); - - let r = unsafe { pthread_cond_timedwait(raw(self), mutex, &timeout) }; - assert!(r == libc::ETIMEDOUT || r == 0); - r == 0 - } -} diff --git a/library/std/src/sys/sync/condvar/windows7.rs b/library/std/src/sys/sync/condvar/windows7.rs index 56eeeda551ebb..f03feef222124 100644 --- a/library/std/src/sys/sync/condvar/windows7.rs +++ b/library/std/src/sys/sync/condvar/windows7.rs @@ -1,5 +1,5 @@ use crate::cell::UnsafeCell; -use crate::sys::sync::{mutex, Mutex}; +use crate::sys::sync::{Mutex, mutex}; use crate::sys::{c, os}; use crate::time::Duration; diff --git a/library/std/src/sys/sync/condvar/xous.rs b/library/std/src/sys/sync/condvar/xous.rs index 007383479a100..b9e5f47abfcc2 100644 --- a/library/std/src/sys/sync/condvar/xous.rs +++ b/library/std/src/sys/sync/condvar/xous.rs @@ -1,7 +1,7 @@ use core::sync::atomic::{AtomicUsize, Ordering}; use crate::os::xous::ffi::{blocking_scalar, scalar}; -use crate::os::xous::services::{ticktimer_server, TicktimerScalar}; +use crate::os::xous::services::{TicktimerScalar, ticktimer_server}; use crate::sys::sync::Mutex; use crate::time::Duration; @@ -20,7 +20,6 @@ unsafe impl Sync for Condvar {} impl Condvar { #[inline] - #[rustc_const_stable(feature = "const_locks", since = "1.63.0")] pub const fn new() -> Condvar { Condvar { counter: AtomicUsize::new(0), timed_out: AtomicUsize::new(0) } } diff --git a/library/std/src/sys/sync/mod.rs b/library/std/src/sys/sync/mod.rs index 52fac5902a296..0691e96785198 100644 --- a/library/std/src/sys/sync/mod.rs +++ b/library/std/src/sys/sync/mod.rs @@ -1,11 +1,14 @@ mod condvar; mod mutex; mod once; +mod once_box; mod rwlock; mod thread_parking; pub use condvar::Condvar; pub use mutex::Mutex; pub use once::{Once, OnceState}; +#[allow(unused)] // Only used on some platforms. +use once_box::OnceBox; pub use rwlock::RwLock; pub use thread_parking::Parker; diff --git a/library/std/src/sys/sync/mutex/fuchsia.rs b/library/std/src/sys/sync/mutex/fuchsia.rs index 81a6361a83a49..3e871285bea01 100644 --- a/library/std/src/sys/sync/mutex/fuchsia.rs +++ b/library/std/src/sys/sync/mutex/fuchsia.rs @@ -40,9 +40,9 @@ use crate::sync::atomic::AtomicU32; use crate::sync::atomic::Ordering::{Acquire, Relaxed, Release}; use crate::sys::futex::zircon::{ - zx_futex_wait, zx_futex_wake_single_owner, zx_handle_t, zx_thread_self, ZX_ERR_BAD_HANDLE, - ZX_ERR_BAD_STATE, ZX_ERR_INVALID_ARGS, ZX_ERR_TIMED_OUT, ZX_ERR_WRONG_TYPE, ZX_OK, - ZX_TIME_INFINITE, + ZX_ERR_BAD_HANDLE, ZX_ERR_BAD_STATE, ZX_ERR_INVALID_ARGS, ZX_ERR_TIMED_OUT, ZX_ERR_WRONG_TYPE, + ZX_OK, ZX_TIME_INFINITE, zx_futex_wait, zx_futex_wake_single_owner, zx_handle_t, + zx_thread_self, }; // The lowest two bits of a `zx_handle_t` are always set, so the lowest bit is used to mark the diff --git a/library/std/src/sys/sync/mutex/futex.rs b/library/std/src/sys/sync/mutex/futex.rs index 81afa94b14787..ce9b2daa5f808 100644 --- a/library/std/src/sys/sync/mutex/futex.rs +++ b/library/std/src/sys/sync/mutex/futex.rs @@ -1,11 +1,11 @@ use crate::sync::atomic::Ordering::{Acquire, Relaxed, Release}; use crate::sys::futex::{self, futex_wait, futex_wake}; -type Atomic = futex::SmallAtomic; +type Futex = futex::SmallFutex; type State = futex::SmallPrimitive; pub struct Mutex { - futex: Atomic, + futex: Futex, } const UNLOCKED: State = 0; @@ -15,7 +15,7 @@ const CONTENDED: State = 2; // locked, and other threads waiting (contended) impl Mutex { #[inline] pub const fn new() -> Self { - Self { futex: Atomic::new(UNLOCKED) } + Self { futex: Futex::new(UNLOCKED) } } #[inline] diff --git a/library/std/src/sys/sync/mutex/itron.rs b/library/std/src/sys/sync/mutex/itron.rs index 8440ffdd33772..dcdfff0c81cd7 100644 --- a/library/std/src/sys/sync/mutex/itron.rs +++ b/library/std/src/sys/sync/mutex/itron.rs @@ -3,7 +3,7 @@ #![forbid(unsafe_op_in_unsafe_fn)] use crate::sys::pal::itron::abi; -use crate::sys::pal::itron::error::{expect_success, expect_success_aborting, fail, ItronError}; +use crate::sys::pal::itron::error::{ItronError, expect_success, expect_success_aborting, fail}; use crate::sys::pal::itron::spin::SpinIdOnceCell; pub struct Mutex { diff --git a/library/std/src/sys/sync/mutex/mod.rs b/library/std/src/sys/sync/mutex/mod.rs index 73d9bd273de17..360df3fc4b55d 100644 --- a/library/std/src/sys/sync/mutex/mod.rs +++ b/library/std/src/sys/sync/mutex/mod.rs @@ -19,7 +19,7 @@ cfg_if::cfg_if! { target_os = "teeos", ))] { mod pthread; - pub use pthread::{Mutex, raw}; + pub use pthread::Mutex; } else if #[cfg(all(target_os = "windows", target_vendor = "win7"))] { mod windows7; pub use windows7::{Mutex, raw}; diff --git a/library/std/src/sys/sync/mutex/no_threads.rs b/library/std/src/sys/sync/mutex/no_threads.rs index 4a13c55fb8bec..7b243575e018e 100644 --- a/library/std/src/sys/sync/mutex/no_threads.rs +++ b/library/std/src/sys/sync/mutex/no_threads.rs @@ -10,7 +10,7 @@ unsafe impl Sync for Mutex {} // no threads on this platform impl Mutex { #[inline] - #[rustc_const_stable(feature = "const_locks", since = "1.63.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_locks", since = "1.63.0"))] pub const fn new() -> Mutex { Mutex { locked: Cell::new(false) } } diff --git a/library/std/src/sys/sync/mutex/pthread.rs b/library/std/src/sys/sync/mutex/pthread.rs index ee0794334fbe3..abd58122523cf 100644 --- a/library/std/src/sys/sync/mutex/pthread.rs +++ b/library/std/src/sys/sync/mutex/pthread.rs @@ -1,25 +1,20 @@ use crate::cell::UnsafeCell; use crate::io::Error; -use crate::mem::{forget, MaybeUninit}; +use crate::mem::{MaybeUninit, forget}; use crate::sys::cvt_nz; -use crate::sys_common::lazy_box::{LazyBox, LazyInit}; +use crate::sys::sync::OnceBox; struct AllocatedMutex(UnsafeCell); pub struct Mutex { - inner: LazyBox, -} - -#[inline] -pub unsafe fn raw(m: &Mutex) -> *mut libc::pthread_mutex_t { - m.inner.0.get() + inner: OnceBox, } unsafe impl Send for AllocatedMutex {} unsafe impl Sync for AllocatedMutex {} -impl LazyInit for AllocatedMutex { - fn init() -> Box { +impl AllocatedMutex { + fn new() -> Box { let mutex = Box::new(AllocatedMutex(UnsafeCell::new(libc::PTHREAD_MUTEX_INITIALIZER))); // Issue #33770 @@ -60,24 +55,6 @@ impl LazyInit for AllocatedMutex { mutex } - - fn destroy(mutex: Box) { - // We're not allowed to pthread_mutex_destroy a locked mutex, - // so check first if it's unlocked. - if unsafe { libc::pthread_mutex_trylock(mutex.0.get()) == 0 } { - unsafe { libc::pthread_mutex_unlock(mutex.0.get()) }; - drop(mutex); - } else { - // The mutex is locked. This happens if a MutexGuard is leaked. - // In this case, we just leak the Mutex too. - forget(mutex); - } - } - - fn cancel_init(_: Box) { - // In this case, we can just drop it without any checks, - // since it cannot have been locked yet. - } } impl Drop for AllocatedMutex { @@ -88,7 +65,7 @@ impl Drop for AllocatedMutex { // On DragonFly pthread_mutex_destroy() returns EINVAL if called on a // mutex that was just initialized with libc::PTHREAD_MUTEX_INITIALIZER. // Once it is used (locked/unlocked) or pthread_mutex_init() is called, - // this behaviour no longer occurs. + // this behavior no longer occurs. debug_assert!(r == 0 || r == libc::EINVAL); } else { debug_assert_eq!(r, 0); @@ -99,11 +76,33 @@ impl Drop for AllocatedMutex { impl Mutex { #[inline] pub const fn new() -> Mutex { - Mutex { inner: LazyBox::new() } + Mutex { inner: OnceBox::new() } + } + + /// Gets access to the pthread mutex under the assumption that the mutex is + /// locked. + /// + /// This allows skipping the initialization check, as the mutex can only be + /// locked if it is already initialized, and allows relaxing the ordering + /// on the pointer load, since the allocation cannot have been modified + /// since the `lock` and the lock must have occurred on the current thread. + /// + /// # Safety + /// Causes undefined behavior if the mutex is not locked. + #[inline] + pub(crate) unsafe fn get_assert_locked(&self) -> *mut libc::pthread_mutex_t { + unsafe { self.inner.get_unchecked().0.get() } } #[inline] - pub unsafe fn lock(&self) { + fn get(&self) -> *mut libc::pthread_mutex_t { + // If initialization fails, the mutex is destroyed. This is always sound, + // however, as the mutex cannot have been locked yet. + self.inner.get_or_init(AllocatedMutex::new).0.get() + } + + #[inline] + pub fn lock(&self) { #[cold] #[inline(never)] fn fail(r: i32) -> ! { @@ -111,7 +110,7 @@ impl Mutex { panic!("failed to lock mutex: {error}"); } - let r = libc::pthread_mutex_lock(raw(self)); + let r = unsafe { libc::pthread_mutex_lock(self.get()) }; // As we set the mutex type to `PTHREAD_MUTEX_NORMAL` above, we expect // the lock call to never fail. Unfortunately however, some platforms // (Solaris) do not conform to the standard, and instead always provide @@ -126,13 +125,29 @@ impl Mutex { #[inline] pub unsafe fn unlock(&self) { - let r = libc::pthread_mutex_unlock(raw(self)); + let r = libc::pthread_mutex_unlock(self.get_assert_locked()); debug_assert_eq!(r, 0); } #[inline] - pub unsafe fn try_lock(&self) -> bool { - libc::pthread_mutex_trylock(raw(self)) == 0 + pub fn try_lock(&self) -> bool { + unsafe { libc::pthread_mutex_trylock(self.get()) == 0 } + } +} + +impl Drop for Mutex { + fn drop(&mut self) { + let Some(mutex) = self.inner.take() else { return }; + // We're not allowed to pthread_mutex_destroy a locked mutex, + // so check first if it's unlocked. + if unsafe { libc::pthread_mutex_trylock(mutex.0.get()) == 0 } { + unsafe { libc::pthread_mutex_unlock(mutex.0.get()) }; + drop(mutex); + } else { + // The mutex is locked. This happens if a MutexGuard is leaked. + // In this case, we just leak the Mutex too. + forget(mutex); + } } } diff --git a/library/std/src/sys/sync/mutex/sgx.rs b/library/std/src/sys/sync/mutex/sgx.rs index d37bd02adf8dd..8529e85797043 100644 --- a/library/std/src/sys/sync/mutex/sgx.rs +++ b/library/std/src/sys/sync/mutex/sgx.rs @@ -1,28 +1,24 @@ -use crate::sys::pal::waitqueue::{try_lock_or_false, SpinMutex, WaitQueue, WaitVariable}; -use crate::sys_common::lazy_box::{LazyBox, LazyInit}; - -/// FIXME: `UnsafeList` is not movable. -struct AllocatedMutex(SpinMutex>); +use crate::sys::pal::waitqueue::{SpinMutex, WaitQueue, WaitVariable, try_lock_or_false}; +use crate::sys::sync::OnceBox; pub struct Mutex { - inner: LazyBox, -} - -impl LazyInit for AllocatedMutex { - fn init() -> Box { - Box::new(AllocatedMutex(SpinMutex::new(WaitVariable::new(false)))) - } + // FIXME: `UnsafeList` is not movable. + inner: OnceBox>>, } // Implementation according to “Operating Systems: Three Easy Pieces”, chapter 28 impl Mutex { pub const fn new() -> Mutex { - Mutex { inner: LazyBox::new() } + Mutex { inner: OnceBox::new() } + } + + fn get(&self) -> &SpinMutex> { + self.inner.get_or_init(|| Box::new(SpinMutex::new(WaitVariable::new(false)))) } #[inline] pub fn lock(&self) { - let mut guard = self.inner.0.lock(); + let mut guard = self.get().lock(); if *guard.lock_var() { // Another thread has the lock, wait WaitQueue::wait(guard, || {}) @@ -35,7 +31,9 @@ impl Mutex { #[inline] pub unsafe fn unlock(&self) { - let guard = self.inner.0.lock(); + // SAFETY: the mutex was locked by the current thread, so it has been + // initialized already. + let guard = unsafe { self.inner.get_unchecked().lock() }; if let Err(mut guard) = WaitQueue::notify_one(guard) { // No other waiters, unlock *guard.lock_var_mut() = false; @@ -46,7 +44,7 @@ impl Mutex { #[inline] pub fn try_lock(&self) -> bool { - let mut guard = try_lock_or_false!(self.inner.0); + let mut guard = try_lock_or_false!(self.get()); if *guard.lock_var() { // Another thread has the lock false diff --git a/library/std/src/sys/sync/mutex/xous.rs b/library/std/src/sys/sync/mutex/xous.rs index 63efa5a0210ab..c6b954c1711e6 100644 --- a/library/std/src/sys/sync/mutex/xous.rs +++ b/library/std/src/sys/sync/mutex/xous.rs @@ -1,5 +1,5 @@ use crate::os::xous::ffi::{blocking_scalar, do_yield}; -use crate::os::xous::services::{ticktimer_server, TicktimerScalar}; +use crate::os::xous::services::{TicktimerScalar, ticktimer_server}; use crate::sync::atomic::Ordering::{Acquire, Relaxed, Release}; use crate::sync::atomic::{AtomicBool, AtomicUsize}; @@ -24,7 +24,6 @@ pub struct Mutex { impl Mutex { #[inline] - #[rustc_const_stable(feature = "const_locks", since = "1.63.0")] pub const fn new() -> Mutex { Mutex { locked: AtomicUsize::new(0), contended: AtomicBool::new(false) } } diff --git a/library/std/src/sys/sync/once/futex.rs b/library/std/src/sys/sync/once/futex.rs index 2c8a054282b01..10bfa81a6d72a 100644 --- a/library/std/src/sys/sync/once/futex.rs +++ b/library/std/src/sys/sync/once/futex.rs @@ -1,39 +1,38 @@ use crate::cell::Cell; use crate::sync as public; -use crate::sync::atomic::AtomicU32; use crate::sync::atomic::Ordering::{Acquire, Relaxed, Release}; use crate::sync::once::ExclusiveState; -use crate::sys::futex::{futex_wait, futex_wake_all}; +use crate::sys::futex::{Futex, Primitive, futex_wait, futex_wake_all}; // On some platforms, the OS is very nice and handles the waiter queue for us. // This means we only need one atomic value with 4 states: /// No initialization has run yet, and no thread is currently using the Once. -const INCOMPLETE: u32 = 0; +const INCOMPLETE: Primitive = 0; /// Some thread has previously attempted to initialize the Once, but it panicked, /// so the Once is now poisoned. There are no other threads currently accessing /// this Once. -const POISONED: u32 = 1; +const POISONED: Primitive = 1; /// Some thread is currently attempting to run initialization. It may succeed, /// so all future threads need to wait for it to finish. -const RUNNING: u32 = 2; +const RUNNING: Primitive = 2; /// Initialization has completed and all future calls should finish immediately. -const COMPLETE: u32 = 3; +const COMPLETE: Primitive = 3; // An additional bit indicates whether there are waiting threads: /// May only be set if the state is not COMPLETE. -const QUEUED: u32 = 4; +const QUEUED: Primitive = 4; // Threads wait by setting the QUEUED bit and calling `futex_wait` on the state // variable. When the running thread finishes, it will wake all waiting threads using // `futex_wake_all`. -const STATE_MASK: u32 = 0b11; +const STATE_MASK: Primitive = 0b11; pub struct OnceState { poisoned: bool, - set_state_to: Cell, + set_state_to: Cell, } impl OnceState { @@ -49,8 +48,8 @@ impl OnceState { } struct CompletionGuard<'a> { - state_and_queued: &'a AtomicU32, - set_state_on_drop_to: u32, + state_and_queued: &'a Futex, + set_state_on_drop_to: Primitive, } impl<'a> Drop for CompletionGuard<'a> { @@ -65,13 +64,13 @@ impl<'a> Drop for CompletionGuard<'a> { } pub struct Once { - state_and_queued: AtomicU32, + state_and_queued: Futex, } impl Once { #[inline] pub const fn new() -> Once { - Once { state_and_queued: AtomicU32::new(INCOMPLETE) } + Once { state_and_queued: Futex::new(INCOMPLETE) } } #[inline] @@ -91,6 +90,15 @@ impl Once { } } + #[inline] + pub(crate) fn set_state(&mut self, new_state: ExclusiveState) { + *self.state_and_queued.get_mut() = match new_state { + ExclusiveState::Incomplete => INCOMPLETE, + ExclusiveState::Poisoned => POISONED, + ExclusiveState::Complete => COMPLETE, + }; + } + #[cold] #[track_caller] pub fn wait(&self, ignore_poisoning: bool) { diff --git a/library/std/src/sys/sync/once/no_threads.rs b/library/std/src/sys/sync/once/no_threads.rs index 12c1d9f5a6c98..fb1b496510aba 100644 --- a/library/std/src/sys/sync/once/no_threads.rs +++ b/library/std/src/sys/sync/once/no_threads.rs @@ -35,7 +35,7 @@ unsafe impl Sync for Once {} impl Once { #[inline] - #[rustc_const_stable(feature = "const_once_new", since = "1.32.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_once_new", since = "1.32.0"))] pub const fn new() -> Once { Once { state: Cell::new(State::Incomplete) } } @@ -55,6 +55,15 @@ impl Once { } } + #[inline] + pub(crate) fn set_state(&mut self, new_state: ExclusiveState) { + self.state.set(match new_state { + ExclusiveState::Incomplete => State::Incomplete, + ExclusiveState::Poisoned => State::Poisoned, + ExclusiveState::Complete => State::Complete, + }); + } + #[cold] #[track_caller] pub fn wait(&self, _ignore_poisoning: bool) { diff --git a/library/std/src/sys/sync/once/queue.rs b/library/std/src/sys/sync/once/queue.rs index 86f72c82008bc..177d0d7744a6d 100644 --- a/library/std/src/sys/sync/once/queue.rs +++ b/library/std/src/sys/sync/once/queue.rs @@ -23,7 +23,7 @@ // You'll find a few more details in the implementation, but that's the gist of // it! // -// Atomic orderings: +// Futex orderings: // When running `Once` we deal with multiple atomics: // `Once.state_and_queue` and an unknown number of `Waiter.signaled`. // * `state_and_queue` is used (1) as a state flag, (2) for synchronizing the @@ -116,7 +116,7 @@ fn to_state(current: StateAndQueue) -> usize { impl Once { #[inline] - #[rustc_const_stable(feature = "const_once_new", since = "1.32.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_once_new", since = "1.32.0"))] pub const fn new() -> Once { Once { state_and_queue: AtomicPtr::new(ptr::without_provenance_mut(INCOMPLETE)) } } @@ -140,6 +140,15 @@ impl Once { } } + #[inline] + pub(crate) fn set_state(&mut self, new_state: ExclusiveState) { + *self.state_and_queue.get_mut() = match new_state { + ExclusiveState::Incomplete => ptr::without_provenance_mut(INCOMPLETE), + ExclusiveState::Poisoned => ptr::without_provenance_mut(POISONED), + ExclusiveState::Complete => ptr::without_provenance_mut(COMPLETE), + }; + } + #[cold] #[track_caller] pub fn wait(&self, ignore_poisoning: bool) { diff --git a/library/std/src/sys/sync/once_box.rs b/library/std/src/sys/sync/once_box.rs new file mode 100644 index 0000000000000..4105af503295f --- /dev/null +++ b/library/std/src/sys/sync/once_box.rs @@ -0,0 +1,82 @@ +//! A racily-initialized alternative to `OnceLock>`. +//! +//! This is used to implement synchronization primitives that need allocation, +//! like the pthread versions. + +#![allow(dead_code)] // Only used on some platforms. + +use crate::mem::replace; +use crate::ptr::null_mut; +use crate::sync::atomic::AtomicPtr; +use crate::sync::atomic::Ordering::{Acquire, Relaxed, Release}; + +pub(crate) struct OnceBox { + ptr: AtomicPtr, +} + +impl OnceBox { + #[inline] + pub const fn new() -> Self { + Self { ptr: AtomicPtr::new(null_mut()) } + } + + /// Gets access to the value, assuming it is already initialized and this + /// initialization has been observed by the current thread. + /// + /// Since all modifications to the pointer have already been observed, the + /// pointer load in this function can be performed with relaxed ordering, + /// potentially allowing the optimizer to turn code like this: + /// ```rust, ignore + /// once_box.get_or_init(|| Box::new(42)); + /// unsafe { once_box.get_unchecked() } + /// ``` + /// into + /// ```rust, ignore + /// once_box.get_or_init(|| Box::new(42)) + /// ``` + /// + /// # Safety + /// This causes undefined behavior if the assumption above is violated. + #[inline] + pub unsafe fn get_unchecked(&self) -> &T { + unsafe { &*self.ptr.load(Relaxed) } + } + + #[inline] + pub fn get_or_init(&self, f: impl FnOnce() -> Box) -> &T { + let ptr = self.ptr.load(Acquire); + match unsafe { ptr.as_ref() } { + Some(val) => val, + None => self.initialize(f), + } + } + + #[inline] + pub fn take(&mut self) -> Option> { + let ptr = replace(self.ptr.get_mut(), null_mut()); + if !ptr.is_null() { Some(unsafe { Box::from_raw(ptr) }) } else { None } + } + + #[cold] + fn initialize(&self, f: impl FnOnce() -> Box) -> &T { + let new_ptr = Box::into_raw(f()); + match self.ptr.compare_exchange(null_mut(), new_ptr, Release, Acquire) { + Ok(_) => unsafe { &*new_ptr }, + Err(ptr) => { + // Lost the race to another thread. + // Drop the value we created, and use the one from the other thread instead. + drop(unsafe { Box::from_raw(new_ptr) }); + unsafe { &*ptr } + } + } + } +} + +unsafe impl Send for OnceBox {} +unsafe impl Sync for OnceBox {} + +impl Drop for OnceBox { + fn drop(&mut self) { + self.take(); + } +} diff --git a/library/std/src/sys/sync/rwlock/futex.rs b/library/std/src/sys/sync/rwlock/futex.rs index 75ecc2ab5c52f..447048edf7622 100644 --- a/library/std/src/sys/sync/rwlock/futex.rs +++ b/library/std/src/sys/sync/rwlock/futex.rs @@ -1,6 +1,5 @@ -use crate::sync::atomic::AtomicU32; use crate::sync::atomic::Ordering::{Acquire, Relaxed, Release}; -use crate::sys::futex::{futex_wait, futex_wake, futex_wake_all}; +use crate::sys::futex::{Futex, Primitive, futex_wait, futex_wake, futex_wake_all}; pub struct RwLock { // The state consists of a 30-bit reader counter, a 'readers waiting' flag, and a 'writers waiting' flag. @@ -10,41 +9,41 @@ pub struct RwLock { // 0x3FFF_FFFF: Write locked // Bit 30: Readers are waiting on this futex. // Bit 31: Writers are waiting on the writer_notify futex. - state: AtomicU32, + state: Futex, // The 'condition variable' to notify writers through. // Incremented on every signal. - writer_notify: AtomicU32, + writer_notify: Futex, } -const READ_LOCKED: u32 = 1; -const MASK: u32 = (1 << 30) - 1; -const WRITE_LOCKED: u32 = MASK; -const MAX_READERS: u32 = MASK - 1; -const READERS_WAITING: u32 = 1 << 30; -const WRITERS_WAITING: u32 = 1 << 31; +const READ_LOCKED: Primitive = 1; +const MASK: Primitive = (1 << 30) - 1; +const WRITE_LOCKED: Primitive = MASK; +const MAX_READERS: Primitive = MASK - 1; +const READERS_WAITING: Primitive = 1 << 30; +const WRITERS_WAITING: Primitive = 1 << 31; #[inline] -fn is_unlocked(state: u32) -> bool { +fn is_unlocked(state: Primitive) -> bool { state & MASK == 0 } #[inline] -fn is_write_locked(state: u32) -> bool { +fn is_write_locked(state: Primitive) -> bool { state & MASK == WRITE_LOCKED } #[inline] -fn has_readers_waiting(state: u32) -> bool { +fn has_readers_waiting(state: Primitive) -> bool { state & READERS_WAITING != 0 } #[inline] -fn has_writers_waiting(state: u32) -> bool { +fn has_writers_waiting(state: Primitive) -> bool { state & WRITERS_WAITING != 0 } #[inline] -fn is_read_lockable(state: u32) -> bool { +fn is_read_lockable(state: Primitive) -> bool { // This also returns false if the counter could overflow if we tried to read lock it. // // We don't allow read-locking if there's readers waiting, even if the lock is unlocked @@ -55,14 +54,14 @@ fn is_read_lockable(state: u32) -> bool { } #[inline] -fn has_reached_max_readers(state: u32) -> bool { +fn has_reached_max_readers(state: Primitive) -> bool { state & MASK == MAX_READERS } impl RwLock { #[inline] pub const fn new() -> Self { - Self { state: AtomicU32::new(0), writer_notify: AtomicU32::new(0) } + Self { state: Futex::new(0), writer_notify: Futex::new(0) } } #[inline] @@ -225,7 +224,7 @@ impl RwLock { /// If both are waiting, this will wake up only one writer, but will fall /// back to waking up readers if there was no writer to wake up. #[cold] - fn wake_writer_or_readers(&self, mut state: u32) { + fn wake_writer_or_readers(&self, mut state: Primitive) { assert!(is_unlocked(state)); // The readers waiting bit might be turned on at any point now, @@ -284,13 +283,13 @@ impl RwLock { futex_wake(&self.writer_notify) // Note that FreeBSD and DragonFlyBSD don't tell us whether they woke // up any threads or not, and always return `false` here. That still - // results in correct behaviour: it just means readers get woken up as + // results in correct behavior: it just means readers get woken up as // well in case both readers and writers were waiting. } /// Spin for a while, but stop directly at the given condition. #[inline] - fn spin_until(&self, f: impl Fn(u32) -> bool) -> u32 { + fn spin_until(&self, f: impl Fn(Primitive) -> bool) -> Primitive { let mut spin = 100; // Chosen by fair dice roll. loop { let state = self.state.load(Relaxed); @@ -303,13 +302,13 @@ impl RwLock { } #[inline] - fn spin_write(&self) -> u32 { + fn spin_write(&self) -> Primitive { // Stop spinning when it's unlocked or when there's waiting writers, to keep things somewhat fair. self.spin_until(|state| is_unlocked(state) || has_writers_waiting(state)) } #[inline] - fn spin_read(&self) -> u32 { + fn spin_read(&self) -> Primitive { // Stop spinning when it's unlocked or read locked, or when there's waiting threads. self.spin_until(|state| { !is_write_locked(state) || has_readers_waiting(state) || has_writers_waiting(state) diff --git a/library/std/src/sys/sync/rwlock/no_threads.rs b/library/std/src/sys/sync/rwlock/no_threads.rs index 789ef9b29e52a..6965e2e2cabe5 100644 --- a/library/std/src/sys/sync/rwlock/no_threads.rs +++ b/library/std/src/sys/sync/rwlock/no_threads.rs @@ -10,7 +10,7 @@ unsafe impl Sync for RwLock {} // no threads on this platform impl RwLock { #[inline] - #[rustc_const_stable(feature = "const_locks", since = "1.63.0")] + #[cfg_attr(bootstrap, rustc_const_stable(feature = "const_locks", since = "1.63.0"))] pub const fn new() -> RwLock { RwLock { mode: Cell::new(0) } } diff --git a/library/std/src/sys/sync/rwlock/queue.rs b/library/std/src/sys/sync/rwlock/queue.rs index 458c16516bbe1..889961915f4e6 100644 --- a/library/std/src/sys/sync/rwlock/queue.rs +++ b/library/std/src/sys/sync/rwlock/queue.rs @@ -8,7 +8,7 @@ //! * `pthread` is an external library, meaning the fast path of acquiring an //! uncontended lock cannot be inlined. //! * Some platforms (at least glibc before version 2.25) have buggy implementations -//! that can easily lead to undefined behaviour in safe Rust code when not properly +//! that can easily lead to undefined behavior in safe Rust code when not properly //! guarded against. //! * On some platforms (e.g. macOS), the lock is very slow. //! @@ -110,10 +110,10 @@ use crate::cell::OnceCell; use crate::hint::spin_loop; use crate::mem; -use crate::ptr::{self, null_mut, without_provenance_mut, NonNull}; +use crate::ptr::{self, NonNull, null_mut, without_provenance_mut}; use crate::sync::atomic::Ordering::{AcqRel, Acquire, Relaxed, Release}; use crate::sync::atomic::{AtomicBool, AtomicPtr}; -use crate::thread::{self, Thread}; +use crate::thread::{self, Thread, ThreadId}; // Locking uses exponential backoff. `SPIN_COUNT` indicates how many times the // locking operation will be retried. @@ -200,7 +200,9 @@ impl Node { fn prepare(&mut self) { // Fall back to creating an unnamed `Thread` handle to allow locking in // TLS destructors. - self.thread.get_or_init(|| thread::try_current().unwrap_or_else(Thread::new_unnamed)); + self.thread.get_or_init(|| { + thread::try_current().unwrap_or_else(|| Thread::new_unnamed(ThreadId::new())) + }); self.completed = AtomicBool::new(false); } diff --git a/library/std/src/sys/sync/rwlock/solid.rs b/library/std/src/sys/sync/rwlock/solid.rs index 0537140202091..7703082f95116 100644 --- a/library/std/src/sys/sync/rwlock/solid.rs +++ b/library/std/src/sys/sync/rwlock/solid.rs @@ -2,7 +2,7 @@ #![forbid(unsafe_op_in_unsafe_fn)] use crate::sys::pal::abi; -use crate::sys::pal::itron::error::{expect_success, expect_success_aborting, fail, ItronError}; +use crate::sys::pal::itron::error::{ItronError, expect_success, expect_success_aborting, fail}; use crate::sys::pal::itron::spin::SpinIdOnceCell; pub struct RwLock { diff --git a/library/std/src/sys/sync/rwlock/teeos.rs b/library/std/src/sys/sync/rwlock/teeos.rs index ef9b1ab51546c..763430223834b 100644 --- a/library/std/src/sys/sync/rwlock/teeos.rs +++ b/library/std/src/sys/sync/rwlock/teeos.rs @@ -14,22 +14,22 @@ impl RwLock { #[inline] pub fn read(&self) { - unsafe { self.inner.lock() }; + self.inner.lock() } #[inline] pub fn try_read(&self) -> bool { - unsafe { self.inner.try_lock() } + self.inner.try_lock() } #[inline] pub fn write(&self) { - unsafe { self.inner.lock() }; + self.inner.lock() } #[inline] pub unsafe fn try_write(&self) -> bool { - unsafe { self.inner.try_lock() } + self.inner.try_lock() } #[inline] diff --git a/library/std/src/sys/sync/thread_parking/darwin.rs b/library/std/src/sys/sync/thread_parking/darwin.rs index 96e3d23c332c4..0553c5e19a91f 100644 --- a/library/std/src/sys/sync/thread_parking/darwin.rs +++ b/library/std/src/sys/sync/thread_parking/darwin.rs @@ -5,7 +5,7 @@ //! rejection from the App Store). //! //! Therefore, we need to look for other synchronization primitives. Luckily, Darwin -//! supports semaphores, which allow us to implement the behaviour we need with +//! supports semaphores, which allow us to implement the behavior we need with //! only one primitive (as opposed to a mutex-condvar pair). We use the semaphore //! provided by libdispatch, as the underlying Mach semaphore is only dubiously //! public. diff --git a/library/std/src/sys/sync/thread_parking/futex.rs b/library/std/src/sys/sync/thread_parking/futex.rs index ce852eaadc4d9..c8f7f26386a01 100644 --- a/library/std/src/sys/sync/thread_parking/futex.rs +++ b/library/std/src/sys/sync/thread_parking/futex.rs @@ -4,7 +4,7 @@ use crate::sync::atomic::Ordering::{Acquire, Release}; use crate::sys::futex::{self, futex_wait, futex_wake}; use crate::time::Duration; -type Atomic = futex::SmallAtomic; +type Futex = futex::SmallFutex; type State = futex::SmallPrimitive; const PARKED: State = State::MAX; @@ -12,7 +12,7 @@ const EMPTY: State = 0; const NOTIFIED: State = 1; pub struct Parker { - state: Atomic, + state: Futex, } // Notes about memory ordering: @@ -39,7 +39,7 @@ impl Parker { /// Constructs the futex parker. The UNIX parker implementation /// requires this to happen in-place. pub unsafe fn new_in_place(parker: *mut Parker) { - unsafe { parker.write(Self { state: Atomic::new(EMPTY) }) }; + unsafe { parker.write(Self { state: Futex::new(EMPTY) }) }; } // Assumes this is only called by the thread that owns the Parker, diff --git a/library/std/src/sys/sync/thread_parking/id.rs b/library/std/src/sys/sync/thread_parking/id.rs index a7b07b509dfd8..6496435183770 100644 --- a/library/std/src/sys/sync/thread_parking/id.rs +++ b/library/std/src/sys/sync/thread_parking/id.rs @@ -10,8 +10,8 @@ use crate::cell::UnsafeCell; use crate::pin::Pin; use crate::sync::atomic::Ordering::{Acquire, Relaxed, Release}; -use crate::sync::atomic::{fence, AtomicI8}; -use crate::sys::thread_parking::{current, park, park_timeout, unpark, ThreadId}; +use crate::sync::atomic::{AtomicI8, fence}; +use crate::sys::thread_parking::{ThreadId, current, park, park_timeout, unpark}; use crate::time::Duration; pub struct Parker { diff --git a/library/std/src/sys/sync/thread_parking/mod.rs b/library/std/src/sys/sync/thread_parking/mod.rs index 0ebc5e093ee2a..f4d8fa0a58c11 100644 --- a/library/std/src/sys/sync/thread_parking/mod.rs +++ b/library/std/src/sys/sync/thread_parking/mod.rs @@ -23,6 +23,7 @@ cfg_if::cfg_if! { mod windows7; pub use windows7::Parker; } else if #[cfg(all(target_vendor = "apple", not(miri)))] { + // Doesn't work in Miri, see . mod darwin; pub use darwin::Parker; } else if #[cfg(target_os = "xous")] { diff --git a/library/std/src/sys/sync/thread_parking/pthread.rs b/library/std/src/sys/sync/thread_parking/pthread.rs index c64600e9e29c3..76df73b2a8e06 100644 --- a/library/std/src/sys/sync/thread_parking/pthread.rs +++ b/library/std/src/sys/sync/thread_parking/pthread.rs @@ -3,7 +3,6 @@ use crate::cell::UnsafeCell; use crate::marker::PhantomPinned; use crate::pin::Pin; -use crate::ptr::addr_of_mut; use crate::sync::atomic::AtomicUsize; use crate::sync::atomic::Ordering::{Acquire, Relaxed, Release}; #[cfg(not(target_os = "nto"))] @@ -98,11 +97,11 @@ impl Parker { /// The constructed parker must never be moved. pub unsafe fn new_in_place(parker: *mut Parker) { // Use the default mutex implementation to allow for simpler initialization. - // This could lead to undefined behaviour when deadlocking. This is avoided + // This could lead to undefined behavior when deadlocking. This is avoided // by not deadlocking. Note in particular the unlocking operation before any // panic, as code after the panic could try to park again. - addr_of_mut!((*parker).state).write(AtomicUsize::new(EMPTY)); - addr_of_mut!((*parker).lock).write(UnsafeCell::new(libc::PTHREAD_MUTEX_INITIALIZER)); + (&raw mut (*parker).state).write(AtomicUsize::new(EMPTY)); + (&raw mut (*parker).lock).write(UnsafeCell::new(libc::PTHREAD_MUTEX_INITIALIZER)); cfg_if::cfg_if! { if #[cfg(any( @@ -112,9 +111,9 @@ impl Parker { target_os = "vita", target_vendor = "apple", ))] { - addr_of_mut!((*parker).cvar).write(UnsafeCell::new(libc::PTHREAD_COND_INITIALIZER)); + (&raw mut (*parker).cvar).write(UnsafeCell::new(libc::PTHREAD_COND_INITIALIZER)); } else if #[cfg(any(target_os = "espidf", target_os = "horizon"))] { - let r = libc::pthread_cond_init(addr_of_mut!((*parker).cvar).cast(), crate::ptr::null()); + let r = libc::pthread_cond_init((&raw mut (*parker).cvar).cast(), crate::ptr::null()); assert_eq!(r, 0); } else { use crate::mem::MaybeUninit; @@ -123,7 +122,7 @@ impl Parker { assert_eq!(r, 0); let r = libc::pthread_condattr_setclock(attr.as_mut_ptr(), libc::CLOCK_MONOTONIC); assert_eq!(r, 0); - let r = libc::pthread_cond_init(addr_of_mut!((*parker).cvar).cast(), attr.as_ptr()); + let r = libc::pthread_cond_init((&raw mut (*parker).cvar).cast(), attr.as_ptr()); assert_eq!(r, 0); let r = libc::pthread_condattr_destroy(attr.as_mut_ptr()); assert_eq!(r, 0); diff --git a/library/std/src/sys/sync/thread_parking/windows7.rs b/library/std/src/sys/sync/thread_parking/windows7.rs index 1000b63b6d01a..f7585e882f055 100644 --- a/library/std/src/sys/sync/thread_parking/windows7.rs +++ b/library/std/src/sys/sync/thread_parking/windows7.rs @@ -35,7 +35,7 @@ // different implementations. // // Unfortunately, NT Keyed Events are an undocumented Windows API. However: -// - This API is relatively simple with obvious behaviour, and there are +// - This API is relatively simple with obvious behavior, and there are // several (unofficial) articles documenting the details. [1] // - `parking_lot` has been using this API for years (on Windows versions // before Windows 8). [2] Many big projects extensively use parking_lot, @@ -43,7 +43,7 @@ // - It is the underlying API used by Windows SRW locks and Windows critical // sections. [3] [4] // - The source code of the implementations of Wine, ReactOs, and Windows XP -// are available and match the expected behaviour. +// are available and match the expected behavior. // - The main risk with an undocumented API is that it might change in the // future. But since we only use it for older versions of Windows, that's not // a problem. @@ -178,7 +178,7 @@ impl Parker { } fn ptr(&self) -> *const c_void { - core::ptr::addr_of!(self.state).cast::() + (&raw const self.state).cast::() } } @@ -190,7 +190,7 @@ mod keyed_events { use core::sync::atomic::Ordering::{Acquire, Relaxed}; use core::time::Duration; - use super::{Parker, EMPTY, NOTIFIED}; + use super::{EMPTY, NOTIFIED, Parker}; use crate::sys::c; pub unsafe fn park(parker: Pin<&Parker>) { diff --git a/library/std/src/sys/sync/thread_parking/xous.rs b/library/std/src/sys/sync/thread_parking/xous.rs index 64b6f731f2377..28c90249dc2c2 100644 --- a/library/std/src/sys/sync/thread_parking/xous.rs +++ b/library/std/src/sys/sync/thread_parking/xous.rs @@ -1,5 +1,5 @@ use crate::os::xous::ffi::{blocking_scalar, scalar}; -use crate::os::xous::services::{ticktimer_server, TicktimerScalar}; +use crate::os::xous::services::{TicktimerScalar, ticktimer_server}; use crate::pin::Pin; use crate::ptr; use crate::sync::atomic::AtomicI8; diff --git a/library/std/src/sys/thread_local/destructors/linux_like.rs b/library/std/src/sys/thread_local/destructors/linux_like.rs index c381be0bf8c76..f473dc4d79df5 100644 --- a/library/std/src/sys/thread_local/destructors/linux_like.rs +++ b/library/std/src/sys/thread_local/destructors/linux_like.rs @@ -47,7 +47,7 @@ pub unsafe fn register(t: *mut u8, dtor: unsafe extern "C" fn(*mut u8)) { dtor, ), t.cast(), - core::ptr::addr_of!(__dso_handle) as *mut _, + (&raw const __dso_handle) as *mut _, ); } } else { diff --git a/library/std/src/sys/thread_local/guard/apple.rs b/library/std/src/sys/thread_local/guard/apple.rs index 6c27f7ae35cba..fa25b116622fc 100644 --- a/library/std/src/sys/thread_local/guard/apple.rs +++ b/library/std/src/sys/thread_local/guard/apple.rs @@ -26,6 +26,7 @@ pub fn enable() { unsafe extern "C" fn run_dtors(_: *mut u8) { unsafe { destructors::run(); + crate::rt::thread_cleanup(); } } } diff --git a/library/std/src/sys/thread_local/guard/key.rs b/library/std/src/sys/thread_local/guard/key.rs index 67c3ca8862767..59581e6f281e6 100644 --- a/library/std/src/sys/thread_local/guard/key.rs +++ b/library/std/src/sys/thread_local/guard/key.rs @@ -3,10 +3,12 @@ //! that will run all native TLS destructors in the destructor list. use crate::ptr; -use crate::sys::thread_local::destructors; -use crate::sys::thread_local::key::{set, LazyKey}; +use crate::sys::thread_local::key::{LazyKey, set}; +#[cfg(target_thread_local)] pub fn enable() { + use crate::sys::thread_local::destructors; + static DTORS: LazyKey = LazyKey::new(Some(run)); // Setting the key value to something other than NULL will result in the @@ -18,6 +20,41 @@ pub fn enable() { unsafe extern "C" fn run(_: *mut u8) { unsafe { destructors::run(); + // On platforms with `__cxa_thread_atexit_impl`, `destructors::run` + // does nothing on newer systems as the TLS destructors are + // registered with the system. But because all of those platforms + // call the destructors of TLS keys after the registered ones, this + // function will still be run last (at the time of writing). + crate::rt::thread_cleanup(); + } + } +} + +/// On platforms with key-based TLS, the system runs the destructors for us. +/// We still have to make sure that [`crate::rt::thread_cleanup`] is called, +/// however. This is done by defering the execution of a TLS destructor to +/// the next round of destruction inside the TLS destructors. +#[cfg(not(target_thread_local))] +pub fn enable() { + const DEFER: *mut u8 = ptr::without_provenance_mut(1); + const RUN: *mut u8 = ptr::without_provenance_mut(2); + + static CLEANUP: LazyKey = LazyKey::new(Some(run)); + + unsafe { set(CLEANUP.force(), DEFER) } + + unsafe extern "C" fn run(state: *mut u8) { + if state == DEFER { + // Make sure that this function is run again in the next round of + // TLS destruction. If there is no futher round, there will be leaks, + // but that's okay, `thread_cleanup` is not guaranteed to be called. + unsafe { set(CLEANUP.force(), RUN) } + } else { + debug_assert_eq!(state, RUN); + // If the state is still RUN in the next round of TLS destruction, + // it means that no other TLS destructors defined by this runtime + // have been run, as they would have set the state to DEFER. + crate::rt::thread_cleanup(); } } } diff --git a/library/std/src/sys/thread_local/guard/solid.rs b/library/std/src/sys/thread_local/guard/solid.rs index 054b2d561c8b4..3bcd46c481dc0 100644 --- a/library/std/src/sys/thread_local/guard/solid.rs +++ b/library/std/src/sys/thread_local/guard/solid.rs @@ -19,6 +19,9 @@ pub fn enable() { } unsafe extern "C" fn tls_dtor(_unused: *mut u8) { - unsafe { destructors::run() }; + unsafe { + destructors::run(); + crate::rt::thread_cleanup(); + } } } diff --git a/library/std/src/sys/thread_local/guard/windows.rs b/library/std/src/sys/thread_local/guard/windows.rs index bf94f7d6e3d13..1752b0e1208af 100644 --- a/library/std/src/sys/thread_local/guard/windows.rs +++ b/library/std/src/sys/thread_local/guard/windows.rs @@ -26,7 +26,7 @@ //! This apparently translates to any callbacks in the ".CRT$XLB" section //! being run on certain events. //! -//! So after all that, we use the compiler's #[link_section] feature to place +//! So after all that, we use the compiler's `#[link_section]` feature to place //! a callback pointer into the magic section so it ends up being called. //! //! # What's up with this callback? @@ -80,13 +80,13 @@ pub static CALLBACK: unsafe extern "system" fn(*mut c_void, u32, *mut c_void) = unsafe extern "system" fn tls_callback(_h: *mut c_void, dw_reason: u32, _pv: *mut c_void) { if dw_reason == c::DLL_THREAD_DETACH || dw_reason == c::DLL_PROCESS_DETACH { - #[cfg(target_thread_local)] unsafe { + #[cfg(target_thread_local)] super::super::destructors::run(); - } - #[cfg(not(target_thread_local))] - unsafe { + #[cfg(not(target_thread_local))] super::super::key::run_dtors(); + + crate::rt::thread_cleanup(); } } } diff --git a/library/std/src/sys/thread_local/key/racy.rs b/library/std/src/sys/thread_local/key/racy.rs index 69f11458c3289..97df8997b80de 100644 --- a/library/std/src/sys/thread_local/key/racy.rs +++ b/library/std/src/sys/thread_local/key/racy.rs @@ -30,7 +30,7 @@ const KEY_SENTVAL: usize = 0; const KEY_SENTVAL: usize = libc::PTHREAD_KEYS_MAX + 1; impl LazyKey { - #[rustc_const_unstable(feature = "thread_local_internals", issue = "none")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "thread_local_internals", issue = "none"))] pub const fn new(dtor: Option) -> LazyKey { LazyKey { key: atomic::AtomicUsize::new(KEY_SENTVAL), dtor } } diff --git a/library/std/src/sys/thread_local/key/tests.rs b/library/std/src/sys/thread_local/key/tests.rs index d82b34e71f0e4..c7d2c8e6301ef 100644 --- a/library/std/src/sys/thread_local/key/tests.rs +++ b/library/std/src/sys/thread_local/key/tests.rs @@ -1,4 +1,4 @@ -use super::{get, set, LazyKey}; +use super::{LazyKey, get, set}; use crate::ptr; #[test] diff --git a/library/std/src/sys/thread_local/key/xous.rs b/library/std/src/sys/thread_local/key/xous.rs index 4fb2fdcc61925..2ab4bba7d8e98 100644 --- a/library/std/src/sys/thread_local/key/xous.rs +++ b/library/std/src/sys/thread_local/key/xous.rs @@ -39,7 +39,7 @@ use core::arch::asm; use crate::mem::ManuallyDrop; -use crate::os::xous::ffi::{map_memory, unmap_memory, MemoryFlags}; +use crate::os::xous::ffi::{MemoryFlags, map_memory, unmap_memory}; use crate::ptr; use crate::sync::atomic::Ordering::{Acquire, Relaxed, Release}; use crate::sync::atomic::{AtomicPtr, AtomicUsize}; @@ -212,4 +212,6 @@ unsafe fn run_dtors() { unsafe { cur = (*cur).next }; } } + + crate::rt::thread_cleanup(); } diff --git a/library/std/src/sys/thread_local/mod.rs b/library/std/src/sys/thread_local/mod.rs index 3d1b91a7ea095..31d3b43906004 100644 --- a/library/std/src/sys/thread_local/mod.rs +++ b/library/std/src/sys/thread_local/mod.rs @@ -31,12 +31,15 @@ cfg_if::cfg_if! { ))] { mod statik; pub use statik::{EagerStorage, LazyStorage, thread_local_inner}; + pub(crate) use statik::{LocalPointer, local_pointer}; } else if #[cfg(target_thread_local)] { mod native; pub use native::{EagerStorage, LazyStorage, thread_local_inner}; + pub(crate) use native::{LocalPointer, local_pointer}; } else { mod os; pub use os::{Storage, thread_local_inner}; + pub(crate) use os::{LocalPointer, local_pointer}; } } @@ -72,36 +75,47 @@ pub(crate) mod destructors { } /// This module provides a way to schedule the execution of the destructor list -/// on systems without a per-variable destructor system. -mod guard { +/// and the [runtime cleanup](crate::rt::thread_cleanup) function. Calling `enable` +/// should ensure that these functions are called at the right times. +pub(crate) mod guard { cfg_if::cfg_if! { if #[cfg(all(target_thread_local, target_vendor = "apple"))] { mod apple; - pub(super) use apple::enable; + pub(crate) use apple::enable; } else if #[cfg(target_os = "windows")] { mod windows; - pub(super) use windows::enable; + pub(crate) use windows::enable; } else if #[cfg(any( - all(target_family = "wasm", target_feature = "atomics"), + target_family = "wasm", + target_os = "uefi", + target_os = "zkvm", ))] { - pub(super) fn enable() { - // FIXME: Right now there is no concept of "thread exit", but - // this is likely going to show up at some point in the form of - // an exported symbol that the wasm runtime is going to be - // expected to call. For now we just leak everything, but if - // such a function starts to exist it will probably need to - // iterate the destructor list with this function: + pub(crate) fn enable() { + // FIXME: Right now there is no concept of "thread exit" on + // wasm, but this is likely going to show up at some point in + // the form of an exported symbol that the wasm runtime is going + // to be expected to call. For now we just leak everything, but + // if such a function starts to exist it will probably need to + // iterate the destructor list with these functions: + #[cfg(all(target_family = "wasm", target_feature = "atomics"))] #[allow(unused)] use super::destructors::run; + #[allow(unused)] + use crate::rt::thread_cleanup; } - } else if #[cfg(target_os = "hermit")] { - pub(super) fn enable() {} + } else if #[cfg(any( + target_os = "hermit", + target_os = "xous", + ))] { + // `std` is the only runtime, so it just calls the destructor functions + // itself when the time comes. + pub(crate) fn enable() {} } else if #[cfg(target_os = "solid_asp3")] { mod solid; - pub(super) use solid::enable; - } else if #[cfg(all(target_thread_local, not(target_family = "wasm")))] { + pub(crate) use solid::enable; + } else { mod key; - pub(super) use key::enable; + pub(crate) use key::enable; } } } diff --git a/library/std/src/sys/thread_local/native/mod.rs b/library/std/src/sys/thread_local/native/mod.rs index 1cc45fe892dee..a5dffe3c45883 100644 --- a/library/std/src/sys/thread_local/native/mod.rs +++ b/library/std/src/sys/thread_local/native/mod.rs @@ -29,6 +29,9 @@ //! eliminates the `Destroyed` state for these values, which can allow more niche //! optimizations to occur for the `State` enum. For `Drop` types, `()` is used. +use crate::cell::Cell; +use crate::ptr; + mod eager; mod lazy; @@ -46,20 +49,21 @@ pub use lazy::Storage as LazyStorage; #[unstable(feature = "thread_local_internals", issue = "none")] #[rustc_macro_transparency = "semitransparent"] pub macro thread_local_inner { - // used to generate the `LocalKey` value for const-initialized thread locals + // NOTE: we cannot import `LocalKey`, `LazyStorage` or `EagerStorage` with a `use` because that + // can shadow user provided type or type alias with a matching name. Please update the shadowing + // test in `tests/thread.rs` if these types are renamed. + + // Used to generate the `LocalKey` value for const-initialized thread locals. (@key $t:ty, const $init:expr) => {{ const __INIT: $t = $init; unsafe { - use $crate::mem::needs_drop; - use $crate::thread::LocalKey; - use $crate::thread::local_impl::EagerStorage; - - LocalKey::new(const { - if needs_drop::<$t>() { + $crate::thread::LocalKey::new(const { + if $crate::mem::needs_drop::<$t>() { |_| { #[thread_local] - static VAL: EagerStorage<$t> = EagerStorage::new(__INIT); + static VAL: $crate::thread::local_impl::EagerStorage<$t> + = $crate::thread::local_impl::EagerStorage::new(__INIT); VAL.get() } } else { @@ -81,21 +85,19 @@ pub macro thread_local_inner { } unsafe { - use $crate::mem::needs_drop; - use $crate::thread::LocalKey; - use $crate::thread::local_impl::LazyStorage; - - LocalKey::new(const { - if needs_drop::<$t>() { + $crate::thread::LocalKey::new(const { + if $crate::mem::needs_drop::<$t>() { |init| { #[thread_local] - static VAL: LazyStorage<$t, ()> = LazyStorage::new(); + static VAL: $crate::thread::local_impl::LazyStorage<$t, ()> + = $crate::thread::local_impl::LazyStorage::new(); VAL.get_or_init(init, __init) } } else { |init| { #[thread_local] - static VAL: LazyStorage<$t, !> = LazyStorage::new(); + static VAL: $crate::thread::local_impl::LazyStorage<$t, !> + = $crate::thread::local_impl::LazyStorage::new(); VAL.get_or_init(init, __init) } } @@ -107,3 +109,31 @@ pub macro thread_local_inner { $crate::thread::local_impl::thread_local_inner!(@key $t, $($init)*); }, } + +#[rustc_macro_transparency = "semitransparent"] +pub(crate) macro local_pointer { + () => {}, + ($vis:vis static $name:ident; $($rest:tt)*) => { + #[thread_local] + $vis static $name: $crate::sys::thread_local::LocalPointer = $crate::sys::thread_local::LocalPointer::__new(); + $crate::sys::thread_local::local_pointer! { $($rest)* } + }, +} + +pub(crate) struct LocalPointer { + p: Cell<*mut ()>, +} + +impl LocalPointer { + pub const fn __new() -> LocalPointer { + LocalPointer { p: Cell::new(ptr::null_mut()) } + } + + pub fn get(&self) -> *mut () { + self.p.get() + } + + pub fn set(&self, p: *mut ()) { + self.p.set(p) + } +} diff --git a/library/std/src/sys/thread_local/os.rs b/library/std/src/sys/thread_local/os.rs index e27b47c3f4536..58f291ffdb985 100644 --- a/library/std/src/sys/thread_local/os.rs +++ b/library/std/src/sys/thread_local/os.rs @@ -1,8 +1,8 @@ -use super::abort_on_dtor_unwind; +use super::key::{Key, LazyKey, get, set}; +use super::{abort_on_dtor_unwind, guard}; use crate::cell::Cell; use crate::marker::PhantomData; use crate::ptr; -use crate::sys::thread_local::key::{get, set, Key, LazyKey}; #[doc(hidden)] #[allow_internal_unstable(thread_local_internals)] @@ -15,19 +15,24 @@ pub macro thread_local_inner { $crate::thread::local_impl::thread_local_inner!(@key $t, { const INIT_EXPR: $t = $init; INIT_EXPR }) }, - // used to generate the `LocalKey` value for `thread_local!` + // NOTE: we cannot import `Storage` or `LocalKey` with a `use` because that can shadow user + // provided type or type alias with a matching name. Please update the shadowing test in + // `tests/thread.rs` if these types are renamed. + + // used to generate the `LocalKey` value for `thread_local!`. (@key $t:ty, $init:expr) => {{ #[inline] fn __init() -> $t { $init } + // NOTE: this cannot import `LocalKey` or `Storage` with a `use` because that can shadow + // user provided type or type alias with a matching name. Please update the shadowing test + // in `tests/thread.rs` if these types are renamed. unsafe { - use $crate::thread::LocalKey; - use $crate::thread::local_impl::Storage; - // Inlining does not work on windows-gnu due to linking errors around // dllimports. See https://github.com/rust-lang/rust/issues/109797. - LocalKey::new(#[cfg_attr(windows, inline(never))] |init| { - static VAL: Storage<$t> = Storage::new(); + $crate::thread::LocalKey::new(#[cfg_attr(windows, inline(never))] |init| { + static VAL: $crate::thread::local_impl::Storage<$t> + = $crate::thread::local_impl::Storage::new(); VAL.get(init, __init) }) } @@ -55,7 +60,7 @@ struct Value { } impl Storage { - #[rustc_const_unstable(feature = "thread_local_internals", issue = "none")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "thread_local_internals", issue = "none"))] pub const fn new() -> Storage { Storage { key: LazyKey::new(Some(destroy_value::)), marker: PhantomData } } @@ -138,5 +143,35 @@ unsafe extern "C" fn destroy_value(ptr: *mut u8) { drop(ptr); // SAFETY: `key` is the TLS key `ptr` was stored under. unsafe { set(key, ptr::null_mut()) }; + // Make sure that the runtime cleanup will be performed + // after the next round of TLS destruction. + guard::enable(); }); } + +#[rustc_macro_transparency = "semitransparent"] +pub(crate) macro local_pointer { + () => {}, + ($vis:vis static $name:ident; $($rest:tt)*) => { + $vis static $name: $crate::sys::thread_local::LocalPointer = $crate::sys::thread_local::LocalPointer::__new(); + $crate::sys::thread_local::local_pointer! { $($rest)* } + }, +} + +pub(crate) struct LocalPointer { + key: LazyKey, +} + +impl LocalPointer { + pub const fn __new() -> LocalPointer { + LocalPointer { key: LazyKey::new(None) } + } + + pub fn get(&'static self) -> *mut () { + unsafe { get(self.key.force()) as *mut () } + } + + pub fn set(&'static self, p: *mut ()) { + unsafe { set(self.key.force(), p as *mut u8) } + } +} diff --git a/library/std/src/sys/thread_local/statik.rs b/library/std/src/sys/thread_local/statik.rs index a3451ab74e04f..4da01a84acf68 100644 --- a/library/std/src/sys/thread_local/statik.rs +++ b/library/std/src/sys/thread_local/statik.rs @@ -1,7 +1,8 @@ //! On some targets like wasm there's no threads, so no need to generate //! thread locals and we can instead just use plain statics! -use crate::cell::UnsafeCell; +use crate::cell::{Cell, UnsafeCell}; +use crate::ptr; #[doc(hidden)] #[allow_internal_unstable(thread_local_internals)] @@ -13,12 +14,11 @@ pub macro thread_local_inner { (@key $t:ty, const $init:expr) => {{ const __INIT: $t = $init; + // NOTE: Please update the shadowing test in `tests/thread.rs` if these types are renamed. unsafe { - use $crate::thread::LocalKey; - use $crate::thread::local_impl::EagerStorage; - - LocalKey::new(|_| { - static VAL: EagerStorage<$t> = EagerStorage { value: __INIT }; + $crate::thread::LocalKey::new(|_| { + static VAL: $crate::thread::local_impl::EagerStorage<$t> = + $crate::thread::local_impl::EagerStorage { value: __INIT }; &VAL.value }) } @@ -93,3 +93,33 @@ impl LazyStorage { // SAFETY: the target doesn't have threads. unsafe impl Sync for LazyStorage {} + +#[rustc_macro_transparency = "semitransparent"] +pub(crate) macro local_pointer { + () => {}, + ($vis:vis static $name:ident; $($rest:tt)*) => { + $vis static $name: $crate::sys::thread_local::LocalPointer = $crate::sys::thread_local::LocalPointer::__new(); + $crate::sys::thread_local::local_pointer! { $($rest)* } + }, +} + +pub(crate) struct LocalPointer { + p: Cell<*mut ()>, +} + +impl LocalPointer { + pub const fn __new() -> LocalPointer { + LocalPointer { p: Cell::new(ptr::null_mut()) } + } + + pub fn get(&self) -> *mut () { + self.p.get() + } + + pub fn set(&self, p: *mut ()) { + self.p.set(p) + } +} + +// SAFETY: the target doesn't have threads. +unsafe impl Sync for LocalPointer {} diff --git a/library/std/src/sys_common/io.rs b/library/std/src/sys_common/io.rs index e386c955f3767..6f6f282d432d6 100644 --- a/library/std/src/sys_common/io.rs +++ b/library/std/src/sys_common/io.rs @@ -3,7 +3,7 @@ pub const DEFAULT_BUF_SIZE: usize = if cfg!(target_os = "espidf") { 512 } else { 8 * 1024 }; #[cfg(test)] -#[allow(dead_code)] // not used on emscripten +#[allow(dead_code)] // not used on emscripten and wasi pub mod test { use rand::RngCore; diff --git a/library/std/src/sys_common/lazy_box.rs b/library/std/src/sys_common/lazy_box.rs deleted file mode 100644 index b45b05f63baaa..0000000000000 --- a/library/std/src/sys_common/lazy_box.rs +++ /dev/null @@ -1,88 +0,0 @@ -#![allow(dead_code)] // Only used on some platforms. - -// This is used to wrap pthread {Mutex, Condvar, RwLock} in. - -use crate::marker::PhantomData; -use crate::ops::{Deref, DerefMut}; -use crate::ptr::null_mut; -use crate::sync::atomic::AtomicPtr; -use crate::sync::atomic::Ordering::{AcqRel, Acquire}; - -pub(crate) struct LazyBox { - ptr: AtomicPtr, - _phantom: PhantomData, -} - -pub(crate) trait LazyInit { - /// This is called before the box is allocated, to provide the value to - /// move into the new box. - /// - /// It might be called more than once per LazyBox, as multiple threads - /// might race to initialize it concurrently, each constructing and initializing - /// their own box. All but one of them will be passed to `cancel_init` right after. - fn init() -> Box; - - /// Any surplus boxes from `init()` that lost the initialization race - /// are passed to this function for disposal. - /// - /// The default implementation calls destroy(). - fn cancel_init(x: Box) { - Self::destroy(x); - } - - /// This is called to destroy a used box. - /// - /// The default implementation just drops it. - fn destroy(_: Box) {} -} - -impl LazyBox { - #[inline] - pub const fn new() -> Self { - Self { ptr: AtomicPtr::new(null_mut()), _phantom: PhantomData } - } - - #[inline] - fn get_pointer(&self) -> *mut T { - let ptr = self.ptr.load(Acquire); - if ptr.is_null() { self.initialize() } else { ptr } - } - - #[cold] - fn initialize(&self) -> *mut T { - let new_ptr = Box::into_raw(T::init()); - match self.ptr.compare_exchange(null_mut(), new_ptr, AcqRel, Acquire) { - Ok(_) => new_ptr, - Err(ptr) => { - // Lost the race to another thread. - // Drop the box we created, and use the one from the other thread instead. - T::cancel_init(unsafe { Box::from_raw(new_ptr) }); - ptr - } - } - } -} - -impl Deref for LazyBox { - type Target = T; - #[inline] - fn deref(&self) -> &T { - unsafe { &*self.get_pointer() } - } -} - -impl DerefMut for LazyBox { - #[inline] - fn deref_mut(&mut self) -> &mut T { - unsafe { &mut *self.get_pointer() } - } -} - -impl Drop for LazyBox { - fn drop(&mut self) { - let ptr = *self.ptr.get_mut(); - if !ptr.is_null() { - T::destroy(unsafe { Box::from_raw(ptr) }); - } - } -} diff --git a/library/std/src/sys_common/mod.rs b/library/std/src/sys_common/mod.rs index 1c884f107beeb..4f7a131f6bb90 100644 --- a/library/std/src/sys_common/mod.rs +++ b/library/std/src/sys_common/mod.rs @@ -22,7 +22,6 @@ mod tests; pub mod fs; pub mod io; -pub mod lazy_box; pub mod process; pub mod wstr; pub mod wtf8; @@ -32,7 +31,8 @@ cfg_if::cfg_if! { all(unix, not(target_os = "l4re")), windows, target_os = "hermit", - target_os = "solid_asp3" + target_os = "solid_asp3", + all(target_os = "wasi", target_env = "p2") ))] { pub mod net; } else { diff --git a/library/std/src/sys_common/net.rs b/library/std/src/sys_common/net.rs index 25ebeb3502d20..5a0ad90758101 100644 --- a/library/std/src/sys_common/net.rs +++ b/library/std/src/sys_common/net.rs @@ -5,7 +5,7 @@ use crate::ffi::{c_int, c_void}; use crate::io::{self, BorrowedCursor, ErrorKind, IoSlice, IoSliceMut}; use crate::net::{Ipv4Addr, Ipv6Addr, Shutdown, SocketAddr}; use crate::sys::common::small_c_string::run_with_cstr; -use crate::sys::net::{cvt, cvt_gai, cvt_r, init, netc as c, wrlen_t, Socket}; +use crate::sys::net::{Socket, cvt, cvt_gai, cvt_r, init, netc as c, wrlen_t}; use crate::sys_common::{AsInner, FromInner, IntoInner}; use crate::time::Duration; use crate::{cmp, fmt, mem, ptr}; @@ -21,6 +21,7 @@ cfg_if::cfg_if! { target_os = "haiku", target_os = "l4re", target_os = "nto", + target_os = "nuttx", target_vendor = "apple", ))] { use crate::sys::net::netc::IPV6_JOIN_GROUP as IPV6_ADD_MEMBERSHIP; @@ -73,7 +74,7 @@ pub fn setsockopt( sock.as_raw(), level, option_name, - core::ptr::addr_of!(option_value) as *const _, + (&raw const option_value) as *const _, mem::size_of::() as c::socklen_t, ))?; Ok(()) @@ -88,7 +89,7 @@ pub fn getsockopt(sock: &Socket, level: c_int, option_name: c_int) -> i sock.as_raw(), level, option_name, - core::ptr::addr_of_mut!(option_value) as *mut _, + (&raw mut option_value) as *mut _, &mut option_len, ))?; Ok(option_value) @@ -102,7 +103,7 @@ where unsafe { let mut storage: c::sockaddr_storage = mem::zeroed(); let mut len = mem::size_of_val(&storage) as c::socklen_t; - cvt(f(core::ptr::addr_of_mut!(storage) as *mut _, &mut len))?; + cvt(f((&raw mut storage) as *mut _, &mut len))?; sockaddr_to_addr(&storage, len as usize) } } @@ -451,7 +452,7 @@ impl TcpListener { pub fn accept(&self) -> io::Result<(TcpStream, SocketAddr)> { let mut storage: c::sockaddr_storage = unsafe { mem::zeroed() }; let mut len = mem::size_of_val(&storage) as c::socklen_t; - let sock = self.inner.accept(core::ptr::addr_of_mut!(storage) as *mut _, &mut len)?; + let sock = self.inner.accept((&raw mut storage) as *mut _, &mut len)?; let addr = sockaddr_to_addr(&storage, len as usize)?; Ok((TcpStream { inner: sock }, addr)) } diff --git a/library/std/src/sys_common/wtf8.rs b/library/std/src/sys_common/wtf8.rs index 063451ad54e1c..19d4c94f45099 100644 --- a/library/std/src/sys_common/wtf8.rs +++ b/library/std/src/sys_common/wtf8.rs @@ -18,7 +18,7 @@ #[cfg(test)] mod tests; -use core::char::{encode_utf16_raw, encode_utf8_raw}; +use core::char::{encode_utf8_raw, encode_utf16_raw}; use core::clone::CloneToUninit; use core::str::next_code_point; @@ -26,7 +26,6 @@ use crate::borrow::Cow; use crate::collections::TryReserveError; use crate::hash::{Hash, Hasher}; use crate::iter::FusedIterator; -use crate::ptr::addr_of_mut; use crate::rc::Rc; use crate::sync::Arc; use crate::sys_common::AsInner; @@ -1055,6 +1054,6 @@ unsafe impl CloneToUninit for Wtf8 { #[cfg_attr(debug_assertions, track_caller)] unsafe fn clone_to_uninit(&self, dst: *mut Self) { // SAFETY: we're just a wrapper around [u8] - unsafe { self.bytes.clone_to_uninit(addr_of_mut!((*dst).bytes)) } + unsafe { self.bytes.clone_to_uninit(&raw mut (*dst).bytes) } } } diff --git a/library/std/src/sys_common/wtf8/tests.rs b/library/std/src/sys_common/wtf8/tests.rs index b57c99a8452a1..bc06eaa2b8fa1 100644 --- a/library/std/src/sys_common/wtf8/tests.rs +++ b/library/std/src/sys_common/wtf8/tests.rs @@ -356,32 +356,32 @@ fn wtf8buf_from_iterator() { fn f(values: &[u32]) -> Wtf8Buf { values.iter().map(|&c| CodePoint::from_u32(c).unwrap()).collect::() } - assert_eq!( - f(&[0x61, 0xE9, 0x20, 0x1F4A9]), - Wtf8Buf { bytes: b"a\xC3\xA9 \xF0\x9F\x92\xA9".to_vec(), is_known_utf8: true } - ); + assert_eq!(f(&[0x61, 0xE9, 0x20, 0x1F4A9]), Wtf8Buf { + bytes: b"a\xC3\xA9 \xF0\x9F\x92\xA9".to_vec(), + is_known_utf8: true + }); assert_eq!(f(&[0xD83D, 0xDCA9]).bytes, b"\xF0\x9F\x92\xA9"); // Magic! - assert_eq!( - f(&[0xD83D, 0x20, 0xDCA9]), - Wtf8Buf { bytes: b"\xED\xA0\xBD \xED\xB2\xA9".to_vec(), is_known_utf8: false } - ); - assert_eq!( - f(&[0xD800, 0xDBFF]), - Wtf8Buf { bytes: b"\xED\xA0\x80\xED\xAF\xBF".to_vec(), is_known_utf8: false } - ); - assert_eq!( - f(&[0xD800, 0xE000]), - Wtf8Buf { bytes: b"\xED\xA0\x80\xEE\x80\x80".to_vec(), is_known_utf8: false } - ); - assert_eq!( - f(&[0xD7FF, 0xDC00]), - Wtf8Buf { bytes: b"\xED\x9F\xBF\xED\xB0\x80".to_vec(), is_known_utf8: false } - ); - assert_eq!( - f(&[0x61, 0xDC00]), - Wtf8Buf { bytes: b"\x61\xED\xB0\x80".to_vec(), is_known_utf8: false } - ); + assert_eq!(f(&[0xD83D, 0x20, 0xDCA9]), Wtf8Buf { + bytes: b"\xED\xA0\xBD \xED\xB2\xA9".to_vec(), + is_known_utf8: false + }); + assert_eq!(f(&[0xD800, 0xDBFF]), Wtf8Buf { + bytes: b"\xED\xA0\x80\xED\xAF\xBF".to_vec(), + is_known_utf8: false + }); + assert_eq!(f(&[0xD800, 0xE000]), Wtf8Buf { + bytes: b"\xED\xA0\x80\xEE\x80\x80".to_vec(), + is_known_utf8: false + }); + assert_eq!(f(&[0xD7FF, 0xDC00]), Wtf8Buf { + bytes: b"\xED\x9F\xBF\xED\xB0\x80".to_vec(), + is_known_utf8: false + }); + assert_eq!(f(&[0x61, 0xDC00]), Wtf8Buf { + bytes: b"\x61\xED\xB0\x80".to_vec(), + is_known_utf8: false + }); assert_eq!(f(&[0xDC00]), Wtf8Buf { bytes: b"\xED\xB0\x80".to_vec(), is_known_utf8: false }); } @@ -396,36 +396,36 @@ fn wtf8buf_extend() { string } - assert_eq!( - e(&[0x61, 0xE9], &[0x20, 0x1F4A9]), - Wtf8Buf { bytes: b"a\xC3\xA9 \xF0\x9F\x92\xA9".to_vec(), is_known_utf8: true } - ); + assert_eq!(e(&[0x61, 0xE9], &[0x20, 0x1F4A9]), Wtf8Buf { + bytes: b"a\xC3\xA9 \xF0\x9F\x92\xA9".to_vec(), + is_known_utf8: true + }); assert_eq!(e(&[0xD83D], &[0xDCA9]).bytes, b"\xF0\x9F\x92\xA9"); // Magic! - assert_eq!( - e(&[0xD83D, 0x20], &[0xDCA9]), - Wtf8Buf { bytes: b"\xED\xA0\xBD \xED\xB2\xA9".to_vec(), is_known_utf8: false } - ); - assert_eq!( - e(&[0xD800], &[0xDBFF]), - Wtf8Buf { bytes: b"\xED\xA0\x80\xED\xAF\xBF".to_vec(), is_known_utf8: false } - ); - assert_eq!( - e(&[0xD800], &[0xE000]), - Wtf8Buf { bytes: b"\xED\xA0\x80\xEE\x80\x80".to_vec(), is_known_utf8: false } - ); - assert_eq!( - e(&[0xD7FF], &[0xDC00]), - Wtf8Buf { bytes: b"\xED\x9F\xBF\xED\xB0\x80".to_vec(), is_known_utf8: false } - ); - assert_eq!( - e(&[0x61], &[0xDC00]), - Wtf8Buf { bytes: b"\x61\xED\xB0\x80".to_vec(), is_known_utf8: false } - ); - assert_eq!( - e(&[], &[0xDC00]), - Wtf8Buf { bytes: b"\xED\xB0\x80".to_vec(), is_known_utf8: false } - ); + assert_eq!(e(&[0xD83D, 0x20], &[0xDCA9]), Wtf8Buf { + bytes: b"\xED\xA0\xBD \xED\xB2\xA9".to_vec(), + is_known_utf8: false + }); + assert_eq!(e(&[0xD800], &[0xDBFF]), Wtf8Buf { + bytes: b"\xED\xA0\x80\xED\xAF\xBF".to_vec(), + is_known_utf8: false + }); + assert_eq!(e(&[0xD800], &[0xE000]), Wtf8Buf { + bytes: b"\xED\xA0\x80\xEE\x80\x80".to_vec(), + is_known_utf8: false + }); + assert_eq!(e(&[0xD7FF], &[0xDC00]), Wtf8Buf { + bytes: b"\xED\x9F\xBF\xED\xB0\x80".to_vec(), + is_known_utf8: false + }); + assert_eq!(e(&[0x61], &[0xDC00]), Wtf8Buf { + bytes: b"\x61\xED\xB0\x80".to_vec(), + is_known_utf8: false + }); + assert_eq!(e(&[], &[0xDC00]), Wtf8Buf { + bytes: b"\xED\xB0\x80".to_vec(), + is_known_utf8: false + }); } #[test] @@ -556,10 +556,9 @@ fn wtf8_encode_wide() { let mut string = Wtf8Buf::from_str("aé "); string.push(CodePoint::from_u32(0xD83D).unwrap()); string.push_char('💩'); - assert_eq!( - string.encode_wide().collect::>(), - vec![0x61, 0xE9, 0x20, 0xD83D, 0xD83D, 0xDCA9] - ); + assert_eq!(string.encode_wide().collect::>(), vec![ + 0x61, 0xE9, 0x20, 0xD83D, 0xD83D, 0xDCA9 + ]); } #[test] diff --git a/library/std/src/thread/current.rs b/library/std/src/thread/current.rs new file mode 100644 index 0000000000000..e6eb90c4c30a5 --- /dev/null +++ b/library/std/src/thread/current.rs @@ -0,0 +1,254 @@ +use super::{Thread, ThreadId}; +use crate::mem::ManuallyDrop; +use crate::ptr; +use crate::sys::thread_local::local_pointer; + +const NONE: *mut () = ptr::null_mut(); +const BUSY: *mut () = ptr::without_provenance_mut(1); +const DESTROYED: *mut () = ptr::without_provenance_mut(2); + +local_pointer! { + static CURRENT; +} + +/// Persistent storage for the thread ID. +/// +/// We store the thread ID so that it never gets destroyed during the lifetime +/// of a thread, either using `#[thread_local]` or multiple `local_pointer!`s. +mod id { + use super::*; + + cfg_if::cfg_if! { + if #[cfg(target_thread_local)] { + use crate::cell::Cell; + + #[thread_local] + static ID: Cell> = Cell::new(None); + + pub(super) const CHEAP: bool = true; + + pub(super) fn get() -> Option { + ID.get() + } + + pub(super) fn set(id: ThreadId) { + ID.set(Some(id)) + } + } else if #[cfg(target_pointer_width = "16")] { + local_pointer! { + static ID0; + static ID16; + static ID32; + static ID48; + } + + pub(super) const CHEAP: bool = false; + + pub(super) fn get() -> Option { + let id0 = ID0.get().addr() as u64; + let id16 = ID16.get().addr() as u64; + let id32 = ID32.get().addr() as u64; + let id48 = ID48.get().addr() as u64; + ThreadId::from_u64((id48 << 48) + (id32 << 32) + (id16 << 16) + id0) + } + + pub(super) fn set(id: ThreadId) { + let val = id.as_u64().get(); + ID0.set(ptr::without_provenance_mut(val as usize)); + ID16.set(ptr::without_provenance_mut((val >> 16) as usize)); + ID32.set(ptr::without_provenance_mut((val >> 32) as usize)); + ID48.set(ptr::without_provenance_mut((val >> 48) as usize)); + } + } else if #[cfg(target_pointer_width = "32")] { + local_pointer! { + static ID0; + static ID32; + } + + pub(super) const CHEAP: bool = false; + + pub(super) fn get() -> Option { + let id0 = ID0.get().addr() as u64; + let id32 = ID32.get().addr() as u64; + ThreadId::from_u64((id32 << 32) + id0) + } + + pub(super) fn set(id: ThreadId) { + let val = id.as_u64().get(); + ID0.set(ptr::without_provenance_mut(val as usize)); + ID32.set(ptr::without_provenance_mut((val >> 32) as usize)); + } + } else { + local_pointer! { + static ID; + } + + pub(super) const CHEAP: bool = true; + + pub(super) fn get() -> Option { + let id = ID.get().addr() as u64; + ThreadId::from_u64(id) + } + + pub(super) fn set(id: ThreadId) { + let val = id.as_u64().get(); + ID.set(ptr::without_provenance_mut(val as usize)); + } + } + } + + #[inline] + pub(super) fn get_or_init() -> ThreadId { + get().unwrap_or_else( + #[cold] + || { + let id = ThreadId::new(); + id::set(id); + id + }, + ) + } +} + +/// Tries to set the thread handle for the current thread. Fails if a handle was +/// already set or if the thread ID of `thread` would change an already-set ID. +pub(crate) fn set_current(thread: Thread) -> Result<(), Thread> { + if CURRENT.get() != NONE { + return Err(thread); + } + + match id::get() { + Some(id) if id == thread.id() => {} + None => id::set(thread.id()), + _ => return Err(thread), + } + + // Make sure that `crate::rt::thread_cleanup` will be run, which will + // call `drop_current`. + crate::sys::thread_local::guard::enable(); + CURRENT.set(thread.into_raw().cast_mut()); + Ok(()) +} + +/// Gets the id of the thread that invokes it. +/// +/// This function will always succeed, will always return the same value for +/// one thread and is guaranteed not to call the global allocator. +#[inline] +pub(crate) fn current_id() -> ThreadId { + // If accessing the persistant thread ID takes multiple TLS accesses, try + // to retrieve it from the current thread handle, which will only take one + // TLS access. + if !id::CHEAP { + let current = CURRENT.get(); + if current > DESTROYED { + unsafe { + let current = ManuallyDrop::new(Thread::from_raw(current)); + return current.id(); + } + } + } + + id::get_or_init() +} + +/// Gets a handle to the thread that invokes it, if the handle has been initialized. +pub(crate) fn try_current() -> Option { + let current = CURRENT.get(); + if current > DESTROYED { + unsafe { + let current = ManuallyDrop::new(Thread::from_raw(current)); + Some((*current).clone()) + } + } else { + None + } +} + +/// Gets a handle to the thread that invokes it. +/// +/// # Examples +/// +/// Getting a handle to the current thread with `thread::current()`: +/// +/// ``` +/// use std::thread; +/// +/// let handler = thread::Builder::new() +/// .name("named thread".into()) +/// .spawn(|| { +/// let handle = thread::current(); +/// assert_eq!(handle.name(), Some("named thread")); +/// }) +/// .unwrap(); +/// +/// handler.join().unwrap(); +/// ``` +#[must_use] +#[stable(feature = "rust1", since = "1.0.0")] +pub fn current() -> Thread { + let current = CURRENT.get(); + if current > DESTROYED { + unsafe { + let current = ManuallyDrop::new(Thread::from_raw(current)); + (*current).clone() + } + } else { + init_current(current) + } +} + +#[cold] +fn init_current(current: *mut ()) -> Thread { + if current == NONE { + CURRENT.set(BUSY); + // If the thread ID was initialized already, use it. + let id = id::get_or_init(); + let thread = Thread::new_unnamed(id); + + // Make sure that `crate::rt::thread_cleanup` will be run, which will + // call `drop_current`. + crate::sys::thread_local::guard::enable(); + CURRENT.set(thread.clone().into_raw().cast_mut()); + thread + } else if current == BUSY { + // BUSY exists solely for this check, but as it is in the slow path, the + // extra TLS write above shouldn't matter. The alternative is nearly always + // a stack overflow. + + // If you came across this message, contact the author of your allocator. + // If you are said author: A surprising amount of functions inside the + // standard library (e.g. `Mutex`, `thread_local!`, `File` when using long + // paths, even `panic!` when using unwinding), need memory allocation, so + // you'll get circular dependencies all over the place when using them. + // I (joboet) highly recommend using only APIs from core in your allocator + // and implementing your own system abstractions. Still, if you feel that + // a particular API should be entirely allocation-free, feel free to open + // an issue on the Rust repository, we'll see what we can do. + rtabort!( + "\n + Attempted to access thread-local data while allocating said data.\n + Do not access functions that allocate in the global allocator!\n + This is a bug in the global allocator.\n + " + ) + } else { + debug_assert_eq!(current, DESTROYED); + panic!( + "use of std::thread::current() is not possible after the thread's + local data has been destroyed" + ) + } +} + +/// This should be run in [`crate::rt::thread_cleanup`] to reset the thread +/// handle. +pub(crate) fn drop_current() { + let current = CURRENT.get(); + if current > DESTROYED { + unsafe { + CURRENT.set(DESTROYED); + drop(Thread::from_raw(current)); + } + } +} diff --git a/library/std/src/thread/local.rs b/library/std/src/thread/local.rs index f147c5fdcd146..9edb3fa41933d 100644 --- a/library/std/src/thread/local.rs +++ b/library/std/src/thread/local.rs @@ -2,7 +2,7 @@ #![unstable(feature = "thread_local_internals", issue = "none")] -#[cfg(all(test, not(target_os = "emscripten")))] +#[cfg(all(test, not(any(target_os = "emscripten", target_os = "wasi"))))] mod tests; #[cfg(test)] @@ -237,7 +237,7 @@ impl LocalKey { reason = "recently added to create a key", issue = "none" )] - #[rustc_const_unstable(feature = "thread_local_internals", issue = "none")] + #[cfg_attr(bootstrap, rustc_const_unstable(feature = "thread_local_internals", issue = "none"))] pub const unsafe fn new(inner: fn(Option<&mut Option>) -> *const T) -> LocalKey { LocalKey { inner } } diff --git a/library/std/src/thread/local/tests.rs b/library/std/src/thread/local/tests.rs index 25019b554bb6a..9d4f52a09218e 100644 --- a/library/std/src/thread/local/tests.rs +++ b/library/std/src/thread/local/tests.rs @@ -1,7 +1,7 @@ use crate::cell::{Cell, UnsafeCell}; use crate::sync::atomic::{AtomicU8, Ordering}; use crate::sync::{Arc, Condvar, Mutex}; -use crate::thread::{self, LocalKey}; +use crate::thread::{self, Builder, LocalKey}; use crate::thread_local; #[derive(Clone, Default)] @@ -103,6 +103,9 @@ fn smoke_dtor() { #[test] fn circular() { + // FIXME(static_mut_refs): Do not allow `static_mut_refs` lint + #![allow(static_mut_refs)] + struct S1(&'static LocalKey>>, &'static LocalKey>>); struct S2(&'static LocalKey>>, &'static LocalKey>>); thread_local!(static K1: UnsafeCell> = UnsafeCell::new(None)); @@ -340,3 +343,34 @@ fn join_orders_after_tls_destructors() { jh2.join().unwrap(); } } + +// Test that thread::current is still available in TLS destructors. +#[test] +fn thread_current_in_dtor() { + // Go through one round of TLS destruction first. + struct Defer; + impl Drop for Defer { + fn drop(&mut self) { + RETRIEVE.with(|_| {}); + } + } + + struct RetrieveName; + impl Drop for RetrieveName { + fn drop(&mut self) { + *NAME.lock().unwrap() = Some(thread::current().name().unwrap().to_owned()); + } + } + + static NAME: Mutex> = Mutex::new(None); + + thread_local! { + static DEFER: Defer = const { Defer }; + static RETRIEVE: RetrieveName = const { RetrieveName }; + } + + Builder::new().name("test".to_owned()).spawn(|| DEFER.with(|_| {})).unwrap().join().unwrap(); + let name = NAME.lock().unwrap(); + let name = name.as_ref().unwrap(); + assert_eq!(name, "test"); +} diff --git a/library/std/src/thread/mod.rs b/library/std/src/thread/mod.rs index 0fc63c5081b03..227ee9d64f375 100644 --- a/library/std/src/thread/mod.rs +++ b/library/std/src/thread/mod.rs @@ -141,7 +141,7 @@ //! [`Result`]: crate::result::Result //! [`Ok`]: crate::result::Result::Ok //! [`Err`]: crate::result::Result::Err -//! [`thread::current`]: current +//! [`thread::current`]: current::current //! [`thread::Result`]: Result //! [`unpark`]: Thread::unpark //! [`thread::park_timeout`]: park_timeout @@ -155,19 +155,21 @@ // Under `test`, `__FastLocalKeyInner` seems unused. #![cfg_attr(test, allow(dead_code))] -#[cfg(all(test, not(target_os = "emscripten")))] +#[cfg(all(test, not(any(target_os = "emscripten", target_os = "wasi"))))] mod tests; +use core::cell::SyncUnsafeCell; +use core::ffi::CStr; +use core::mem::MaybeUninit; + use crate::any::Any; -use crate::cell::{Cell, OnceCell, UnsafeCell}; -use crate::ffi::CStr; +use crate::cell::UnsafeCell; use crate::marker::PhantomData; -use crate::mem::{self, forget, ManuallyDrop}; +use crate::mem::{self, ManuallyDrop, forget}; use crate::num::NonZero; use crate::pin::Pin; -use crate::ptr::addr_of_mut; -use crate::sync::atomic::{AtomicUsize, Ordering}; use crate::sync::Arc; +use crate::sync::atomic::{AtomicUsize, Ordering}; use crate::sys::sync::Parker; use crate::sys::thread as imp; use crate::sys_common::{AsInner, IntoInner}; @@ -178,7 +180,13 @@ use crate::{env, fmt, io, panic, panicking, str}; mod scoped; #[stable(feature = "scoped_threads", since = "1.63.0")] -pub use scoped::{scope, Scope, ScopedJoinHandle}; +pub use scoped::{Scope, ScopedJoinHandle, scope}; + +mod current; + +#[stable(feature = "rust1", since = "1.0.0")] +pub use current::current; +pub(crate) use current::{current_id, drop_current, set_current, try_current}; //////////////////////////////////////////////////////////////////////////////// // Thread-local storage @@ -472,7 +480,11 @@ impl Builder { amt }); - let my_thread = name.map_or_else(Thread::new_unnamed, Thread::new); + let id = ThreadId::new(); + let my_thread = match name { + Some(name) => Thread::new(id, name.into()), + None => Thread::new_unnamed(id), + }; let their_thread = my_thread.clone(); let my_packet: Arc> = Arc::new(Packet { @@ -510,6 +522,14 @@ impl Builder { let f = MaybeDangling::new(f); let main = move || { + if let Err(_thread) = set_current(their_thread.clone()) { + // Both the current thread handle and the ID should not be + // initialized yet. Since only the C runtime and some of our + // platform code run before this, this point shouldn't be + // reachable. Use an abort to save binary size (see #123356). + rtabort!("something here is badly broken!"); + } + if let Some(name) = their_thread.cname() { imp::Thread::set_name(name); } @@ -517,7 +537,6 @@ impl Builder { crate::io::set_output_capture(output_capture); let f = f.into_inner(); - set_current(their_thread); let try_result = panic::catch_unwind(panic::AssertUnwindSafe(|| { crate::sys::backtrace::__rust_begin_short_backtrace(f) })); @@ -665,6 +684,19 @@ impl Builder { /// println!("{result}"); /// ``` /// +/// # Notes +/// +/// This function has the same minimal guarantee regarding "foreign" unwinding operations (e.g. +/// an exception thrown from C++ code, or a `panic!` in Rust code compiled or linked with a +/// different runtime) as [`catch_unwind`]; namely, if the thread created with `thread::spawn` +/// unwinds all the way to the root with such an exception, one of two behaviors are possible, +/// and it is unspecified which will occur: +/// +/// * The process aborts. +/// * The process does not abort, and [`join`] will return a `Result::Err` +/// containing an opaque type. +/// +/// [`catch_unwind`]: ../../std/panic/fn.catch_unwind.html /// [`channels`]: crate::sync::mpsc /// [`join`]: JoinHandle::join /// [`Err`]: crate::result::Result::Err @@ -678,84 +710,6 @@ where Builder::new().spawn(f).expect("failed to spawn thread") } -thread_local! { - // Invariant: `CURRENT` and `CURRENT_ID` will always be initialized together. - // If `CURRENT` is initialized, then `CURRENT_ID` will hold the same value - // as `CURRENT.id()`. - static CURRENT: OnceCell = const { OnceCell::new() }; - static CURRENT_ID: Cell> = const { Cell::new(None) }; -} - -/// Sets the thread handle for the current thread. -/// -/// Aborts if the handle has been set already to reduce code size. -pub(crate) fn set_current(thread: Thread) { - let tid = thread.id(); - // Using `unwrap` here can add ~3kB to the binary size. We have complete - // control over where this is called, so just abort if there is a bug. - CURRENT.with(|current| match current.set(thread) { - Ok(()) => CURRENT_ID.set(Some(tid)), - Err(_) => rtabort!("thread::set_current should only be called once per thread"), - }); -} - -/// Gets a handle to the thread that invokes it. -/// -/// In contrast to the public `current` function, this will not panic if called -/// from inside a TLS destructor. -pub(crate) fn try_current() -> Option { - CURRENT - .try_with(|current| { - current - .get_or_init(|| { - let thread = Thread::new_unnamed(); - CURRENT_ID.set(Some(thread.id())); - thread - }) - .clone() - }) - .ok() -} - -/// Gets the id of the thread that invokes it. -#[inline] -pub(crate) fn current_id() -> ThreadId { - CURRENT_ID.get().unwrap_or_else(|| { - // If `CURRENT_ID` isn't initialized yet, then `CURRENT` must also not be initialized. - // `current()` will initialize both `CURRENT` and `CURRENT_ID` so subsequent calls to - // `current_id()` will succeed immediately. - current().id() - }) -} - -/// Gets a handle to the thread that invokes it. -/// -/// # Examples -/// -/// Getting a handle to the current thread with `thread::current()`: -/// -/// ``` -/// use std::thread; -/// -/// let handler = thread::Builder::new() -/// .name("named thread".into()) -/// .spawn(|| { -/// let handle = thread::current(); -/// assert_eq!(handle.name(), Some("named thread")); -/// }) -/// .unwrap(); -/// -/// handler.join().unwrap(); -/// ``` -#[must_use] -#[stable(feature = "rust1", since = "1.0.0")] -pub fn current() -> Thread { - try_current().expect( - "use of std::thread::current() is not possible \ - after the thread's local data has been destroyed", - ) -} - /// Cooperatively gives up a timeslice to the OS scheduler. /// /// This calls the underlying OS scheduler's yield primitive, signaling @@ -924,7 +878,7 @@ pub fn sleep(dur: Duration) { /// /// # Platform-specific behavior /// -/// This function uses [`sleep`] internally, see its platform-specific behaviour. +/// This function uses [`sleep`] internally, see its platform-specific behavior. /// /// /// # Examples @@ -995,7 +949,7 @@ pub fn sleep_until(deadline: Instant) { } /// Used to ensure that `park` and `park_timeout` do not unwind, as that can -/// cause undefined behaviour if not handled correctly (see #102398 for context). +/// cause undefined behavior if not handled correctly (see #102398 for context). struct PanicGuard; impl Drop for PanicGuard { @@ -1174,7 +1128,7 @@ pub fn park_timeout(dur: Duration) { let guard = PanicGuard; // SAFETY: park_timeout is called on the parker owned by this thread. unsafe { - current().inner.as_ref().parker().park_timeout(dur); + current().0.parker().park_timeout(dur); } // No panic occurred, do not abort. forget(guard); @@ -1214,7 +1168,7 @@ pub struct ThreadId(NonZero); impl ThreadId { // Generate a new unique thread ID. - fn new() -> ThreadId { + pub(crate) fn new() -> ThreadId { #[cold] fn exhausted() -> ! { panic!("failed to generate unique thread ID: bitspace exhausted") @@ -1257,6 +1211,11 @@ impl ThreadId { } } + #[cfg(not(target_thread_local))] + fn from_u64(v: u64) -> Option { + NonZero::new(v).map(ThreadId) + } + /// This returns a numeric identifier for the thread identified by this /// `ThreadId`. /// @@ -1276,30 +1235,31 @@ impl ThreadId { // Thread //////////////////////////////////////////////////////////////////////////////// -/// The internal representation of a `Thread`'s name. -enum ThreadName { - Main, - Other(ThreadNameString), - Unnamed, -} - // This module ensures private fields are kept private, which is necessary to enforce the safety requirements. mod thread_name_string { use core::str; - use super::ThreadName; use crate::ffi::{CStr, CString}; /// Like a `String` it's guaranteed UTF-8 and like a `CString` it's null terminated. pub(crate) struct ThreadNameString { inner: CString, } + + impl ThreadNameString { + pub fn as_str(&self) -> &str { + // SAFETY: `self.inner` is only initialised via `String`, which upholds the validity invariant of `str`. + unsafe { str::from_utf8_unchecked(self.inner.to_bytes()) } + } + } + impl core::ops::Deref for ThreadNameString { type Target = CStr; fn deref(&self) -> &CStr { &self.inner } } + impl From for ThreadNameString { fn from(s: String) -> Self { Self { @@ -1307,34 +1267,82 @@ mod thread_name_string { } } } - impl ThreadName { - pub fn as_cstr(&self) -> Option<&CStr> { - match self { - ThreadName::Main => Some(c"main"), - ThreadName::Other(other) => Some(other), - ThreadName::Unnamed => None, - } - } - - pub fn as_str(&self) -> Option<&str> { - // SAFETY: `as_cstr` can only return `Some` for a fixed CStr or a `ThreadNameString`, - // which is guaranteed to be UTF-8. - self.as_cstr().map(|s| unsafe { str::from_utf8_unchecked(s.to_bytes()) }) - } - } } pub(crate) use thread_name_string::ThreadNameString; -/// The internal representation of a `Thread` handle -struct Inner { - name: ThreadName, // Guaranteed to be UTF-8 +static MAIN_THREAD_INFO: SyncUnsafeCell<(MaybeUninit, MaybeUninit)> = + SyncUnsafeCell::new((MaybeUninit::uninit(), MaybeUninit::uninit())); + +/// The internal representation of a `Thread` that is not the main thread. +struct OtherInner { + name: Option, id: ThreadId, parker: Parker, } +/// The internal representation of a `Thread` handle. +#[derive(Clone)] +enum Inner { + /// Represents the main thread. May only be constructed by Thread::new_main. + Main(&'static (ThreadId, Parker)), + /// Represents any other thread. + Other(Pin>), +} + impl Inner { - fn parker(self: Pin<&Self>) -> Pin<&Parker> { - unsafe { Pin::map_unchecked(self, |inner| &inner.parker) } + fn id(&self) -> ThreadId { + match self { + Self::Main((thread_id, _)) => *thread_id, + Self::Other(other) => other.id, + } + } + + fn cname(&self) -> Option<&CStr> { + match self { + Self::Main(_) => Some(c"main"), + Self::Other(other) => other.name.as_deref(), + } + } + + fn name(&self) -> Option<&str> { + match self { + Self::Main(_) => Some("main"), + Self::Other(other) => other.name.as_ref().map(ThreadNameString::as_str), + } + } + + fn into_raw(self) -> *const () { + match self { + // Just return the pointer to `MAIN_THREAD_INFO`. + Self::Main(ptr) => crate::ptr::from_ref(ptr).cast(), + Self::Other(arc) => { + // Safety: We only expose an opaque pointer, which maintains the `Pin` invariant. + let inner = unsafe { Pin::into_inner_unchecked(arc) }; + Arc::into_raw(inner) as *const () + } + } + } + + /// # Safety + /// + /// See [`Thread::from_raw`]. + unsafe fn from_raw(ptr: *const ()) -> Self { + // If the pointer is to `MAIN_THREAD_INFO`, we know it is the `Main` variant. + if crate::ptr::eq(ptr.cast(), &MAIN_THREAD_INFO) { + Self::Main(unsafe { &*ptr.cast() }) + } else { + // Safety: Upheld by caller + Self::Other(unsafe { Pin::new_unchecked(Arc::from_raw(ptr as *const OtherInner)) }) + } + } + + fn parker(&self) -> Pin<&Parker> { + match self { + Self::Main((_, parker_ref)) => Pin::static_ref(parker_ref), + Self::Other(inner) => unsafe { + Pin::map_unchecked(inner.as_ref(), |inner| &inner.parker) + }, + } } } @@ -1357,42 +1365,56 @@ impl Inner { /// should instead use a function like `spawn` to create new threads, see the /// docs of [`Builder`] and [`spawn`] for more details. /// -/// [`thread::current`]: current -pub struct Thread { - inner: Pin>, -} +/// [`thread::current`]: current::current +pub struct Thread(Inner); impl Thread { /// Used only internally to construct a thread object without spawning. - pub(crate) fn new(name: String) -> Thread { - Self::new_inner(ThreadName::Other(name.into())) + pub(crate) fn new(id: ThreadId, name: String) -> Thread { + Self::new_inner(id, Some(ThreadNameString::from(name))) } - pub(crate) fn new_unnamed() -> Thread { - Self::new_inner(ThreadName::Unnamed) + pub(crate) fn new_unnamed(id: ThreadId) -> Thread { + Self::new_inner(id, None) } - // Used in runtime to construct main thread - pub(crate) fn new_main() -> Thread { - Self::new_inner(ThreadName::Main) + /// Used in runtime to construct main thread + /// + /// # Safety + /// + /// This must only ever be called once, and must be called on the main thread. + pub(crate) unsafe fn new_main(thread_id: ThreadId) -> Thread { + // Safety: As this is only called once and on the main thread, nothing else is accessing MAIN_THREAD_INFO + // as the only other read occurs in `main_thread_info` *after* the main thread has been constructed, + // and this function is the only one that constructs the main thread. + // + // Pre-main thread spawning cannot hit this either, as the caller promises that this is only called on the main thread. + let main_thread_info = unsafe { &mut *MAIN_THREAD_INFO.get() }; + + unsafe { Parker::new_in_place((&raw mut main_thread_info.1).cast()) }; + main_thread_info.0.write(thread_id); + + // Store a `'static` ref to the initialised ThreadId and Parker, + // to avoid having to repeatedly prove initialisation. + Self(Inner::Main(unsafe { &*MAIN_THREAD_INFO.get().cast() })) } - fn new_inner(name: ThreadName) -> Thread { + fn new_inner(id: ThreadId, name: Option) -> Thread { // We have to use `unsafe` here to construct the `Parker` in-place, // which is required for the UNIX implementation. // // SAFETY: We pin the Arc immediately after creation, so its address never // changes. let inner = unsafe { - let mut arc = Arc::::new_uninit(); + let mut arc = Arc::::new_uninit(); let ptr = Arc::get_mut_unchecked(&mut arc).as_mut_ptr(); - addr_of_mut!((*ptr).name).write(name); - addr_of_mut!((*ptr).id).write(ThreadId::new()); - Parker::new_in_place(addr_of_mut!((*ptr).parker)); + (&raw mut (*ptr).name).write(name); + (&raw mut (*ptr).id).write(id); + Parker::new_in_place(&raw mut (*ptr).parker); Pin::new_unchecked(arc.assume_init()) }; - Thread { inner } + Self(Inner::Other(inner)) } /// Like the public [`park`], but callable on any handle. This is used to @@ -1401,7 +1423,7 @@ impl Thread { /// # Safety /// May only be called from the thread to which this handle belongs. pub(crate) unsafe fn park(&self) { - unsafe { self.inner.as_ref().parker().park() } + unsafe { self.0.parker().park() } } /// Atomically makes the handle's token available if it is not already. @@ -1437,7 +1459,7 @@ impl Thread { #[stable(feature = "rust1", since = "1.0.0")] #[inline] pub fn unpark(&self) { - self.inner.as_ref().parker().unpark(); + self.0.parker().unpark(); } /// Gets the thread's unique identifier. @@ -1457,7 +1479,7 @@ impl Thread { #[stable(feature = "thread_id", since = "1.19.0")] #[must_use] pub fn id(&self) -> ThreadId { - self.inner.id + self.0.id() } /// Gets the thread's name. @@ -1500,11 +1522,57 @@ impl Thread { #[stable(feature = "rust1", since = "1.0.0")] #[must_use] pub fn name(&self) -> Option<&str> { - self.inner.name.as_str() + self.0.name() } fn cname(&self) -> Option<&CStr> { - self.inner.name.as_cstr() + self.0.cname() + } + + /// Consumes the `Thread`, returning a raw pointer. + /// + /// To avoid a memory leak the pointer must be converted + /// back into a `Thread` using [`Thread::from_raw`]. + /// + /// # Examples + /// + /// ``` + /// #![feature(thread_raw)] + /// + /// use std::thread::{self, Thread}; + /// + /// let thread = thread::current(); + /// let id = thread.id(); + /// let ptr = Thread::into_raw(thread); + /// unsafe { + /// assert_eq!(Thread::from_raw(ptr).id(), id); + /// } + /// ``` + #[unstable(feature = "thread_raw", issue = "97523")] + pub fn into_raw(self) -> *const () { + self.0.into_raw() + } + + /// Constructs a `Thread` from a raw pointer. + /// + /// The raw pointer must have been previously returned + /// by a call to [`Thread::into_raw`]. + /// + /// # Safety + /// + /// This function is unsafe because improper use may lead + /// to memory unsafety, even if the returned `Thread` is never + /// accessed. + /// + /// Creating a `Thread` from a pointer other than one returned + /// from [`Thread::into_raw`] is **undefined behavior**. + /// + /// Calling this function twice on the same raw pointer can lead + /// to a double-free if both `Thread` instances are dropped. + #[unstable(feature = "thread_raw", issue = "97523")] + pub unsafe fn from_raw(ptr: *const ()) -> Thread { + // Safety: Upheld by caller. + unsafe { Thread(Inner::from_raw(ptr)) } } } @@ -1737,7 +1805,7 @@ impl JoinHandle { /// operations that happen after `join` returns. /// /// If the associated thread panics, [`Err`] is returned with the parameter given - /// to [`panic!`]. + /// to [`panic!`] (though see the Notes below). /// /// [`Err`]: crate::result::Result::Err /// [atomic memory orderings]: crate::sync::atomic @@ -1759,6 +1827,18 @@ impl JoinHandle { /// }).unwrap(); /// join_handle.join().expect("Couldn't join on the associated thread"); /// ``` + /// + /// # Notes + /// + /// If a "foreign" unwinding operation (e.g. an exception thrown from C++ + /// code, or a `panic!` in Rust code compiled or linked with a different + /// runtime) unwinds all the way to the thread root, the process may be + /// aborted; see the Notes on [`thread::spawn`]. If the process is not + /// aborted, this function will return a `Result::Err` containing an opaque + /// type. + /// + /// [`catch_unwind`]: ../../std/panic/fn.catch_unwind.html + /// [`thread::spawn`]: spawn #[stable(feature = "rust1", since = "1.0.0")] pub fn join(self) -> Result { self.0.join() diff --git a/library/std/src/thread/scoped.rs b/library/std/src/thread/scoped.rs index ba27c9220aea5..b2305b1eda7e1 100644 --- a/library/std/src/thread/scoped.rs +++ b/library/std/src/thread/scoped.rs @@ -1,8 +1,8 @@ -use super::{current, park, Builder, JoinInner, Result, Thread}; +use super::{Builder, JoinInner, Result, Thread, current, park}; use crate::marker::PhantomData; -use crate::panic::{catch_unwind, resume_unwind, AssertUnwindSafe}; -use crate::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; +use crate::panic::{AssertUnwindSafe, catch_unwind, resume_unwind}; use crate::sync::Arc; +use crate::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; use crate::{fmt, io}; /// A scope to spawn scoped threads in. diff --git a/library/std/src/thread/tests.rs b/library/std/src/thread/tests.rs index aa464d56f95b2..ff45e82bd9c71 100644 --- a/library/std/src/thread/tests.rs +++ b/library/std/src/thread/tests.rs @@ -2,7 +2,7 @@ use super::Builder; use crate::any::Any; use crate::panic::panic_any; use crate::sync::atomic::{AtomicBool, Ordering}; -use crate::sync::mpsc::{channel, Sender}; +use crate::sync::mpsc::{Sender, channel}; use crate::sync::{Arc, Barrier}; use crate::thread::{self, Scope, ThreadId}; use crate::time::{Duration, Instant}; diff --git a/library/std/src/time.rs b/library/std/src/time.rs index ae46670c25e61..9f4f8a0d0880c 100644 --- a/library/std/src/time.rs +++ b/library/std/src/time.rs @@ -178,9 +178,9 @@ pub struct Instant(time::Instant); /// system. /// /// A `SystemTime` does not count leap seconds. -/// `SystemTime::now()`'s behaviour around a leap second +/// `SystemTime::now()`'s behavior around a leap second /// is the same as the operating system's wall clock. -/// The precise behaviour near a leap second +/// The precise behavior near a leap second /// (e.g. whether the clock appears to run slow or fast, or stop, or jump) /// depends on platform and configuration, /// so should not be relied on. @@ -280,6 +280,7 @@ impl Instant { /// ``` #[must_use] #[stable(feature = "time2", since = "1.8.0")] + #[cfg_attr(not(test), rustc_diagnostic_item = "instant_now")] pub fn now() -> Instant { Instant(time::Instant::now()) } diff --git a/library/std/src/time/tests.rs b/library/std/src/time/tests.rs index de36dc4c9fd16..e88f2d5e80676 100644 --- a/library/std/src/time/tests.rs +++ b/library/std/src/time/tests.rs @@ -1,7 +1,7 @@ use core::fmt::Debug; #[cfg(not(target_arch = "wasm32"))] -use test::{black_box, Bencher}; +use test::{Bencher, black_box}; use super::{Duration, Instant, SystemTime, UNIX_EPOCH}; @@ -235,8 +235,8 @@ macro_rules! bench_instant_threaded { #[bench] #[cfg(not(target_arch = "wasm32"))] fn $bench_name(b: &mut Bencher) -> crate::thread::Result<()> { - use crate::sync::atomic::{AtomicBool, Ordering}; use crate::sync::Arc; + use crate::sync::atomic::{AtomicBool, Ordering}; let running = Arc::new(AtomicBool::new(true)); diff --git a/library/std/tests/create_dir_all_bare.rs b/library/std/tests/create_dir_all_bare.rs index 8becf713205ee..30f800c5aa2e6 100644 --- a/library/std/tests/create_dir_all_bare.rs +++ b/library/std/tests/create_dir_all_bare.rs @@ -1,4 +1,4 @@ -#![cfg(all(test, not(any(target_os = "emscripten", target_env = "sgx"))))] +#![cfg(all(test, not(any(target_os = "emscripten", target_os = "wasi", target_env = "sgx"))))] //! Note that this test changes the current directory so //! should not be in the same process as other tests. diff --git a/library/std/tests/process_spawning.rs b/library/std/tests/process_spawning.rs index d249eb7d50aa5..3e72e371ade19 100644 --- a/library/std/tests/process_spawning.rs +++ b/library/std/tests/process_spawning.rs @@ -5,7 +5,7 @@ use std::{env, fs, process, str}; mod common; #[test] -#[cfg_attr(miri, ignore)] // Process spawning not supported by Miri +#[cfg_attr(any(miri, target_os = "wasi"), ignore)] // Process spawning not supported by Miri and wasi fn issue_15149() { // If we're the parent, copy our own binary to a new directory. let my_path = env::current_exe().unwrap(); diff --git a/library/std/tests/run-time-detect.rs b/library/std/tests/run-time-detect.rs index dcd5cd7f6b9c7..dd14c0266aa4d 100644 --- a/library/std/tests/run-time-detect.rs +++ b/library/std/tests/run-time-detect.rs @@ -82,6 +82,7 @@ fn aarch64_linux() { println!("sha2: {}", is_aarch64_feature_detected!("sha2")); println!("sha3: {}", is_aarch64_feature_detected!("sha3")); println!("sm4: {}", is_aarch64_feature_detected!("sm4")); + println!("sme-b16b16: {}", is_aarch64_feature_detected!("sme-b16b16")); println!("sme-f16f16: {}", is_aarch64_feature_detected!("sme-f16f16")); println!("sme-f64f64: {}", is_aarch64_feature_detected!("sme-f64f64")); println!("sme-f8f16: {}", is_aarch64_feature_detected!("sme-f8f16")); diff --git a/library/std/tests/thread.rs b/library/std/tests/thread.rs index 79a981d0b0d18..1bb17d149fa10 100644 --- a/library/std/tests/thread.rs +++ b/library/std/tests/thread.rs @@ -4,7 +4,7 @@ use std::thread; use std::time::Duration; #[test] -#[cfg_attr(target_os = "emscripten", ignore)] +#[cfg_attr(any(target_os = "emscripten", target_os = "wasi"), ignore)] // no threads #[cfg_attr(miri, ignore)] // Miri does not like the thread leak fn sleep_very_long() { let finished = Arc::new(Mutex::new(false)); @@ -37,3 +37,44 @@ fn thread_local_containing_const_statements() { assert_eq!(CELL.get(), 1); assert_eq!(REFCELL.take(), 1); } + +#[test] +fn thread_local_hygeiene() { + // Previously `thread_local_inner!` had use imports for `LocalKey`, `Storage`, `EagerStorage` + // and `LazyStorage`. The use imports will shadow a user-provided type or type alias if the + // user-provided type or type alias has the same name. Make sure that this does not happen. See + // . + // + // NOTE: if the internal implementation details change (i.e. get renamed), this test should be + // updated. + + #![allow(dead_code)] + type LocalKey = (); + type Storage = (); + type LazyStorage = (); + type EagerStorage = (); + thread_local! { + static A: LocalKey = const { () }; + static B: Storage = const { () }; + static C: LazyStorage = const { () }; + static D: EagerStorage = const { () }; + } +} + +#[test] +// Include an ignore list on purpose, so that new platforms don't miss it +#[cfg_attr( + any( + target_os = "redox", + target_os = "l4re", + target_env = "sgx", + target_os = "solid_asp3", + target_os = "teeos", + target_os = "wasi" + ), + should_panic +)] +fn available_parallelism() { + // check that std::thread::available_parallelism() returns a valid value + assert!(thread::available_parallelism().is_ok()); +} diff --git a/library/stdarch b/library/stdarch index d9466edb4c53c..ff9a4445038ea 160000 --- a/library/stdarch +++ b/library/stdarch @@ -1 +1 @@ -Subproject commit d9466edb4c53cece8686ee6e17b028436ddf4151 +Subproject commit ff9a4445038eae46fd095188740946808581bc0e diff --git a/library/sysroot/Cargo.toml b/library/sysroot/Cargo.toml index 7165c3e48af42..aa6c3dc32e2ba 100644 --- a/library/sysroot/Cargo.toml +++ b/library/sysroot/Cargo.toml @@ -6,6 +6,7 @@ edition = "2021" # this is a dummy crate to ensure that all required crates appear in the sysroot [dependencies] proc_macro = { path = "../proc_macro" } +profiler_builtins = { path = "../profiler_builtins", optional = true } std = { path = "../std" } test = { path = "../test" } @@ -23,7 +24,7 @@ system-llvm-libunwind = ["std/system-llvm-libunwind"] panic-unwind = ["std/panic_unwind"] panic_immediate_abort = ["std/panic_immediate_abort"] optimize_for_size = ["std/optimize_for_size"] -profiler = ["std/profiler"] +profiler = ["dep:profiler_builtins"] std_detect_file_io = ["std/std_detect_file_io"] std_detect_dlsym_getauxval = ["std/std_detect_dlsym_getauxval"] std_detect_env_override = ["std/std_detect_env_override"] diff --git a/library/test/src/bench.rs b/library/test/src/bench.rs index b71def3b03223..62e51026b818c 100644 --- a/library/test/src/bench.rs +++ b/library/test/src/bench.rs @@ -1,15 +1,15 @@ //! Benchmarking module. -use std::panic::{catch_unwind, AssertUnwindSafe}; +use std::panic::{AssertUnwindSafe, catch_unwind}; use std::sync::{Arc, Mutex}; use std::time::{Duration, Instant}; use std::{cmp, io}; +use super::Sender; use super::event::CompletedTest; use super::options::BenchMode; use super::test_result::TestResult; use super::types::{TestDesc, TestId}; -use super::Sender; use crate::stats; /// An identity function that *__hints__* to the compiler to be maximally pessimistic about what diff --git a/library/test/src/lib.rs b/library/test/src/lib.rs index 632f8d161affa..30ccfe2af8dbd 100644 --- a/library/test/src/lib.rs +++ b/library/test/src/lib.rs @@ -18,6 +18,7 @@ #![doc(test(attr(deny(warnings))))] #![doc(rust_logo)] #![feature(rustdoc_internals)] +#![feature(file_buffered)] #![feature(internal_output_capture)] #![feature(staged_api)] #![feature(process_exitcode_internals)] @@ -28,17 +29,17 @@ pub use cli::TestOpts; -pub use self::bench::{black_box, Bencher}; +pub use self::ColorConfig::*; +pub use self::bench::{Bencher, black_box}; pub use self::console::run_tests_console; pub use self::options::{ColorConfig, Options, OutputFormat, RunIgnored, ShouldPanic}; pub use self::types::TestName::*; pub use self::types::*; -pub use self::ColorConfig::*; // Module to be used by rustc to compile tests in libtest pub mod test { pub use crate::bench::Bencher; - pub use crate::cli::{parse_opts, TestOpts}; + pub use crate::cli::{TestOpts, parse_opts}; pub use crate::helpers::metrics::{Metric, MetricMap}; pub use crate::options::{Options, RunIgnored, RunStrategy, ShouldPanic}; pub use crate::test_result::{TestResult, TrFailed, TrFailedMsg, TrIgnored, TrOk}; @@ -53,9 +54,9 @@ pub mod test { use std::collections::VecDeque; use std::io::prelude::Write; use std::mem::ManuallyDrop; -use std::panic::{self, catch_unwind, AssertUnwindSafe, PanicHookInfo}; +use std::panic::{self, AssertUnwindSafe, PanicHookInfo, catch_unwind}; use std::process::{self, Command, Termination}; -use std::sync::mpsc::{channel, Sender}; +use std::sync::mpsc::{Sender, channel}; use std::sync::{Arc, Mutex}; use std::time::{Duration, Instant}; use std::{env, io, thread}; @@ -649,8 +650,8 @@ fn run_test_in_process( io::set_output_capture(None); let test_result = match result { - Ok(()) => calc_result(&desc, Ok(()), &time_opts, &exec_time), - Err(e) => calc_result(&desc, Err(e.as_ref()), &time_opts, &exec_time), + Ok(()) => calc_result(&desc, Ok(()), time_opts.as_ref(), exec_time.as_ref()), + Err(e) => calc_result(&desc, Err(e.as_ref()), time_opts.as_ref(), exec_time.as_ref()), }; let stdout = data.lock().unwrap_or_else(|e| e.into_inner()).to_vec(); let message = CompletedTest::new(id, desc, test_result, exec_time, stdout); @@ -711,7 +712,8 @@ fn spawn_test_subprocess( formatters::write_stderr_delimiter(&mut test_output, &desc.name); test_output.extend_from_slice(&stderr); - let result = get_result_from_exit_code(&desc, status, &time_opts, &exec_time); + let result = + get_result_from_exit_code(&desc, status, time_opts.as_ref(), exec_time.as_ref()); (result, test_output, exec_time) })(); @@ -723,8 +725,8 @@ fn run_test_in_spawned_subprocess(desc: TestDesc, runnable_test: RunnableTest) - let builtin_panic_hook = panic::take_hook(); let record_result = Arc::new(move |panic_info: Option<&'_ PanicHookInfo<'_>>| { let test_result = match panic_info { - Some(info) => calc_result(&desc, Err(info.payload()), &None, &None), - None => calc_result(&desc, Ok(()), &None, &None), + Some(info) => calc_result(&desc, Err(info.payload()), None, None), + None => calc_result(&desc, Ok(()), None, None), }; // We don't support serializing TrFailedMsg, so just diff --git a/library/test/src/term/terminfo/mod.rs b/library/test/src/term/terminfo/mod.rs index 67eec3ca50f48..974b8afd598dd 100644 --- a/library/test/src/term/terminfo/mod.rs +++ b/library/test/src/term/terminfo/mod.rs @@ -3,15 +3,14 @@ use std::collections::HashMap; use std::fs::File; use std::io::prelude::*; -use std::io::{self, BufReader}; use std::path::Path; -use std::{env, error, fmt}; +use std::{env, error, fmt, io}; -use parm::{expand, Param, Variables}; +use parm::{Param, Variables, expand}; use parser::compiled::{msys_terminfo, parse}; use searcher::get_dbpath_for_term; -use super::{color, Terminal}; +use super::{Terminal, color}; /// A parsed terminfo database entry. #[allow(unused)] @@ -102,8 +101,7 @@ impl TermInfo { } // Keep the metadata small fn _from_path(path: &Path) -> Result { - let file = File::open(path).map_err(Error::IoError)?; - let mut reader = BufReader::new(file); + let mut reader = File::open_buffered(path).map_err(Error::IoError)?; parse(&mut reader, false).map_err(Error::MalformedTerminfo) } } diff --git a/library/test/src/term/win.rs b/library/test/src/term/win.rs index ce9cad37f306b..c77e6aac478bc 100644 --- a/library/test/src/term/win.rs +++ b/library/test/src/term/win.rs @@ -5,7 +5,7 @@ use std::io; use std::io::prelude::*; -use super::{color, Terminal}; +use super::{Terminal, color}; /// A Terminal implementation that uses the Win32 Console API. pub(crate) struct WinConsole { diff --git a/library/test/src/test_result.rs b/library/test/src/test_result.rs index c5f4b03bfc96c..79fe07bc1ac5c 100644 --- a/library/test/src/test_result.rs +++ b/library/test/src/test_result.rs @@ -42,8 +42,8 @@ pub enum TestResult { pub fn calc_result<'a>( desc: &TestDesc, task_result: Result<(), &'a (dyn Any + 'static + Send)>, - time_opts: &Option, - exec_time: &Option, + time_opts: Option<&time::TestTimeOptions>, + exec_time: Option<&time::TestExecTime>, ) -> TestResult { let result = match (&desc.should_panic, task_result) { (&ShouldPanic::No, Ok(())) | (&ShouldPanic::Yes, Err(_)) => TestResult::TrOk, @@ -96,8 +96,8 @@ pub fn calc_result<'a>( pub fn get_result_from_exit_code( desc: &TestDesc, status: ExitStatus, - time_opts: &Option, - exec_time: &Option, + time_opts: Option<&time::TestTimeOptions>, + exec_time: Option<&time::TestExecTime>, ) -> TestResult { let result = match status.code() { Some(TR_OK) => TestResult::TrOk, diff --git a/library/test/src/tests.rs b/library/test/src/tests.rs index ba2f35362c54f..e85e61090a91b 100644 --- a/library/test/src/tests.rs +++ b/library/test/src/tests.rs @@ -3,11 +3,11 @@ use crate::{ console::OutputLocation, formatters::PrettyFormatter, test::{ - parse_opts, MetricMap, // FIXME (introduced by #65251) // ShouldPanic, StaticTestName, TestDesc, TestDescAndFn, TestOpts, TestTimeOptions, // TestType, TrFailedMsg, TrIgnored, TrOk, + parse_opts, }, time::{TestTimeOptions, TimeThreshold}, }; diff --git a/library/unwind/Cargo.toml b/library/unwind/Cargo.toml index 590de31a678ca..569a1b3299e5f 100644 --- a/library/unwind/Cargo.toml +++ b/library/unwind/Cargo.toml @@ -22,7 +22,7 @@ cfg-if = "1.0" libc = { version = "0.2.140", features = ['rustc-dep-of-std'], default-features = false } [target.'cfg(target_os = "xous")'.dependencies] -unwinding = { version = "0.2.1", features = ['rustc-dep-of-std', 'unwinder', 'fde-custom'], default-features = false } +unwinding = { version = "0.2.3", features = ['rustc-dep-of-std', 'unwinder', 'fde-custom'], default-features = false } [features] diff --git a/library/unwind/src/lib.rs b/library/unwind/src/lib.rs index 26ed00bfbd53e..79baa5b0b83ec 100644 --- a/library/unwind/src/lib.rs +++ b/library/unwind/src/lib.rs @@ -2,12 +2,10 @@ #![unstable(feature = "panic_unwind", issue = "32837")] #![feature(link_cfg)] #![feature(staged_api)] -#![feature(strict_provenance)] -#![cfg_attr(target_arch = "wasm64", feature(simd_wasm64))] #![cfg_attr(not(target_env = "msvc"), feature(libc))] #![cfg_attr( all(target_family = "wasm", not(target_os = "emscripten")), - feature(link_llvm_intrinsics) + feature(simd_wasm64, wasm_exception_handling_intrinsics) )] #![allow(internal_features)] @@ -23,6 +21,7 @@ cfg_if::cfg_if! { target_os = "none", target_os = "espidf", target_os = "rtems", + target_os = "nuttx", ))] { // These "unix" family members do not have unwinder. } else if #[cfg(any( diff --git a/library/unwind/src/libunwind.rs b/library/unwind/src/libunwind.rs index e5e28f32e4dbf..715f8b57876ae 100644 --- a/library/unwind/src/libunwind.rs +++ b/library/unwind/src/libunwind.rs @@ -33,10 +33,10 @@ pub const unwinder_private_data_size: usize = 2; #[cfg(all(target_arch = "x86_64", target_os = "windows"))] pub const unwinder_private_data_size: usize = 6; -#[cfg(all(target_arch = "arm", not(all(target_vendor = "apple", not(target_os = "watchos")))))] +#[cfg(all(target_arch = "arm", not(target_vendor = "apple")))] pub const unwinder_private_data_size: usize = 20; -#[cfg(all(target_arch = "arm", all(target_vendor = "apple", not(target_os = "watchos"))))] +#[cfg(all(target_arch = "arm", target_vendor = "apple"))] pub const unwinder_private_data_size: usize = 5; #[cfg(all(target_arch = "aarch64", target_pointer_width = "64", not(target_os = "windows")))] @@ -123,8 +123,11 @@ extern "C" { } cfg_if::cfg_if! { -if #[cfg(any(all(target_vendor = "apple", not(target_os = "watchos")), target_os = "netbsd", not(target_arch = "arm")))] { +if #[cfg(any(target_vendor = "apple", target_os = "netbsd", not(target_arch = "arm")))] { // Not ARM EHABI + // + // 32-bit ARM on iOS/tvOS/watchOS use either DWARF/Compact unwinding or + // "setjmp-longjmp" / SjLj unwinding. #[repr(C)] #[derive(Copy, Clone, PartialEq)] pub enum _Unwind_Action { @@ -219,14 +222,14 @@ if #[cfg(any(all(target_vendor = "apple", not(target_os = "watchos")), target_os pub unsafe fn _Unwind_GetGR(ctx: *mut _Unwind_Context, reg_index: c_int) -> _Unwind_Word { let mut val: _Unwind_Word = core::ptr::null(); _Unwind_VRS_Get(ctx, _UVRSC_CORE, reg_index as _Unwind_Word, _UVRSD_UINT32, - core::ptr::addr_of_mut!(val) as *mut c_void); + (&raw mut val) as *mut c_void); val } pub unsafe fn _Unwind_SetGR(ctx: *mut _Unwind_Context, reg_index: c_int, value: _Unwind_Word) { let mut value = value; _Unwind_VRS_Set(ctx, _UVRSC_CORE, reg_index as _Unwind_Word, _UVRSD_UINT32, - core::ptr::addr_of_mut!(value) as *mut c_void); + (&raw mut value) as *mut c_void); } pub unsafe fn _Unwind_GetIP(ctx: *mut _Unwind_Context) @@ -259,8 +262,8 @@ if #[cfg(any(all(target_vendor = "apple", not(target_os = "watchos")), target_os cfg_if::cfg_if! { if #[cfg(all(target_vendor = "apple", not(target_os = "watchos"), target_arch = "arm"))] { - // 32-bit ARM Apple (except for watchOS) uses SjLj and does not provide - // _Unwind_Backtrace() + // 32-bit ARM Apple (except for watchOS armv7k specifically) uses SjLj and + // does not provide _Unwind_Backtrace() extern "C-unwind" { pub fn _Unwind_SjLj_RaiseException(e: *mut _Unwind_Exception) -> _Unwind_Reason_Code; } diff --git a/library/unwind/src/unwinding.rs b/library/unwind/src/unwinding.rs index b3460791ce5ca..1b94005ab6cd0 100644 --- a/library/unwind/src/unwinding.rs +++ b/library/unwind/src/unwinding.rs @@ -32,7 +32,7 @@ pub use unwinding::abi::{UnwindContext, UnwindException}; pub enum _Unwind_Context {} pub use unwinding::custom_eh_frame_finder::{ - set_custom_eh_frame_finder, EhFrameFinder, FrameInfo, FrameInfoKind, + EhFrameFinder, FrameInfo, FrameInfoKind, set_custom_eh_frame_finder, }; pub type _Unwind_Exception_Class = u64; diff --git a/library/unwind/src/wasm.rs b/library/unwind/src/wasm.rs index f4ffac1ba16da..2d36a8be00469 100644 --- a/library/unwind/src/wasm.rs +++ b/library/unwind/src/wasm.rs @@ -40,29 +40,25 @@ pub unsafe fn _Unwind_DeleteException(exception: *mut _Unwind_Exception) { } pub unsafe fn _Unwind_RaiseException(exception: *mut _Unwind_Exception) -> _Unwind_Reason_Code { - #[cfg(panic = "unwind")] - extern "C" { - /// LLVM lowers this intrinsic to the `throw` instruction. - // FIXME(coolreader18): move to stdarch - #[link_name = "llvm.wasm.throw"] - fn wasm_throw(tag: i32, ptr: *mut u8) -> !; - } - // The wasm `throw` instruction takes a "tag", which differentiates certain // types of exceptions from others. LLVM currently just identifies these // via integers, with 0 corresponding to C++ exceptions and 1 to C setjmp()/longjmp(). // Ideally, we'd be able to choose something unique for Rust, but for now, // we pretend to be C++ and implement the Itanium exception-handling ABI. cfg_if::cfg_if! { - // for now, unless we're -Zbuild-std with panic=unwind, never codegen a throw. + // panic=abort is default for wasm targets. Because an unknown instruction is a load-time + // error on wasm, instead of a runtime error like on traditional architectures, we never + // want to codegen a `throw` instruction, as that would break users using runtimes that + // don't yet support exceptions. The only time this first branch would be selected is if + // the user explicitly opts in to wasm exceptions, via -Zbuild-std with -Cpanic=unwind. if #[cfg(panic = "unwind")] { - wasm_throw(0, exception.cast()) + // corresponds with llvm::WebAssembly::Tag::CPP_EXCEPTION + // in llvm-project/llvm/include/llvm/CodeGen/WasmEHFuncInfo.h + const CPP_EXCEPTION_TAG: i32 = 0; + core::arch::wasm::throw::(exception.cast()) } else { let _ = exception; - #[cfg(target_arch = "wasm32")] - core::arch::wasm32::unreachable(); - #[cfg(target_arch = "wasm64")] - core::arch::wasm64::unreachable(); + core::arch::wasm::unreachable() } } } diff --git a/library/windows_targets/src/lib.rs b/library/windows_targets/src/lib.rs index 1965b6cf4ce8f..395cd6a4bab55 100644 --- a/library/windows_targets/src/lib.rs +++ b/library/windows_targets/src/lib.rs @@ -38,4 +38,5 @@ pub macro link { #[link(name = "ntdll")] #[link(name = "userenv")] #[link(name = "ws2_32")] +#[link(name = "dbghelp")] // required for backtrace-rs symbolization extern "C" {} diff --git a/rust-toolchain.toml b/rust-toolchain.toml index 9f6970c17ee5f..00f10cd5d5c3a 100644 --- a/rust-toolchain.toml +++ b/rust-toolchain.toml @@ -2,5 +2,5 @@ # standard library we currently track. [toolchain] -channel = "nightly-2024-09-08" +channel = "nightly-2024-11-03" components = ["llvm-tools-preview", "rustc-dev", "rust-src", "rustfmt"] diff --git a/tool_config/kani-version.toml b/tool_config/kani-version.toml index 59b19da6a5958..f75f2058e67f2 100644 --- a/tool_config/kani-version.toml +++ b/tool_config/kani-version.toml @@ -1,5 +1,5 @@ # This version should be updated whenever there is a change that makes this version of kani -# incomaptible with the verify-std repo. +# incompatible with the verify-std repo. [kani] -commit = "5f8f513d297827cfdce4c48065e51973ba563068" +commit = "2565ef65767a696f1d519b42621b4e502e8970d0"

(&mut self, predicate: P) -> Option where P: FnMut(Self::Item) -> bool, @@ -3149,7 +3103,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn max(self) -> Option where Self: Sized, @@ -3186,7 +3139,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn min(self) -> Option where Self: Sized, @@ -3209,7 +3161,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "iter_cmp_by_key", since = "1.6.0")] - #[rustc_do_not_const_check] fn max_by_key(self, f: F) -> Option where Self: Sized, @@ -3243,7 +3194,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "iter_max_by", since = "1.15.0")] - #[rustc_do_not_const_check] fn max_by(self, compare: F) -> Option where Self: Sized, @@ -3271,7 +3221,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "iter_cmp_by_key", since = "1.6.0")] - #[rustc_do_not_const_check] fn min_by_key(self, f: F) -> Option where Self: Sized, @@ -3305,7 +3254,6 @@ pub trait Iterator { /// ``` #[inline] #[stable(feature = "iter_min_by", since = "1.15.0")] - #[rustc_do_not_const_check] fn min_by(self, compare: F) -> Option where Self: Sized, @@ -3343,7 +3291,6 @@ pub trait Iterator { #[inline] #[doc(alias = "reverse")] #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn rev(self) -> Rev where Self: Sized + DoubleEndedIterator, @@ -3380,7 +3327,6 @@ pub trait Iterator { /// assert_eq!(z, [3, 6]); /// ``` #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] fn unzip(self) -> (FromA, FromB) where FromA: Default + Extend, @@ -3411,7 +3357,7 @@ pub trait Iterator { /// assert_eq!(v_map, vec![1, 2, 3]); /// ``` #[stable(feature = "iter_copied", since = "1.36.0")] - #[rustc_do_not_const_check] + #[cfg_attr(not(test), rustc_diagnostic_item = "iter_copied")] fn copied<'a, T: 'a>(self) -> Copied where Self: Sized + Iterator, @@ -3459,7 +3405,7 @@ pub trait Iterator { /// assert_eq!(&[vec![23]], &faster[..]); /// ``` #[stable(feature = "rust1", since = "1.0.0")] - #[rustc_do_not_const_check] + #[cfg_attr(not(test), rustc_diagnostic_item = "iter_cloned")] fn cloned<'a, T: 'a>(self) -> Cloned where Self: Sized + Iterator, @@ -3492,7 +3438,6 @@ pub trait Iterator { /// ``` #[stable(feature = "rust1", since = "1.0.0")] #[inline] - #[rustc_do_not_const_check] fn cycle(self) -> Cycle where Self: Sized + Clone, @@ -3536,7 +3481,6 @@ pub trait Iterator { /// ``` #[track_caller] #[unstable(feature = "iter_array_chunks", reason = "recently added", issue = "100450")] - #[rustc_do_not_const_check] fn array_chunks(self) -> ArrayChunks where Self: Sized, @@ -3568,7 +3512,6 @@ pub trait Iterator { /// assert_eq!(sum, 6); /// ``` #[stable(feature = "iter_arith", since = "1.11.0")] - #[rustc_do_not_const_check] fn sum(self) -> S where Self: Sized, @@ -3601,7 +3544,6 @@ pub trait Iterator { /// assert_eq!(factorial(5), 120); /// ``` #[stable(feature = "iter_arith", since = "1.11.0")] - #[rustc_do_not_const_check] fn product