Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace owning_ref with a safer datastructure #97770

Closed
wants to merge 3 commits into from

Conversation

Noratrieb
Copy link
Member

owning_ref has soundness problems when used together with Box<T>, since moving a box invalidates all pointers to it under stacked borrows.
It's only used in once place, in metadata handling, where it's used to have a subslice view into an owned slice. So this pull request adds a new datastructure, this kind of owned_slice, which gives a subslice view into an owned slice with almost no unsafe code, that passes miri in strict mode.
I replaced the only usage of owning_ref with this new datastructure, and then subsequently removed owning_ref from rustc_data_structures.

@rustbot rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Jun 5, 2022
@rust-highfive
Copy link
Collaborator

r? @cjgillot

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 5, 2022
@Noratrieb Noratrieb changed the title Replace owning_ref by a safer datastructure Replace owning_ref with a safer datastructure Jun 5, 2022
@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

The owned slice represents an owned value with a view into that
owned slice. For example, it might own a `Vec<i32>` of `[1, 2]`,
but only deref to `[2]`. It's almost entirely written using safe
code, except for one unsafe block to calculate a pointer difference.
`owning_ref` of a `Box<T>` has soundness problems because it stores
pointers to a box that is then invalidated. Instead, use the safer
`owned_slice`, which presents a more specialized abstraction.
This doesn't change the general layout or indirection of the structure,
but only the datastructure used as a wrapper.
`owning_ref` has soundness problems when used with `Box<T>`, and the
previous commit replaced its usages with a better abstraction.
This means that we can now get rid of it.
@JakobDegen
Copy link
Contributor

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 5, 2022
@bors
Copy link
Contributor

bors commented Jun 5, 2022

⌛ Trying commit 99419ea with merge c79658f2ae2271230d400d079a30d01d842d33be...

@bors
Copy link
Contributor

bors commented Jun 6, 2022

☀️ Try build successful - checks-actions
Build commit: c79658f2ae2271230d400d079a30d01d842d33be (c79658f2ae2271230d400d079a30d01d842d33be)

@rust-timer
Copy link
Collaborator

Queued c79658f2ae2271230d400d079a30d01d842d33be with parent fee3a45, future comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (c79658f2ae2271230d400d079a30d01d842d33be): comparison url.

Instruction count

  • Primary benchmarks: 😿 relevant regressions found
  • Secondary benchmarks: 😿 relevant regressions found
mean1 max count2
Regressions 😿
(primary)
0.6% 2.0% 93
Regressions 😿
(secondary)
1.3% 2.8% 48
Improvements 🎉
(primary)
N/A N/A 0
Improvements 🎉
(secondary)
N/A N/A 0
All 😿🎉 (primary) 0.6% 2.0% 93

Max RSS (memory usage)

Results
  • Primary benchmarks: no relevant changes found
  • Secondary benchmarks: 😿 relevant regression found
mean1 max count2
Regressions 😿
(primary)
N/A N/A 0
Regressions 😿
(secondary)
2.2% 2.2% 1
Improvements 🎉
(primary)
N/A N/A 0
Improvements 🎉
(secondary)
N/A N/A 0
All 😿🎉 (primary) N/A N/A 0

Cycles

Results
  • Primary benchmarks: no relevant changes found
  • Secondary benchmarks: mixed results
mean1 max count2
Regressions 😿
(primary)
N/A N/A 0
Regressions 😿
(secondary)
2.7% 4.0% 3
Improvements 🎉
(primary)
N/A N/A 0
Improvements 🎉
(secondary)
-3.0% -3.0% 1
All 😿🎉 (primary) N/A N/A 0

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf +perf-regression

Footnotes

  1. the arithmetic mean of the percent change 2 3

  2. number of relevant changes 2 3

@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Jun 6, 2022
@Noratrieb
Copy link
Member Author

That's not great, but I think I can optimize this.

@cjgillot
Copy link
Contributor

cjgillot commented Jun 6, 2022

I was going to use this in #88186 to keep Lrcs pointing to sub-items. Is that unsound? What do you recommend as a replacement?
See 3f18306

@Noratrieb
Copy link
Member Author

Hmm, using it with Lrc is not really unsound currently as far as I know, since Rc/Arc don't have the magic of Box currently. So it should be fine. If keeping owning_ref is desired, I will not remove it here. But since it's used together with Box here right now, it is unsound, and should be changed.

Though, now that I think about it, it might be a better idea to add a AliasableBox<T> datastructure here, that behaves like Box<T> except without the aliasing magic that makes an owning_ref of it unsound. This way we could keep the current behaviour with owning_ref and keep owning_ref, but making it clear that it should be used with AliasableBox<T> instead.

@ibraheemdev
Copy link
Member

Could also use ouroboros which is based on aliasable::AliasableBox.

@bjorn3
Copy link
Member

bjorn3 commented Jun 9, 2022

owning_ref has soundness problems when used together with Box, since moving a box invalidates all pointers to it under stacked borrows.

Would it make sense to remove the StableDeref impl for Box<T> then?

@Noratrieb
Copy link
Member Author

Yes. Under stacked borrows (currently), box does not obey the rules of stable deref.

@petrochenkov
Copy link
Contributor

Allocating on the heap is the standard way to ensure address stability, and Box is a standard way to allocate on the heap.
So it's kind of questionable that we have to do it at all.

@bjorn3
Copy link
Member

bjorn3 commented Jun 9, 2022

I believe rustc only uses owning ref with Vec<u8>, Arc<[u8]> and Mmap.

@Noratrieb
Copy link
Member Author

This Mmap is a boxed trait object IIRC (for the metadata)

@bjorn3
Copy link
Member

bjorn3 commented Jun 9, 2022

No, it is either an memmap2::Mmap or an Vec depending on the target:

pub struct Mmap(memmap2::Mmap);
memmap2::Mmap contains a raw pointer, not Box.

@Noratrieb
Copy link
Member Author

In rustc_data_structures/src/sync.rs (I can't link it I'm on mobile)

pub type MetadataRef = OwningRef<Box<dyn Erased + Send + Sync>, [u8]>;

Here a box is used

@bjorn3
Copy link
Member

bjorn3 commented Jun 9, 2022

Right. The Box<dyn Erased> may be replacable with Mmap. Or alternatively a new type for erased owners that doesn't use Box could be added.

@apiraino
Copy link
Contributor

Switching to waiting on author based on this comment. Feel free to request a review when ready, thanks!

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 30, 2022
@Noratrieb
Copy link
Member Author

I'll block this on the hope that we can make box aliasable, therefore making this PR useless. If that doesn't happen, I'd add an AliasableBox and use that with the owning_ref here.

@JohnCSimon JohnCSimon added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 13, 2022
@JohnCSimon
Copy link
Member

Ping from triage:

I'll block this on the hope that we can make box aliasable, therefore making this PR useless. If that doesn't happen, I'd add an AliasableBox and use that with the owning_ref here.

@Nilstrieb What is the status of this PR? If you're not moving forward with this please close it.

@Noratrieb
Copy link
Member Author

I'll close it for now, I might pick it up in the future again depending on what happens with box.

@Noratrieb Noratrieb closed this Oct 3, 2022
@Noratrieb Noratrieb deleted the yeet-owning-ref branch December 3, 2022 19:40
@Noratrieb Noratrieb restored the yeet-owning-ref branch December 3, 2022 19:45
@Noratrieb Noratrieb deleted the yeet-owning-ref branch December 3, 2022 19:46
@Noratrieb Noratrieb restored the yeet-owning-ref branch December 3, 2022 19:46
@Noratrieb Noratrieb deleted the yeet-owning-ref branch December 23, 2022 21:05
This was referenced Apr 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
perf-regression Performance regression. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.