Tracking issue: deref patterns

[nrc]

Tracking issue for implementing deref patterns (#[feature(deref_patterns)]).

deref patterns project group repo
lang team initiative issue

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps

• Implement the feature
  • String -> &str: string_deref_patterns implementation: Minimal implementation of implicit deref patterns for Strings Minimal implementation of implicit deref patterns for Strings #98914 (stabilization not started)
  • Deref patterns for other std types
• Adjust documentation (see instructions on rustc-dev-guide)
• Write an RFC
• Stabilization PR (see instructions on rustc-dev-guide)

Status

The current implementation uses a placeholder deref!(<pat>) syntax and is limited to types in the standard library. See the design proposal document for more details.

We limit to types in the standard library using the unstable trait DerefPure (#[feature(deref_pure_trait)]). It is not intended to be stabilized at this stage.

• Supported std types: Box, Rc, Arc, Vec, String, Cow, Pin, ManuallyDrop, Ref, RefMut.
• Unstable types that would be eligible : ThinBox, UniqueRc, LazyCell, LazyLock,

There is also a feature gate just for matching string literals on Strings, under #[feature(string_deref_patterns)].

Unresolved Questions

None at this stage

Implementation history

Beta Give feedback

1. Minimal implementation of implicit deref patterns for Strings #98914
   F-deref_patterns S-waiting-on-bors T-compiler T-libs merged-by-bors +5
   
2. [WIP] implement explicit deref patterns through k#deref #119467
   F-deref_patterns S-waiting-on-author T-compiler T-rustdoc +4
   
3. deref patterns: bare-bones feature gate and typechecking #122222
   F-deref_patterns S-waiting-on-bors T-compiler +3
   
4. Implement macro-based deref!() syntax for deref patterns #122793
   F-deref_patterns S-waiting-on-bors T-compiler T-libs +4
   
5. Require DerefMut and DerefPure on deref!() patterns when appropriate #122835
   F-deref_patterns S-waiting-on-bors T-compiler T-libs T-rustdoc +5
   
6. match lowering: make false edges more precise #123324
   S-waiting-on-bors T-compiler +2
   
7. deref patterns: lower deref patterns to MIR #122598
   F-deref_patterns S-waiting-on-bors T-compiler +3
   
8. deref patterns: impl DerefPure for more std types #123480
   F-deref_patterns S-waiting-on-author T-libs +3
   

[nrc]

@roxelo or @matthewjasper would you be able to mentor @chorman0773 on implementing the compiler parts of this please?

[roxelo]

@nrc I would not consider myself an expert on the pattern matching code but I can try to help mentor...

[AlbertMarashi]

I have this code, it does not compile. I am trying to match a Box<str> inside of a pattern, but the error is:

expected str, found `&str`

but there's no good way to match Box<str> in a deeply nested pattern...

#![feature(box_patterns)]
struct Type {
    pub name: Box<str>,
    pub generics: Option<Vec<Type>>
}

fn main(){
    let kind = Type {
        name: "User".into(),
        generics: Some(Vec::new())
    };

    match kind {
        Type {
            name: box "User",
            generics
        } => {}
        _ => unimplemented!()
    }
}

[chorman0773]

This would be should get solved by deref patterns.

…

On Fri, Feb 4, 2022 at 05:39 Albert Marashi ***@***.***> wrote: I have this code, it does not compile. I am trying to match a Box<str> inside of a pattern, but the error is: expected str, found '&str' but there's no good way to match Box<str> in a deeply nested pattern... #![feature(box_patterns)]struct Type { pub name: Box<str>, pub generics: Option<Vec<Type>> } fn main(){ let kind = Type { name: "User".into(), generics: Some(Vec::new()) }; match kind { Type { name: box "User", generics } => {} _ => unimplemented!() } } — Reply to this email directly, view it on GitHub <#87121 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGLD27M7YLN2UD7MELBLH3UZOUG5ANCNFSM5AKRWKLA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AlbertMarashi]

@chorman0773 anything I can do to help move this forward? I'd really love to see this feature in rust, it would be insanely useful for deep matching of things like ASTs.

[rdrpenguin04]

Just bonked my head into this again; how can I help?

[safinaskar]

@rdrpenguin04 and everybody else who cannot wait: I made a proc macro crate, which implements deref patterns in stable Rust: https://crates.io/crates/match_deref

[tgross35]

Is anybody involved here able to provide a rough outline or example of what this will look like when stable? I'm imagining this will be something that's automatic for anything that implements Deref

if let Some(u8) = Some(Rc::new(10)) { ... }
if let Some("yellow world") = Some(Box::new(String::from("hello world"))) { ... }

edit: according to Mario on Zulip, this would be keywordless, maybe with a & or a * needed - that's not yet decided. Super elegant!

[simensgreen]

Why can't we make

trait Match {
    type Target: ?Sized;
    fn do_match(&self) -> &Self::Target;
}
trait MatchMut: Match {
    fn do_match_mut(&mut self) -> &mut Self::Target;
}

how is it done with Deref and Derefmut, respectively?
The compiler somehow correlates

*x
// and
<&XType as Deref>::deref(&x)

why not do the same with match?

match x {}
// and
<XType as Match>::do_match(&x)

This would solve how to match strings, SmartPointers, and other more complex constructs without adding new syntax to the language.

[tgross35]

@simensgreen the trait isn't really an issue at this point, since Deref/DerefMut already gives the information needed (possibly with the marker trait DerefPure that has been discussed a few places). I think at this point, the goal is to start out supporting only builtin types (&str/String, &T/Vec, &T/Box, Rc, Arc).

And I think the blocker at this point is just time/people. Look at #98914 needed just to support strings, it's a pretty big change

[tgross35]

@nrc would you maybe want to update this tracking issue to point to relevant per-type implementations? Probably just:

• String -> &str: string_deref_patterns implementation Minimal implementation of implicit deref patterns for Strings #98914 (stabilization not started)
• Box<T> -> &T: box_deref_patterns (not started)
• Rc<T> -> &T, Arc<T> -> &T: rc_deref_patterns (not started)
• Vec<T> -> &[T]: slice_deref_patterns (not started)
• impl DerefPure<Target = U> -> &U: (further discussion needed)

Assuming the goal is to chunk this huge project up into bits that could be stabilized individually.

[simensgreen]

@tgross35 The existence of such a trait will help to implement match not only for standard types, including strings, but also for any arbitrary types, like Mutex. Deref does not have enough information to make a match

[tgross35]

You wouldn't be able to pattern match directly on a Mutex, and that's a good thing because implicitly locking would lead to some very difficult to follow code. However, you could lock it yourself and then pattern match on the MutexGuard, which does Deref to &T.

Also, to quote zulip

doing it in general faces some thorny issues around speccing what happens when the deref has side effects, since we want to be able to optimize matches in ways that may not preserve how many deref calls are made

This would be the reason for DerefPure that's come up a few times, an (unsafe?) marker trait to specify a special case of Deref that has no side effects (which is already the case for much of the Deref use in std, there's just no way to express that)

[RalfJung]

This would be the reason for DerefPure that's come up a few times, an (unsafe?) marker trait to specify a special case of Deref that has no side effects (which is already the case for much of the Deref use in std, there's just no way to express that)

How exactly would this interact with our usual safety requirements / UB? Saying "it is UB to impl DerefPure on a non-pure impl Deref" is quite problematic as this would be UB which cannot be checked by a sanitizer such as Miri. Ideally we'd end up in a situation where a wrong DerefPure can lead to surprising behavior, but is not in and of itself UB.

Cc @rust-lang/opsem

[CAD97]

I would initially expect it to behave like any other unsafe trait implementation: it gives downstream code permission to rely on the guaranteed properties, but doesn't result in UB until downstream code relies on that property to maintain soundness. Or IOW, library UB.

I think it probably does need to be unsafe, though, because it's not just about collapsing multiple calls into one, but also introducing calls where they wouldn't otherwise exist by strictly inorder trial evaluation of match arms. Since this makes otherwise dead code live, that feels unsafe, but it also can be justified as just safe misbehavior.

"I don't say how many times this gets executed" is a pretty textbook example of library instability which can't be validated operationally, although in this case it's as part of the language rather than library. The only guarantee I think the deref impl gets is that it's evaluated at least once if an arm containing it is

I also think there might be an element of one deref place evaluation being sufficient for both by-ref and by-ref mut binding modes, which while not technically unsafe (it's safe to do &*&mut *it to use DerefMut to get &T) also feels like it should be an unsafe guarantee that both are equivalent and interchangable places modulo provenance.

What sanitizers could do is always execute all structurally reachable pattern derefs from any arms, whether that arm is even reachable or not. Randomized sanitizers could also probabilistically vary how many times the derefs get evaluated, including zero times if unnecessary, and many more times than present in patterns.

Alternatively, the strong position is to go super strict about pattern derefs being not just pure but also just being composed from place computation primitives; in this case such implementations could even be verified statically to be only composed of such operations. Even if this isn't stated required at a language level, validating such composition dynamically ensures actual purity and as such skipping dynamic validation of spurious (non)evaluation.

Then, only partially unrelated, is the question on whether DerefPure's purity promise is utilized anywhere outside of deref patterns. E.g. it could potentially be used to enable borrow splitting, either by caching the place evaluation and/or not invalidating previously split borrows, although the latter means such place evaluation can't use typical reborrowing references.

Properly built-in place evaluation has a number of extra differences to overloadable place evaluation (pattern transparency, borrow splitting, "deref move", purity, const), and it needs to be decided how much or how little of that DerefPure should provide before it can properly be discussed whether that requires it to be unsafe.

[tgross35]

@ThePuzzlemaker awesome! @Nadrieril or @CAD97 would be able to give you more details about where exactly to look, but it will touch a lot of the areas that were relevant in making this work for String #98914 (actually a lot fewer areas than that since we don’t want to do anything type-specific)

You should also drop by Zulip to coordinate efforts and ask any smaller questions, the relevant stream is here: https://rust-lang.zulipchat.com/#narrow/stream/281601-project-deref-patterns.

A reasonable first step is to make any pattern work for a single match for anything that is Deref, I don’t think there is any need to implement multiple matches (as discussed above) as part of the first PR (unless it comes naturally with the implementation).

[ThePuzzlemaker]

I've realized that I think this is a little out of my ability at the moment 😅, sorry
I realized that this requires a lot more knowledge of the compiler internals than I have right now and it would take me a looong time to get that knowledge, tbh

[Nadrieril]

Dw, a bunch of us are eager to get this and are slowly making progress

[cloudhead]

Would this allow for moving out of boxes? Or only getting a reference of the inner value? It's quite common for interpreters to have a recursive type like

enum Expr {
   If(Box<Expr>, Box<Expr>, Box<Expr>),
   ...
}

And being able to destructure that by moving out of Box would make code much simpler, eg.

match expr {
  If(cond, left, right) => { /* Use owned `Expr` values here */ }
}

[Nadrieril]

I think we want to get something like that eventually, but that's not included in this proposal. The difficulty is from a language design pov: what syntax to use, how/whether to generalize that to other types than Box (which draws in the thorny DerefMove situation).

[clarfonthey]

I mean, right now as it stands, Box already has its own DerefMove sugar (specifically, *b), so, I can't imagine it'd be that bad to add custom support for moving out of deref before a fully generic solution exists.

[RalfJung]

If this does not allow moving out of boxes, we should be careful to be forward-compatible with eventually allowing moving out of boxes.

[Nadrieril]

Work on the feature is restarting. I'd like to remind everyone that

A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

especially since the design aspects are contentious. Thank you! See you on zulip for discussions.

[Nadrieril]

Dear T-lang, the liaison for this initiative (cramertj) has retired from the lang team. Progress is therefore halted until a new liaison is found. I would like to raise this for the next triage meeting so a liaison can be found, if the team still wants this feature to happen. I will be there to answer questions.

[traviscross]

@rustbot labels -I-lang-nominated

We discussed this in the lang triage call today. The consensus was that I would pick up the liaisoning here, so please let me know whenever anything is needed to move this along. Thanks to @Nadrieril for pushing this forward; we'll all excited to see the outcome of that.

[AlbertMarashi]

#![feature(string_deref_patterns)] doesn't seem to work with rust-analyzer. Any workarounds?

[Nadrieril]

This is still pretty experimental. To make it work, rust-analyzer would have to update their type-checker to recognize these new patterns. It feels premature at this point of the development of the feature.