Introduce Pointee and DynSized

[nox]

Those two new language traits are at the intersection of #1861, #2580, #2594, and this RFC's raison d'être is solely to try to get some progress for those 3 RFCs all at once by specifying their common core part.

Rendered

[kennytm]

since ?DynSized is proposed here, you may want to address the concern in the comments of #2255 (#2310)...

[nox]

Thanks for the pointer, I missed this one and will now look into it.

[nox]

I probably missed a lot of small concerns, but here is a first stream of thought with some direct pointers to past comments.

---

So the first concern mentioned in #2255 is that opt-out bounds such as ?Sized are confusing to users, so adding ?DynSized would add confusion. This concern was first raised by @withoutboats in rust-lang/rust#46108 (comment) back in December 2017. In January 2018, @arielb1 argued in #2255 (comment) that this may not be as much as a problem as initially thought because it makes sense given the hierarchical relation Sized: DynSized.

My own framing on that is that going from T to T: ?Sized to T: DynSized is like peeling an onion, removing a layer of implicit bounds with each increasingly bigger opt-out bound.

@mikeyhew then adds in #2255 (comment) that they would rather allow the user to write normal bounds of supertraits of Sized and make the compiler remove implicit bounds of the corresponding subtraits. I.e. writing T: Pointee would imply T: ?DynSized, and T: DynSized would imply T: ?Sized. @withoutboats later partially agrees to that in #2255 (comment).

I agree with @mikeyhew that this is something that could be done, but my understanding is that it is mechanically equivalent to allowing T: ?DynSized and thus could be a major change introduced in a later Rust edition where we would allow code using T: ?Sized (or T: ?DynSized) to be rewritten to T: DynSized (or T: Pointee).

In April 2018, @nikomatsakis writes in rust-lang/rust#43467 (comment) that he doesn't think that T: ?Sized should imply T: DynSized and instead there should just be lints or runtime checks against using size_of_val with a !DynSized type, and that this kind of concern around size_of_val should not block extern types from landing.

In my opinion, extern types still didn't land exactly because we still don't have a DynSized concept to prevent extern types from being used with size_of_val, because it makes things very vaguely defined and this doesn't motivate anyone from pushing extern types to completion.

Finally in September 2019, @Ixrec in #2255 (comment) follows-up that ?Move has been abandoned and absolutely nobody else ever suggested a different opt-out trait with a ?Trait syntax.

In my opinion, we can confidently say that an additional ?DynSized thing wouldn't confuse users more than ?Sized may have confused them.

---

The second concern is raised by @eddyb in #2255 (comment) and is about whether we should plan for !Pointee types later down the road. If so, should T: Pointee be implied all the time, meaning that !Pointee types are then only allowed in T: ?Pointee code? @eddyb later in #2255 (comment) suggests that !Pointee types could instead be encoded as Pointee<Meta = !>.

---

@kennytm in #2255 also mentions that introducing ?DynSized means downstream crates will have to reconsider all their
?Sized bounds to check if they could benefit from being ?DynSized.

I don't think extending the language to support some kinds of non-dynamically-sized types should be blocked on downstream crates not supporting them from the get-go.

---

To conclude, reading through the comments of #2255 reaffirms my belief that this RFC defines the most viable core that we should go for as a first step for extern types and pointer metadata APIs, given that the gist of the concerns from the language team is the ergonomic cost of ?DynSized and we can always revisit that in a later language edition where we would make T: DynSized imply T: ?Sized as opposed to T: ?Sized implying T: DynSized.

[withoutboats]

Finally in September 2019, @Ixrec in #2255 (comment) follows-up that ?Move has been abandoned and absolutely nobody else ever suggested a different opt-out trait with a ?Trait syntax.

This isn't the case. People suggest additional question mark types with regularity. Some people want to revisit Move, some people want a ?Trait to add linear types, its even been proposed (though I think only offline) to add a ?Trait governing whether a type exists in 1 memory space or more, to better support wasm reference types. I think all of these would be a poor choice, for many of the same reasons as ?DynSized.

People want to use the ?Trait feature to add new exceptions to the requirements that Rust currently assumes every type adheres to. I don't think we should ever do this, because going from one ?Trait to more than one is, in my opinion, remarkably expensive in terms of onboarding new users and headaches for existing users, concerns you mention but dismiss.

In my opinion, extern types still didn't land exactly because we still don't have a DynSized concept to prevent extern types from being used with size_of_val, because it makes things very vaguely defined and this doesn't motivate anyone from pushing extern types to completion.

This isn't why we haven't made progress on extern types. We haven't made progress because no one is driving the feature. I think within the lang team there is (weak) consensus that making size_of_val/align_of_val panic when passed an extern type would be an adequate solution, and I don't think the feature is blocked on having a compile-time solution.

[nox]

This isn't the case. People suggest additional question mark types with regularity. Some people want to revisit Move, some people want a ?Trait to add linear types, its even been proposed (though I think only offline) to add a ?Trait governing whether a type exists in 1 memory space or more, to better support wasm reference types. I think all of these would be a poor choice, for many of the same reasons as ?DynSized.

Ack, I meant that I don't see people from any Rust team pushing for those, but maybe I'm just not seeing those discussions.

People want to use the ?Trait feature to add new exceptions to the requirements that Rust currently assumes every type adheres to. I don't think we should ever do this, because going from one ?Trait to more than one is, in my opinion, remarkably expensive in terms of onboarding new users and headaches for existing users, concerns you mention but dismiss.

I didn't dismiss anything, I literally started my comment by saying that I probably missed some concerns. Given there was never more than one ?Trait thing, how can we conclude that more than one would be more painful? Some people argued in the other threads than having more than one would actually help because it would make it less of a special case. None of the two hypotheses have been tested because there was never more than one opt-out bound.

This isn't why we haven't made progress on extern types. We haven't made progress because no one is driving the feature. I think within the lang team there is (weak) consensus that making size_of_val/align_of_val panic when passed an extern type would be an adequate solution, and I don't think the feature is blocked on having a compile-time solution.

Ack. I shouldn't have mentioned size_of_val specifically but from an external point of view, it feels like no one is driving the feature because the gist of it interacts with half a dozen RFC which all have their own individual concerns without really seeing how it fits with the others, hence that RFC. As for having a compile-time solution for size_of_val, I was biased by the fact that I need a compile-time solution for "accept all thin pointers whether sized or not", which then gives us a way to compile-time reject non-sensical size_of_val uses for free.

[nox]

I'll amend the RFC to include more unresolved questions based on this discussion. I don't care if the PR ends up closed, I just feel like it would be a good thing to have a document that centralise them anyway.

[burdges]

I doubt this works but.. Is there any value in distinguishing between raw pointers and borrowed references here? In other words, could &T simply not exist whenever size_of_val::<T> makes no sense? We'd still have *const T though so we'd then need roughly #1403 so that *const T could be wrapped into some reference type for safe borrowing.

[nox]

I doubt this works but.. Is there any value in distinguishing between raw pointers and borrowed references here? In other words, could &T simply not exist whenever size_of_val::<T> makes no sense? We'd still have *const T though so we'd then need roughly #1403 so that *const T could be wrapped into some reference type for safe borrowing.

I don't understand why that would be useful, extern types definitely let you get a &T but size_of_val makes no sense for them.

[burdges]

Apologies for the derail. I'm asking if &T should make sense when T is an extern type.

[nox]

Apologies for the derail. I'm asking if &T should make sense when T is an extern type.

Oh I see. My own use case for extern types is to improve bindings for things such as the Carbon APIs on macOS, which are all defined as typedef struct __CFFoo CFFoo; on the C side. Being able to represent that as extern { type CFFoo; } on the Rust side makes the safe wrappers as thin (no pun intended) as they could be with no need to unwrap or rewrap things constantly in the code.

[Diggsey]

Safe C-style string wrappers would also benefit from &T being allowed. (ie. where the size can only be determined by scanning for the null)

[nox]

Cc @retep998 who makes winapi and who also expressed interest for a proper support of unsized types at the type-level.

[withoutboats]

From my perspective, both extern types and DST metadata manipulation are not blocked so much as they are just not being driven anywhere. I don't think these need to intersect. I'm fully in favor of stabilizing extern types with size_of_val panicking as well as introducing a way to express as a bound that the metadata for a type is ().

[nox]

@withoutboats Could you expand on why you are ok with Pointee but not DynSized to reject size_of_val from being used with extern types? Is it just because of the additional opt-out bound? Edit: "from being used", not "from being rejected"

[nox]

From my own perspective, there are quite a few community actors that maintain FFI crates that want something like DynSized, and a majority of language team members that are strongly against it, hence my conclusion that the whole process is blocked, because it looked like ultimately no one wanted to make the jump and take a final decision. My reading seems to be wrong about this, but I nonetheless agree with the people who want proper type support rather than runtime aborts and ad-hoc lints.

[withoutboats]

@nox Yes, I think the costs of adding a ?Trait are very high, as I said, and I don't think converting a panic in size_of_val on an extern type to a compile time error is a strong enough justification.

As an alternative design for the pointee type, I would make the Meta associated type nameable viable Sized trait, which I realize does not make sense exactly, but I think is more convenient for users. All types with no metadata would be T: ?Sized<Meta = ()> for example, whereas all trait objects would be T: ?Sized<Meta = VTable> and slices/strings would be T: ?Sized<Meta = usize>.

[nox]

If there is no such additional ?Trait at all, doesn't that set in stone that there will never be any custom dynamically sized types? How do you retrofit a trait to define dynamically sized types under Sized once extern types with an adhoc lint or ?Sized<Meta = …> are shipped?

[withoutboats]

I'll try to reiterate the objection to DynSized.

I realize that users who are deeply invested in writing FFI bindings would like their bindings to be as typesafe as possible, and would like the compiler to check that users do not pass extern types to APIs which cannot rationally accept them, but this has to be balanced against the cost to all other users. This cost is twofold: the burden added to writing and maintaining correct Rust libraries, and the burden added to learning Rust.

By adding ?DynSized, every API which uses a ?Sized bound currently needs to review whether it ought to be ?DynSized bound. When I reviewed std in 2018, every API accept Box could have taken ?DynSized, as I recall, because Box needs the layout of the type to call its destructor. Forcing everyone to consider this question would very be disruptive. But the disruption is not one-and-done, because now every new ?Sized bound added in the future needs to decide whether it can take extern types or not.

I also think adding a ?Trait may not be backward compatible because of the assumptions made about which types implement Sized or don't today. DynSized may be excepted from this issue because of its relationship with Sized, but I'm not sure.

Similarly, the ?Sized feature is very confusing to new users, and I think ?DynSized would compound that. Adding ?Sized would mean "my type is maybe a DST" whereas adding ?DynSized would mean "my type is maybe an extern type. This is not intuitive if you don't already understand the whole system. Error messages about Sized would in some cases probably now say DynSized, complecting the existing issues they present to users. Not understanding how Sized works has been the biggest user confusion with the trait system in my experience.

I just don't think compile time checking that no one tries to get the layout of an extern type is valuable enough to justify these burdens.

---

If there is no such additional ?Trait at all, doesn't that set in stone that there will never be any custom dynamically sized types? How do you retrofit a trait to define dynamically sized types under Sized once extern types with an adhoc lint or ?Sized<Meta = …> are shipped?

The syntax to define the Meta type could be special, impl !Sized for MyType, maybe, for example, or we could indeed use a trait like Pointee. The point is not to introduce this additional burdens I described above.

[nox]

I realize that users who are deeply invested in writing FFI bindings would like their bindings to be as typesafe as possible, and would like the compiler to check that users do not pass extern types to APIs which cannot rationally accept them, but this has to be balanced against the cost to all other users.

Thanks for that.

This cost is twofold: the burden added to writing and maintaining correct Rust libraries, and the burden added to learning Rust.

Are you completely closed to the idea that maybe having more than one opt-out bound would actually help understand how opt-out bounds work?

By adding ?DynSized, every API which uses a ?Sized bound currently needs to review whether it ought to be ?DynSized bound. When I reviewed std in 2018, every API accept Box could have taken ?DynSized, as I recall, because Box needs the layout of the type to call its destructor. Forcing everyone to consider this question would very be disruptive. But the disruption is not one-and-done, because now every new ?Sized bound added in the future needs to decide whether it can take extern types or not.

I don't see this as an argument against DynSized at all: it boils down to crates that predate extern types may not support extern types where they could, but how is that a problem given extern types didn't exist when these crates were created?

To me that's like arguing against no_std because some crates could have been no_std before no_std was a thing.

Similarly, the ?Sized feature is very confusing to new users, and I think ?DynSized would compound that. Adding ?Sized would mean "my type is maybe a DST" whereas adding ?DynSized would mean "my type is maybe an extern type. This is not intuitive if you don't already understand the whole system. Error messages about Sized would in some cases probably now say DynSized, complecting the existing issues they present to users. Not understanding how Sized works has been the biggest user confusion with the trait system in my experience.

Fair, I personally don't think 1 or 2 of those change the complexity of the system but that's a matter of opinion anyway.

I just don't think compile time checking that no one tries to get the layout of an extern type is valuable enough to justify these burdens.

Also fair, but I really feel like DynSized strengthen all those RFCs about pointer metadata, extern types and potential custom dynamically sized types all at once, which sounds great to me. Things that look like a solid foundation for multiple concepts at once are good in general.

The syntax to define the Meta type could be special, impl !Sized for MyType, maybe, for example, or we could indeed use a trait like Pointee. The point is not to introduce this additional burdens I described above.

To me, that seems like way more of a burden than ?Sized/?DinSized on the education front. "Yeah so you are opting-out from a trait (Sized), but at the same time you bound on something like an associated type to the supertrait of Sized". See what I mean?

[withoutboats]

Are you completely closed to the idea that maybe having more than one opt-out bound would actually help understand how opt-out bounds work?

To me, that seems like way more of a burden than ?Sized/?DinSized on the education front.

I think these comments represent a disagreement that I often have with contributors about what it is to learn how to use Rust. Users approaching Rust have a very fuzzy, informal understanding of the type system, based on limited and incomplete information filled in with assumptions or inferences which in may cases, if investigated thoroughly, would reveal themselves as contradictory and false. As they learn Rust, they slowly correct these misunderstandings based on the response of the compiler and the supporting documentation they read as they introduce errors into their code.

Often, user perspective on what change would be clear, helpful, confusing, etc is based on a perspective that makes a lot of sense if you already understand the entire system formally, but in my opinion is not at all the case if you do not understand what's going on.

Adding more exceptions to the rules, more points of decision making presented to the user early on, more distinct ways that errors can be surfaced - these all are examples of things that are very confusing for learning users. On the other hand, making the one exceptional case more exceptional is usually not a problem, because learning users have been signalled to ignore it as exceptional when trying to understand things and users with a formal understanding can just enumerate the exceptions.

[nox]

Thanks for expanding.

Apart from ?DynSized, is there anything else that feels like a blocker to you? Hear me out: RFCs are sometimes for experimental stuff and have no guarantee to ever be stabilised, right? Let's assume that the code for Pointee/DynSized exist, would it be useful to experiment around the idea in nightly? Maybe with a separate feature that encodes that stuff as positive bounds of the supertrait (e.g. T: DynSized instead of T: ?Sized and T: Pointee instead of T: ?DynSized)? Obviously, someone would need to write that code, and that's a hard task, but in theory, that would be interesting, right?

[nox]

Wouldn't it be a good idea to be able to replace that bound list itself with another auto trait in the future (e.g. at least leave that as an option/implementation detail)? e.g.

Auto traits cannot have supertraits so I encoded things that way, but that was my first idea too.

[fogti]

Ah ok, but could it work if it is not an auto trait? I don't think it would really need to, especially because currently, there are only roughly three or four cases:

trait PointeeMeta: 'static + Copy + Eq + Ord + Send + Sync + Unpin + … {}
trait Pointee {
    type Meta: PointeeMeta;
}

/// no metadata at all
impl PointeeMeta for () {}

/// metadata for slices
impl PointeeMeta for usize {}

/// metadata for & dyn trait's
/// Basic Variant
impl PointeeMeta for VTable {}
/// or maybe multiple VTables (e.g. &dyn A + B, where neither A nor B are auto traits)
/// X=number of VTables, would need const generics (https://github.com/rust-lang/rust/issues/44580)
/// but could also be compiler- or macro-generated
impl<X: usize> PointeeMeta for VTable<X> {}

[withoutboats]

Apart from ?DynSized, is there anything else that feels like a blocker to you? Hear me out: RFCs are sometimes for experimental stuff and have no guarantee to ever be stabilised, right? Let's assume that the code for Pointee/DynSized exist, would it be useful to experiment around the idea in nightly? Maybe with a separate feature that encodes that stuff as positive bounds of the supertrait (e.g. T: DynSized instead of T: ?Sized and T: Pointee instead of T: ?DynSized)? Obviously, someone would need to write that code, and that's a hard task, but in theory, that would be interesting, right?

Yea, I think pushing on the space that this and 2580 are in would be interesting and valuable. The big problem then would be bandwidth.

[nox]

Good to know! Totally understand for bandwidth, especially with pandemic.

---

But anyway… sorry if I missed this concern in one of the dozen related RFCs and threads, but I just realised something: if we implement what I just proposed, extern types end up not even DynSized, right? And a definition such as trait Foo {} must continue to imply Self: DynSized otherwise existing code using size_of_val starts breaking. So with my scheme, an extern type cannot implement say, Debug, because we can't change trait Debug to mean trait Debug: ?DynSized. So in the end, none of the fancy extern types that I wish to use directly in a neat safe wrapper can even implement some of the most basic traits. Am I correct?

[burdges]

If size_of_val::<T> panics for an extern type T then really it should refuse to compile, yes? We might theoretically then run into size_of_val::<dyn Trait> where T: Trait, except we already forbid trait objects for unsized types: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=73b52341ef8dbcf594ca2ede878d9fbc It should therefore work if size_of_val::<T> etc. just refuse to compile, but never encode the reason why behind some trait, right?

[petertodd]

Sorry if I missed someone bringing this up already. But note that Rc/Arc assume that you can calculate the size of a dyn type from a pointer due to the fact that Weak needs it to calculate the size of the allocation after the last Rc/Arc drops the value: https://github.com/rust-lang/rust/blob/94b4de0e0793c8921d30e0fb886be712d17db6e5/library/alloc/src/rc.rs#L2001 (note how technically this code creates a reference to dropped content; it should be using size_of_val_raw() and align_of_val_raw())

For example, to support the common case of null-terminated strings, I think you'd need some kind of MaybeDropped concept for types where the size of a value can be dynamically determined from the byte representation, after any indirect resources have been deallocated by Drop.

As @withoutboats points out, to fully support such types things like Box now need different, looser, type bounds than Rc/Arc - a significant amount of churn for the ecosystem to support a relatively small number of use-cases.

[nox]

@petertodd I am not sure how that relates to this RFC, whether or not it is decided to adopt something similar to the DynSized trait here, it doesn’t let you implement custom DST so being able to support Box<ExternType> is out of scope, isn’t it? It is not supported either in the current implementation of extern types because size_of_val etc don’t have any sensical value to return.

[petertodd]

@petertodd I al not sure how that relates to this RFC, whether or not it is decided to adopt something similar to the DynSized trait here, it doesn’t let you implement custom DST so not being to support Box<ExternType> is out of scope, isn’t it? It is not supported either in the current implementation of extern types because size_of_val etc don’t have any sensical value to return.

Ah! I see what you mean: Box<T> for T: !DynSized would of course remain impossible, and my point is actually about custom, metadataless, DSTs. Sorry, brainfart on my part.

[withoutboats]

I disagree and I think it would be a quite a wart to land extern types that can trivially make the program abort just passing them to size_of_val ... I don't think I would use extern types that may make my program abort if someone uses them in an unexpected way, I'll probably stick to zero-sized empty structures.

The current behavior as I recall is for extern types to return 0 when passed to size_of_val. I thought we had universal consensus that this was the worst of the three options, but perhaps I'm wrong. Making them return 0 as they do today would give you the same behavior as using a zero-sized struct, which you say would prefer to use, except that it would at least prevent you from passing them to generics that require a Sized type. Perhaps you could make the case for returning 0 instead of panicking?

[nox]

Returning 0 would be a-ok I think and I didn't know it was the current default, I guess I should have tried it at some point. Then all the rest that I mentioned to later introduce an opt-out DynSized gets downgraded to a simple "nice to have" thing in my book.

[Ericson2314]

I recently read https://matklad.github.io/2020/09/20/why-not-rust.html which was great, and give me a new argument for DynSized.

One quote was this

Rust also lacks an analog for the pimpl idiom, which means that changing a crate requires recompiling (and not just relinking) all of its reverse dependencies.

extern type really ought to be the C++-beating solution for this. by "demoting" a dependency's type to an abstract type, we are forced to always work with it boxed, but don't have to rebuild so often.

But do do that safely, we need to ensure that in demoting the type, any still-allowed program as unchanged semantics. That means that changing size_of_val from n to 0 or panicking is a total non-starter. The only safe thing to do is statically ban calling any function whose meaning would need to change, and that in turn means we need ?DynSized.

[burdges]

I'd think dyn(rust_2021) Trait etc. would address that better than extern type, no? Invoking methods upon extern types sounds messy.

[nox]

Invoking methods upon extern types sounds messy.

How so? It's just a type that is not sized, but otherwise is just the same as any other type. It's not weirder than having methods on dyn Any or usize.

[burdges]

I read pimpl as invoking extern fns defined on the type, presumably including inherent and trait methods, not just passing around the opaque extern type. There are several issues here like clarifying calling convention, but trait objects sound like one plausible goal, and the preceding comment took a turn that direction.

There are at least two approaches to pimpl trait objects, either dyn(convention) Trait that clarifies both vtable layout and calling convention, or else piercing the pimpl boundary to specify wrappers so that just dyn Trait still works, but both address the layout question.

[Ericson2314]

@burdges pimpl is sub-par because not only does it require boxing, it chooses the type of boxing for you. This is more like impl Trait, except not only is the type abstract to the downstream code, it's also abstract to downstream compilation.

[withoutboats]

extern type really ought to be the C++-beating solution for this. by "demoting" a dependency's type to an abstract type, we are forced to always work with it boxed, but don't have to rebuild so often.

Extern types (or any type that wouldn't implement DynSized) can't even be boxed using Box, because Box::drop uses size_of_val/align_of_val to get the layout of the type its deallocating. I'm very incredulous this would be how we would solve this problem, as opposed to custom DSTs that do have some way of getting their size and alignment at runtime.

[withoutboats]

That being said, if we ever want to revisit whether or not to add an additional opt-out language trait, I think we could anyway through a Rust edition: we could pretend that Rust 2018 extern types implement, say, the Contiguous trait from the custom DST RFC with aborting methods, while extern types in the next edition just don't implement that trait. Coupled with a system that implies the Contiguous trait bound everywhere in Rust 2018 and doesn't in the new edition, this would mean we could introduce truly unsized types later even if we land custom DSTs or extern types now.

I just want to note also that I'm skeptical of this working also, because all crates across all editions need to be coherent with one another. We need the answer to the question does T implement Trait to be the same regardless of where you are asking it. I think you mean for this edition to be determined per-compilation by the edition of the binary, and so some libraries will stop compiling when linked into a binary using a new edition, which might work, but like a new question mark trait this is the sort of change we are very hesitant to consider.

This is the take away that I would really want to impart on the thread: Rust has allowed users to make certain assumptions about the properties of all types, and of all Sized types. Trying to introduce types that violate one of those assumptions is extremely fraught with backward incompatibility hazards, so the lang team is not eager to try to make it work without extremely compelling motivation ("we strongly need this to be relevant in one of our core use cases" kind of motivation). Even if it can work, its extremely disruptive to the ecosystem and a lot of work for the project.

[clarfonthey]

It took me forever to understand the reasoning behind the existence of this when it has no items of its own, until I read more closely and noticed that extern types are the only things that are excluded. Would recommend making that clear in the summary for this so that it's more immediately apparent.

[carbotaniuman]

Is there a reason size_of_val can be made to panic but allocation failure can't be switched from aborting to unwinding without major changes? It feels to me that unsafe code is likely to assume that such a function would not panic...

[matthieu-m]

In the context of Is custom allocators the right abstraction?, where I proposed defining "storages" instead, the need to, generically, slice a pointer into meta-data + data-pointer and then later stitch them back together has come again.

I've been thinking quite a bit about the API I would need, and in terms of this RFC it would be something such as:

// All in core::ptr module.

trait MetaData: 'static + Copy + Eq + Ord + Hash + Send + Sync + Unpin + ... {
    fn layout(&self) -> Layout;
}

trait Pointee {
    type Meta: MetaData;
}

unsafe fn into_raw_parts<T: ?Sized + Pointee>(ptr: NonNull<T>) -> (T::Meta, NonNull<()>);

unsafe fn from_raw_parts<T: ?Sized + Pointee>(meta: T::Meta, ptr: NonNull<()>) -> NonNull<T>;

Where Pointee would be implemented automatically for all T with the following set of MetaData:

• If T: Sized: SizedMeta<T>(Invariant<T>).
• If T is a slice: SliceMeta<[T]>(usize, Invariant<T>) -- forward compatible with SliceMeta<[T][U]>(usize, usize, Invariant<T>), etc...
• If T is a trait: TraitMeta<T>(*mut (), Invariant<T>) -- forward compatible with TraitMeta<T + U>(*mut (), *mut (), Invariant<T>), etc...

(With Invariant<T> being a synonym for PhantomData<fn(T) -> T>, non-owning, non-borrowing, and invariant).

---

The functions into_raw_parts and from_raw_parts would be implemented by intrinsics.

Custom DST would require the user to create their own struct implementing MetaData, and indicate to the compiler how to implement into_raw_parts and from_raw_parts in some way.

I also think it is better to encapsulate (and hide) the exact implementation of the meta-data, and not make any guarantee on the representation, so as to keep our options open going forward.

[SimonSapin]

@matthieu-m This is almost exactly #2580 except it proposes a metadata function instead of into_raw_parts, since .cast() can already be used on (possibly-wide) pointers to extract the data address as a thin pointer.

[SNCPlay42]

I want to bring rust-lang/rust#79409 to attention here; I'm not sure how that could be resolved without something like DynSized.

[Ericson2314]

To repeat what I wrote in rust-lang/rust#43467 (comment), we now have a good argument from @SimonSapin for Sized: Thin. IMO If Size is no longer the root of the hierarchy with Pointee, there's a lot less of a ergonomic downside to also having DynSized.

Don't fight the math; hiding the essential complexity only makes more for accidental complexity.

[SimonSapin]

You’re putting words in my mouth. That the trait resolver should be able to deduce Metadata == () from T: Sized is not an argument for adding a new opt-out ?Trait, nor does it alleviate the practical concerns with those proposals.

[Ericson2314]

Yes, all I mean to say you support is T: Sized => T::Metadata = (). I follow that up with saying Sized: Metadata is the best way to present that, so it follows from human-readible definitions and isn't an ad-hoc rule, something which you may or may not agree with, but you did at least try yourself.

To me, having ? and having Size have supertraits is enough Rubicons crossed. I simply don't see any significance in having more ? traits, any more than having more than one default-feature in Cargo is significant.

[clarfonthey]

Since #2580 was merged, does that mean this RFC is redundant?

[Ericson2314]

@clarfonthey No, I think they are actually quite complementary, though by no means does the other one require this one.

[jmillikin]

Possibly of interest to folks subscribed to this issue: I've proposed an RFC to add DynSized but with a different definition, which would mean a type that can compute its own size based on a reference.

RFC 3536: Trait for !Sized thin pointers

I think this would be broadly compatible with extern type and the custom pointer metadata stuff, but is substantially smaller and directly enables native representation of header-prefixed dynamic-sized values.