Making Declaration of T* where T : struct Possible

[whoisj]

It should be possible to create unsafe pointers to generic types so long as said types are value types (ala where T : struct). I find myself today writting code like the following for every type of value type I want to support (and there are many)

        public unsafe void Write(ulong value)
        {
            lock (_buffer)
            {
                fixed (byte* b = _buffer)
                {
                    *((ulong*)b) = value;
                }
                Write(_buffer, 0, sizeof(ulong));
            }
        }

Meaning I have a dozen methods or more, when a single method should be sufficient. If T* where possible when T : struct I could construct the following method and reduce the amount of boilerplate in my code significantly.

        public unsafe void Write(T value)
            where T : blittable struct
        {
            lock (_buffer)
            {
                fixed (byte* b = _buffer)
                {
                    *((T*)b) = value;
                }
                Write(_buffer, 0, sizeof(T));
            }
        }

Support for sizeof(T) is #3208

[HaloFour]

A struct isn't necessarily blittable or of a predictable size. If the struct contains a reference or native integer it will have different size at runtime. C# also doesn't specify a default pack leaving it up to the runtime. Both sizeof(T) and all pointer arithmetic must be able to calculate byte offsets at compile time and embed those offsets into the IL.

[HaloFour]

Some of the same concerns were covered in #126.

[whoisj]

Interesting. Would be nice if there were a way to signal that a struct is friendly to unsafe usage and then use that signal in generics for this kinds of issues listed above.

[axel-habermaier]

I agree that generic pointers and sizeof would definitely be very nice to have in certain situations. Note that a generic sizeof is supported by the runtime already, it's just not exposed by C#.

@HaloFour: True, but the compiler knows these things at compile-time already. So it can always check whether the struct can be used in such a context. On the other hand, the runtime would also have to support this for reflection-based scenarios. I don't know if it does. Even if not, the compiler could simply generate the required code for each invocation with a specific type, similar to how C++'s templates work; F# does that too in certain cases. I'm not sure if that's a desirable approach however.

[HaloFour]

@axel-habermaier

The compiler is capable of figuring out this specific information if you're using a specific struct, yes. But when using generics it's not possible for the compiler to determine if generic type parameter is safe at compile time, let alone calculate the byte offsets that must be emitted directly into the IL of the generic method body.

[axel-habermaier]

@HaloFour: Ah, now I see the problem. I originally assumed that the proposal added an additional generic constraint to indicate that sizeof(T) and T* must be valid for the actual struct substituted for T. Since it doesn't, you're absolutely right. As far as I remember, there were several other proposals that suggested to add such a constraint.

[whoisj]

@axel-habermaier, @HaloFour an added constraint would be ideal here. I'll happily update the suggestion. What would be a good name for the constraint? Perhaps something like where T: ISafeUnsafe or where T : unsafe struct which implies a compiler verified contract that only non-managed fields are members of the struct?

[axel-habermaier]

@whoisj: I don't like where T : ISafeUnsafe, as that looks like the type is required to implement some interface. where T : unsafe struct is better, but it's not (necessarily) the struct that is unsafe, but the way it is used. In another thread (here on GitHub or back on CodePlex) someone suggested where T : blittable, which from the technical perspective might be the best choice; but it might not be the most discoverable one. Also: Would this feature require CLR support or is such a constraint already supported? Could we follow the CLR's naming of the constraint, provided it exists?

[HaloFour]

You would need some kind of new constraint, as well as some mechanism in the CLR to actually care about and enforce that the struct is predictably blittable. But that doesn't solve the issue of indexing where the generated IL for the method needs to know the exact byte offsets and there is no way that the compiler can know the size of any given generic struct at compile time.

[whoisj]

@axel-habermaier I honestly do not know if the CLR understand this concept specifically, though one would assume it would have to otherwise I could grab an unsafe pointer to a struct with managed types in it via IL. I like the where T : blittable. I come from a graphic background and while I usually think of blitting as moving pixels, it really is a perfect match. For now, I'll update the suggestion with it: where T: blittable struct.

[HaloFour]

@whoisj

I honestly do not know if the CLR understand this concept specifically, though one would assume it would have to otherwise I could grab an unsafe pointer to a struct with managed types in it via IL.

It's only the C# compiler that attempts to enforce this. The CLR doesn't know and doesn't care. It will happily let you index into an arbitrary offset, even out of bounds or with unpredictable types. The verifier stops bothering attempting to ensure correctness as soon as you break out the pointers.

[axel-habermaier]

@HaloFour: When would the exact byte offsets be required? You can't access any fields of the struct in a generic context.

[HaloFour]

@axel-habermaier

Running through some tests it actually looks like the compiler is willing to break out the sizeof IL instruction in many cases when using a custom value type which it then uses to calculate the offset at run time, which might make that issue moot. Although it's arguable whether the compiler should permit the use of arbitrary value types in such contexts if the size may change depending on the platform, etc.

[whoisj]

The verifier stops bothering attempting to ensure correctness as soon as you break out the pointers.

Yay, that's fantastic and good to know when I'm pushing for performance. 🎈 Do wish the compiler wouldn't get in the way here. Once I've stated unsafe I understand the implications, now stop holding my hand! 😏

Although it's arguable whether the compiler should permit the use of arbitrary value types in such contexts if the size may change depending on the platform, etc.

"Arguable"? If the developer has explicitly said unsafe let them run without the training wheels. 💀

[HaloFour]

Hence, "arguable". Clearly the language designers didn't think that unsafe meant carte blanche otherwise you'd probably already be able to do these things. I assume that they would have expected you to jump from "unsafe" to "unmanaged" in those cases.

[whoisj]

The problem with "unmanaged" is it requires marshalling and back again, otherwise I would. 😞

I do understand the implications, and reasoning. It's just unfortunate that I often find myself thinking: we'd be a lot more productive in C#, too bad it's slow for this kind of stuff. With more lax unsafe restrictions, it would less often be trade off of developer productivity vs program performance.

[gafter]

We are now taking language feature discussion on https://github.com/dotnet/csharplang for C# specific issues, https://github.com/dotnet/vblang for VB-specific features, and https://github.com/dotnet/csharplang for features that affect both languages.