System.IntPtr and System.UIntPtr Operators

[tannergooding]

Rationale

The .NET framework provides the System.IntPtr and System.UIntPtr types which wrap the native int IL primitive (as per ECMA-335).

These types are frequently used to represent both handles/pointers and integers whose size depend on the underlying platform.

When being used to represent the latter (platform sized integer) type, users may find a difficult time performing some basic mathematical operations on these types.

As such, these types should be modified to expose a the basic set of operators defined in ECMA-335.

Proposed API

The proposed API here only shows the members that would be new to the types.

System.IntPtr

public struct IntPtr
{
    // Unary Operators
    public static IntPtr operator  +(IntPtr value);
    public static IntPtr operator  -(IntPtr value);
    public static IntPtr operator  ~(IntPtr value);
    public static IntPtr operator ++(IntPtr value);
    public static IntPtr operator --(IntPtr value);

    // Binary Operators
    public static IntPtr operator  +(IntPtr left, IntPtr right);
    public static IntPtr operator  -(IntPtr left, IntPtr right);
    public static IntPtr operator  *(IntPtr left, IntPtr right);
    public static IntPtr operator  /(IntPtr left, IntPtr right);
    public static IntPtr operator  %(IntPtr left, IntPtr right);
    public static IntPtr operator  &(IntPtr left, IntPtr right);
    public static IntPtr operator  |(IntPtr left, IntPtr right);
    public static IntPtr operator  ^(IntPtr left, IntPtr right);
    public static IntPtr operator <<(IntPtr value, int shiftby);
    public static IntPtr operator >>(IntPtr value, int shiftby);

    // Comparison Operators
    public static bool operator < (IntPtr left, IntPtr right);
    public static bool operator > (IntPtr left, IntPtr right);
    public static bool operator <=(IntPtr left, IntPtr right);
    public static bool operator >=(IntPtr left, IntPtr right);
}

System.UIntPtr

public struct UIntPtr
{
    // Unary Operators
    public static UIntPtr operator  +(UIntPtr value);
    public static UIntPtr operator  ~(UIntPtr value);
    public static UIntPtr operator ++(UIntPtr value);
    public static UIntPtr operator --(UIntPtr value);

    // Binary Operators
    public static UIntPtr operator  +(UIntPtr left, UIntPtr right);
    public static UIntPtr operator  -(UIntPtr left, UIntPtr right);
    public static UIntPtr operator  *(UIntPtr left, UIntPtr right);
    public static UIntPtr operator  /(UIntPtr left, UIntPtr right);
    public static UIntPtr operator  %(UIntPtr left, UIntPtr right);
    public static UIntPtr operator  &(UIntPtr left, UIntPtr right);
    public static UIntPtr operator  |(UIntPtr left, UIntPtr right);
    public static UIntPtr operator  ^(UIntPtr left, UIntPtr right);
    public static UIntPtr operator <<(UIntPtr left, int shiftby);
    public static UIntPtr operator >>(UIntPtr left, int shiftby);

    // Comparison Operators
    public static bool operator < (UIntPtr left, UIntPtr right);
    public static bool operator > (UIntPtr left, UIntPtr right);
    public static bool operator <=(UIntPtr left, UIntPtr right);
    public static bool operator >=(UIntPtr left, UIntPtr right);
}

[tannergooding]

This is related to https://github.com/dotnet/corefx/issues/20256

[tannergooding]

Today we already expose:

public static unsafe bool operator ==(IntPtr value1, IntPtr value2);
public static unsafe bool operator !=(IntPtr value1, IntPtr value2);
public static unsafe IntPtr operator +(IntPtr pointer, int offset)
public static unsafe IntPtr operator -(IntPtr pointer, int offset)

It may be worth discussing whether we also want to expose (IntPtr left, int right) overloads for the proposed operators above (this at least makes some of the common cases, such as address % 16 easier to do).

• Allowing implicit conversion from int to IntPtr would also work, and be lossless.

[tannergooding]

Today we also expose "friendly named" static methods:

public static IntPtr Add(IntPtr pointer, int offset);
public static IntPtr Subtract(IntPtr pointer, int offset);

Should we also expose these for the other operators.

[tannergooding]

CC. @joperezr; since it looks like you are the area owners and you also marked the previous one as ready-for-review

[tannergooding]

As per https://github.com/dotnet/corefx/issues/20256#issuecomment-428764392 and https://github.com/dotnet/corefx/issues/20256#issuecomment-428764666. This should not be blocked by the corresponding language change that is also being discussed (but which keeeps getting delayed).

This would also allow use to simplify a bit of code in CoreCLR/CoreFX and ML.NET

[tannergooding]

CC. @GrabYourPitchforks

[GSPP]

Older C# compilers can generate code to call these operators which would then be inlined. Newer C# compiler versions could emit slim IL directly.

Any chance to change the currently broken ToString output (see my comment)?

[morganbr]

@jkotas, this sounds similar to your thoughts on nint/nuint.

The reason not to have math operators on IntPtr and UIntPtr is that they often represent a native pointer value and manipulating that value isn't safe.

Having a different type that represents platform-sized integers would avoid the safety concern (and have a better name).

[tannergooding]

The reason not to have math operators on IntPtr and UIntPtr is that they often represent a native pointer value and manipulating that value isn't safe.

I don't think this is valid at all. If you actually have a pointer you are free to do arithmetic with it (outside of void*).

System.IntPtr and System.UIntPtr:

• are primitive types as far as the runtime is concerned (they are explicitly mapped to be the native int and native unsigned int types).
• match the names of equivalent typedefs defined by the C/C++ language standards (intptr_t and uintptr_t)
• are used today as pointers, opaque handles, and native sized integers (for both interop and unsafe code, as well as in other places).
• were already updated (in net40) to have two of the operators (+/-) but don't expose the rest which would make them actually useful
• are explicitly documented to be an integer type: The IntPtr type is designed to be an integer whose size is platform-specific
• etc

[jkotas]

@jkotas, this sounds similar to your thoughts on nint/nuint.

Right, this is an alternative for the first class nint/nuint support in the language that we do not seem to be able to make progress on.

[morganbr]

@tannergooding, you've described good reasons to have a platform-native integer type with arithmetic operators. That would certainly justify moving forward with nint and nuint. Naturally, at the IL level, nint and IntPtr should be the same. You're correct that we allow a little bit of math, but that's at least somewhat sensible for pointers. Multiplication, division, mod, etc are much less likely to be meaningful.

[tannergooding]

Having a different type that represents platform-sized integers would avoid the safety concern (and have a better name).

Regardless of whether or not someone thinks that IntPtr/UIntPtr should be used to represent integers, the fact is that they are and have been used to represent just that (among other things) since the the type was first created, so the ship of having a "separate type" has long since sailed (IMO) and introducing a new type just means that people have to go through additional steps to work with their existing signatures and code.

While having the language support this directly would be ideal, there has not been much progress made on that front. Supporting this via an API change would improve a lot of code in the framework and elsewhere, while still not being a blocker for the language; As per @jaredpar https://github.com/dotnet/corefx/issues/20256#issuecomment-428765687, this should not be a blocker unless they wanted to take a breaking change (since it would make such a break more likely to be impactful). The language would still be free to support this via partial erasure (in which case they would have a new keyword, nint, which would be emitted as a native int with an attribute to differentiate it from the IntPtr case).

[tannergooding]

Multiplication, division, mod, etc are much less likely to be meaningful.

This isn't true. Detecting the alignment of a pointer can be very important for performance oriented code. Many vectorized algorithms will perform at most 2 unaligned operations (in order to become aligned) and then operate on the rest of the data as aligned, in order to achieve "optimal" performance.

We are already doing the above in ML.NET and are having to work around the fact that you can't easily do address % 16. Many other operations, such as indexing the correct element, also rely on doing multiplication, shifting, etc (that is the whole purpose of things like the LEA instruction and the scale, index, base addressing option).

[tannergooding]

You can also find scattering of using nint = System.Int64; throughout CoreCLR, which prevents that code from being properly shared with CoreFX or elsewhere (due to most of those binaries not being cross-compiled for 32-bit and 64-bit specific DLLs). Many of these places are doing this to do arithmetic on pointers, again to work around the fact that you can't do it with System.IntPtr today.

[crowo]

I can confirm JIT doesnt inline casting operators e.g. (void*)ptr. im fed up c# blocking my way when working with native int's and producing inefficient code. unsafe code is usually used in high performance scenarios and its not good. im trying to use byte* void* or void** instead and that intptr comes in the way somehow and cant avoid it. If you ease working with native ints and optimize produced code it would deserve a major version such as C# 10.0 :)

[AndyAyersMS]

@crowo can you share an example? If there are JIT issues blocking you I'd like to make sure we know about them.

[tannergooding]

Closing in favor of the language feature: dotnet/csharplang#435