numpy scalars vs unsigned types

[inducer]

This is a follow-up to #24 of sorts, in which I've

• sadly, rediscovered what prompted that PR, and
• dug a bit deeper into the problem.

The following diff (against 90d2445, for specificity) reproduces what I'm up against within the context of the nanobind test suite:

diff --git a/tests/test_functions.cpp b/tests/test_functions.cpp
index c545656..f11da84 100644
--- a/tests/test_functions.cpp
+++ b/tests/test_functions.cpp
@@ -125,4 +125,7 @@ NB_MODULE(test_functions_ext, m) {
             result[k] = v;
         return result;
     });
+
+    // Function taking an unsigned type, for numpy compatbiility testing
+    m.def("test_11", [](unsigned x) -> unsigned { return x; });
 }
diff --git a/tests/test_functions.py b/tests/test_functions.py
index 8788fe2..3f535ef 100644
--- a/tests/test_functions.py
+++ b/tests/test_functions.py
@@ -185,3 +185,6 @@ def test21_numpy_overloads():
 
     assert t.test_05(np.int32(0)) == 1
     assert t.test_05(np.float64(0.0)) == 2
+
+    # a function taking an unsigned type
+    assert t.test_11(np.int32(17)) == 17

Running the tests ends up with

Traceback (most recent call last):
  File "/home/andreas/pack/nanobind/build/tests/test_functions.py", line 190, in test21_numpy_overloads
    assert t.test_11(np.int32(17)) == 17
TypeError: test_11(): incompatible function arguments. The following argument types are supported:
    1. test_11(arg: int, /) -> int

Invoked with types: numpy.int32

I don't quite know what to make of this. PyLong_AsUnsignedLong is less happy with numpy.int32 than PyLong_AsLong:

https://github.com/wjakob/nanobind/blob/90d24455315f5c546a9ee80e4183e64ef68ad4e7/include/nanobind/nb_cast.h#L81-L82=

This is borne out by the CPython source, where PyLong_AsLongAndOverflow calls _PyNumber_Index:

https://github.com/python/cpython/blob/440332072706c5e422e6c54a2ec0ebb88e09c85c/Objects/longobject.c#L471=

but PyLong_AsUnsignedLong does not:

https://github.com/python/cpython/blob/440332072706c5e422e6c54a2ec0ebb88e09c85c/Objects/longobject.c#L620=

While that seems weirdly inconsistent, the behavior is as documented:

https://docs.python.org/3/c-api/long.html?highlight=pylong_asunsignedlong#c.PyLong_AsUnsignedLong

in that PyLong_AsUnsignedLong is not documented to use __index__ (but, strangely, PyLong_AsUnsignedLongMask, IMO mostly useless for our purpose, is). Possibly, the CPython folks would be receptive to fixing this. But even so, it would be a while before an upstream fix, which might be enough to warrant a workaround in nanobind.

@wjakob I'd be grateful for your take on this.

Edit: To be clear, as far as workarounds go, I am not lobbying for a revival of #24. I agree that that is too blunt an instrument. But something lighter-weight, such as attempting to cast unsigned types via (signed) long long upon failure, seems somewhat reasonable to me.

[wjakob]

This is now fixed on master.

[inducer]

Thanks!

[EdoPro98]

I am currently facing a similar issue involving numpy.uint32.

My C++ class signature is described in the following wrapping code:

.def("randGaussian", static_cast<vector<double> (SiPMRandom::*)(const double, const double, const uint32_t)>(&SiPMRandom::randGaussian))

And i call it in python with the following code:

t = rng.randGaussian(25.0, 0.01, npe[i])

Where npe[i] is a np.array with dtype=np.uint32.

I get the following error even thou the signature and the arguments passed are correct to me (here I am calling the second overload)

TypeError: randGaussian(): incompatible function arguments. The following argument types are supported:
    1. randGaussian(self, arg0: float, arg1: float, /) -> float
    2. randGaussian(self, arg0: float, arg1: float, arg2: int, /) -> list[float]

Invoked with types: SiPMRandom, float, float, numpy.uint32

[wjakob]

Can you try whether this is an issue with the latest master version of nanobind? The fix discussed earlier on this thread isn't available in the PyPI version yet.

[EdoPro98]

Thank! The version in master solved the issue.

[wjakob]

Thanks for confirming!