multiple inheritance with nanobind on the python side

[kaschau]

I just want to be certain about what aspect of multiple inheritance is not part of nanobind (vs pybind11), is it on the python side? As in, I cannot define a python class as:

class myClass(aPythonClass, aBoundCppClass):
    ...

where aBoundCppClass is as the name suggests? Is this correct?

[wjakob]

This is correct. It will currently fail with the exception

RuntimeError: nb_type_init(): invalid number of bases!

It is possible that multiple inheritance on the Python side could be handled when the underlying class hierarchy doesn't mix in more than one C++-based class. I have not though too much about this special case since none of my own projects need it. I am open to PRs in this regard if's just a matter of relaxing that condition without adding further complexity/performance cost to the project.

[WillAyd]

Just wanted to add another consideration point to this. I am trying to implement a pandas extension array using nanobind, which requires third parties to subclass their own ABC to properly register an extension. Getting rid of the base class "check" in nanobind hacks things together in the simple case:

from my_ext import ArrayClass # assume this is from a nanobind compiled extension
from pandas.api.extensions import ExtensionArray

class NanobindArray(ArrayClass, ExtensionArray):
    ...

But the limitation I am running into is that the pandas extension interface will apply an algorithm to my ArrayClass and in some cases expect back the identity or same class. With the above pattern I would be doing:

>>> arr = NanobindArray()
>>> res = pd.some_algorithm(arr)
>>> type(res)
ArrayClass  # ideally need back an ExtensionArray

Though I am guessing there is no simple way to express this type of multiple inheritance from a mix of nanobind/Python classes in the extension itself

[WillAyd]

Cool I'll give that a look. Here is a prior art in Cython to what I am trying to accomplish:

cdef class BaseClass:

    def new(self):
        return BaseClass.__new__(type(self))

You can cythonize that and compile to a quick shared lib on linux with:

$ cythonize foo.pyx
$ gcc -g3 -Wall -Werror -std=c17 -shared -fPIC -I$(python -c "from sysconfig import get_paths as gp; print(gp()['include'])") foo.c -o foo.so

And here the is the behavior below - I get the functionality from the cdef extension but maintain the instance of the subclass:

from foo import BaseClass

class SomeMixin:
    ...

class Child(BaseClass, SomeMixin):
    ...

assert type(Child()) == type(Child().new())
assert isinstance(Child().new(), SomeMixin)

[wjakob]

I can't say much about Cython, which uses a rather different implementation and object model. So the main question of interest would be if this branch simply works.

[WillAyd]

Fair enough. Happy to give this feedback here or wherever else is more appropriate for the branch, but here is a small reproducer in nanobind:

#include <nanobind/nanobind.h>

namespace nb = nanobind;

class Base {
public:
  auto New() const -> Base {
    return Base();
  }
};


NB_MODULE(my_ext, m) {
  nb::class_<Base>(m, "Base")
    .def(nb::init<>())
    .def("new", &Base::New);
}

Using that from Python:

from my_ext import Base

class SomeMixin:
    ...

class Child(Base, SomeMixin):
    ...

assert type(Child()) == type(Child().new())  # AssertionError

So the multiple inheritance works, but I couldn't figure out a way to ensure that operations on an instance of Child don't end up returning a Base

[wjakob]

That's not obvious but also not really the fault of the nanobind API change being discussed here.
Two ideas:

• You could use the trampoline mechanism in Base and then override internal object creation methods in Child
• You could store the Python type object of Child somewhere (as an nb::object at the top level, or as an instance variable of Base, which Child could set when it calls the Base constructor)

In any case, returning a Base object doesn't really make sense anymore. All of the functions now need to construct a Python object, because that's what you wanted. One option would be to have a kind of copy-constructor in Child that takes a Base instance as input and then creates the Python version of that.

[WillAyd]

Cool thanks for the ideas - I have some research to do. I think the point in returning a Base object from the New method is that other C++ methods could call that and not have to go through the Python runtime. So you could potentially chain together methods in your extension but defer conversion to the subclass until you actually do want to go back into Python (the mix of Python/extension function calling is similar to what Cython offers with cpdef).

Anyway you are correct we've strayed a bit from the OP of this discussion. I need to research and formalize better any request I might be looking for, if there even ultimately is one. Thanks as always for your input and this great library