Default conversion pathway from python types to C++ functions taking abstract class const. refs using nanobind

[bilmes]

I have a C++ library with a class hierarchy one segment of which has something like this:

Base -> Child

where Base is an abstract C++ class. In fact, there are many children of Base, Child1, Child2, etc. but this is not super pertinent to my current question. There are many functions in the library that take a const Base& b argument which are called with children types and that use virtual function dispatch, and this all works fine on the C++ side of things. I would like to export all of these functions to python using nanobind as well as all of the children classes so that these functions can be called with an instance of an exported child on the python side. This all currently works fine with nanobind.

On the other hand, it also makes sense to call these functions with an const std::set<int>& type argument. Each of the children in fact have constructors with const std::set<int>& arguments. On the python side, I'd like to be able to call the functions directly with python sets of ints, i.e., things like {0,1,2,3,4}. Currently, this can be done by first explicitly creating one of the children and then calling the function. This also works fine, but what I would like to do is avoid forcing the python programmer to do the explicit conversion to a child first before calling one of the functions that takes a const Base&b argument directly. Rather, I'd like to have a default conversion pathway so that the functions taking a const Base&b can be called directly with a python set, even if it is only a syntactic ability (i.e., I don't mind a conversion going on to one of the children so-designated behind the scenes).

Obviously, I can't export an nb::init<const std::set<int>&>() method for Base since Base is abstract. I can also create a GrandChild class which is a child of Child, and use a nanobind::detail::type_caster to automatically convert from a nb::set (i.e., python set) to a std::set<int>, but this conversion does not happen for functions with arguments of type const Base&b. The conversion does work for functions with arguments of type const GrandChild& gc but that would mean on the C++ side having a version of every function that takes a const Base&b also have an overloaded function that takes corresponding arguments of type const GrandChild& gc (as there are many of these functions, this is quite undesirable and harder to maintain).

Hence, my question is if in nanobind there is some way to use all of the functions that take a const Base&b argument have python sets automatically converted using some default conversion pathway (i.e., using one of the Child classes which can be designated and hard-coded but done implicitly on the python side)? I've looked at trampolines docs but that does not seem to be the right approach, but then I'm not sure.

Also note that this is possible to do strictly on the C++ side using C++'s "universal reference" or "forwarding reference" mechanism, but that I do not think helps on the python side with the above, and it would mean doing this for every function in question.

Here is some simple example code that concretely demonstrates the question:

#include <nanobind/nanobind.h>
#include <nanobind/stl/set.h>

#include <iostream>
#include <set>

namespace nb = nanobind;

// Abstract base class
class Base {
 public:
  virtual ~Base() = default;
  virtual void foo() const = 0;
};

// This is one of the children. There are many of them.
class Child : public Base {
 public:
  Child() {}
  Child(const std::set<int>& s) : s1_(s) {}
  void foo() const override {
    std::cout << "Child::foo() with set of size " << s1_.size() << std::endl;
  }
  std::set<int> s1_;
};

// Here is the grandchild used with the type caster idea.
class GrandChild : public Child {
 public:
  using Child::Child;
  void foo() const override {
    std::cout << "GrandChild::foo() with set of size " << s1_.size()
              << std::endl;
  }
};

// This is an example function that I'd like to call for all children.
void processBase(const Base& b) { b.foo(); }

// This is a function that takes a child directly.
void processChild(const Child& c) { c.foo(); }

// And here is one that takes a grandchild directly.
void processGrandChild(const GrandChild& gc) { gc.foo(); }

// Here is a custom type caster for sets <-> grand child.
namespace nanobind {
namespace detail {
template <>
struct type_caster<GrandChild> {
 public:
  NB_TYPE_CASTER(GrandChild, const_name("gc_set"));

  // Conversion from Python to C++, old load.
  bool from_python(handle src, uint8_t flags, cleanup_list* cleanup) noexcept {
    if (!nb::isinstance<nb::set>(src)) {
      return false;
    }
    try {
      auto py_set = nb::set(src);
      for (const auto element : py_set) {
        value.s1_.insert(nb::cast<int>(element));
      }
      return true;
    } catch (...) {
      PyErr_Clear();
      return false;
    }
  }

  // Conversion from C++ to Python, old cast.
  static handle from_cpp(const std::set<int>& src, rv_policy /* policy */,
                         cleanup_list* /* cleanup */) noexcept {
    try {
      nb::set py_set;
      for (const auto element : src) {
        py_set.add(element);
      }
      return py_set.release();
    } catch (...) {
      PyErr_Format(PyExc_RuntimeError, "Failed to convert set to Python set");
      return nb::handle();
    }
  }
};
}  // namespace detail
}  // namespace nanobind

// Here at last is the actual module definition.
NB_MODULE(my_ext, m) {
  nb::class_<Base>(m, "Base")
      // .def(nb::init<const std::set<int>&>()) // can't uncomment since Base is abstract.
      .def("foo", &Base::foo);

  nb::class_<Child, Base>(m, "Child")
      .def(nb::init<const std::set<int>&>())
      .def("foo", &Child::foo);

  m.def("processBase", &processBase);
  m.def("processChild", &processChild);
  m.def("processGrandChild", &processGrandChild);
}

and here is some python code demonstrating the problem:

import my_ext

print("--- creating a child with a python set")
child = my_ext.Child({1,2,3}) # <-- this works fine.

print("--- processing child with processBase")
my_ext.processBase(child) # <-- this also works fine.
print("--- processing child with processChild")
my_ext.processChild(child) # <-- this too also works fine.

print("--- processing python set with processGrandChild")
my_ext.processGrandChild({1,2,3,4}) # <-- this works fine thanks to the type_caster.

print("--- processing python set indirectly via Child with processChild")
my_ext.processChild(my_ext.Child({1,2,3,4,5})) # <-- this works fine.

print("--- processing python set with processChild")
# my_ext.processChild({1,2,3,4,5}) # <-- this will fail.

print("--- processing python set indirectly via Child with processBase")
my_ext.processBase(my_ext.Child({1,2,3,4,5,6})) # <-- this also works fine.

print("--- processing python set with processBase")
# my_ext.processBase({1,2,3,4,5,6}) # <-- this will fail too.

Thanks very much for any ideas!

[ThoreWietzke]

If I understand your problem correctly, you want to implicitly cast a python set to a specific Child, when you are calling processBase or processChild? Why not using function overloading?

[bilmes]

Yes, implicitly cast a python set to a specific predesignated child for the purposes of calling functions that take an abstract const Base& argument. The reason for not using function overloading on the C++ side is that it would require many hundreds of new such functions to be created on the C++ side which is undesirable and harder to maintain. One also can't use a type caster with a Base class since Base is abstract (i.e., if Base was not abstract, a type caster would work immediately in one go).