Allow to set the return value policy on std::function parameters

[se7kn8]

The default rv_policy for std::function parameters is rv_policy::automatic_reference. This results in an runtime error when the following code is executed:

C++

#include <cstdio>
#include <functional>
#include <nanobind/nanobind.h>
#include <nanobind/stl/function.h>

namespace nb = nanobind;

using namespace nb::literals;

class NonCopyable {
public:
    NonCopyable() = default;
    NonCopyable(const NonCopyable&) = delete;
    NonCopyable& operator=(const NonCopyable&) = delete;

    void test() {
        std::printf("Test\n");
    }
};

NB_MODULE(nanobind_example_ext, m) {
    nb::class_<NonCopyable> c(m, "NonCopyable");
    c.def(nb::init<>());
    c.def("test", &NonCopyable::test);

    m.def("func", [](std::function<void(NonCopyable&)> func){
        NonCopyable c;
        func(c);
    });
}

Python

import nanobind_example


def callback(value):
    value.test()


nanobind_example.func(callback)

While this can be fixed by changing the std::function parameter list from NonCopyable& to NonCopyable*, it can be still an issue when binding to other code which can't be easily changed. So my question is, if there is any way to change the return value policy when calling an std::function

[wjakob]

In principle, a custom return value policy could be routed there. But how should nanobind expose this? I haven't bee able to find a good solution, external ideas/suggestions/PRs are welcomed.

[se7kn8]

I tried to make an design analog to the way how the nb::typed struct work.

I'm not fully happy how it is, but it might be the start for an discussion:
My idea is to provide a type rv_function which can be used to specify the rv_policy for std::function:
(Most of the code is copied from the default default std::function type-cast, just with the coded for the wrapper added)

template <rv_policy rv, typename Return, typename ...Args> struct rv_function { std::function<Return(Args...)> value; };

template <rv_policy rv, typename Return, typename... Args>
struct type_caster<rv_function<rv, Return, Args...>> {
    using ReturnCaster = make_caster<
        std::conditional_t<std::is_void_v<Return>, void_type, Return>>;

    using TypeName = rv_function<rv, Return, Args...>;

    NB_TYPE_CASTER(TypeName,
                   const_name("Callable[[") +
                       concat(make_caster<Args>::Name...) + const_name("], ") +
                       ReturnCaster::Name + const_name("]"))

    struct pyfunc_wrapper_t : pyfunc_wrapper {
        using pyfunc_wrapper::pyfunc_wrapper;

        Return operator()(Args... args) const {
            gil_scoped_acquire acq;
            return cast<Return>(handle(f).operator()<rv>((forward_t<Args>) args...));
        }
    };

    bool from_python(handle src, uint8_t flags, cleanup_list *) noexcept {
        if (src.is_none())
            return flags & cast_flags::convert;

        if (!PyCallable_Check(src.ptr()))
            return false;

        value = {pyfunc_wrapper_t(src.ptr())};

        return true;
    }

    static handle from_cpp(const Value &value, rv_policy rvp,
                           cleanup_list *) noexcept {
        const pyfunc_wrapper_t *wrapper = value.value.template target<pyfunc_wrapper_t>();
        if (wrapper)
            return handle(wrapper->f).inc_ref();

        if (rvp == rv_policy::none)
            return handle();

        if (!value)
            return none().release();

        return cpp_function(value.value).release();
    }
};

This allows function specifications like this:

    m.def("func2", [](nb::rv_function<nb::rv_policy::reference, void, NonCopyable&> func){
        NonCopyable c;
        func.value(c);
    });

[wjakob]

Thanks, that seems quite reasonable to me. I am not sure that there are much better alternatives. But I would name it differently -- let's call the interface: nb::set_rv_policy<std::function<...>, rv_policy>, so that it's clear that a std::function is being wrapped. The code duplication would be nice to avoid -- probably, both of these type casters can be implemented based on the same implementation.

[se7kn8]

While tinkering around with this I found out that this might be a suboptimal way. The problem is that this would set the rv_policy for every argument of the function which might not always be the desired way. A better approach would be to be able to set the rv_policy for every parameter individually. But this would require changes in the nb_call protocol. What do you think of this?

[wjakob]

There was a similar extension being integrated into the smart_holder branch of nanobind, where one can set a per-element return value policy for functions returning tuples.

I'm open to simple changes, but if it ends up imposing a big complexity cost to the rest of the project (especially if it is complexity that leaks into the main codebase and cannot be contained within the nanobind/stl/* header files), then my answer would be: one can always do the binding "by hand" by writing custom code that takes a nb::callable and does the conversion from/to std::function.