Eigen::Map readwrite access from Python - difference between pybind11 and nanobind

[ThoreWietzke]

Hello,

I have a C++ Struct which contains Eigen::Maps which point to allocated memory from a C interface.
The layout is as follows:

struct Vec
{
    Map<VectorXd> map;

    Vec() : map(NULL, 0){} // initialize the map with an empty data pointer

    void setData(double* data, int size)
    {
        // placement new for reusing the allocated memory 
        new (&map) Map<VectorXd> (data, size);
    }
};

The binding code for my struct is as follows:

NB_MODULE(test_eigen, m)
{
    nanobind::class_<Vec>(m, "Vec")
        .def(nanobind::init<>())
        .def_rw("map", &Vec::map);
}

In pybind11 such binding code results in an error, because the load method in their type caster gets explicitly deleted because of the read/write access of the map. In nanobind this works fine, because the corresponding from_python method exits for this binding.

My question is, are there any issues in allowing this conversion?
Compilation works fine and setting the elements of the exposed map in Python also works fine.

[ThoreWietzke]

One additional information: in pybind there exist an issue with a similar question: pybind/pybind11#1402, but an explanaition for this behaviour ist not mentioned.

[wjakob]

nanobind will allow you to do this, but that doesn't mean that it's safe. Who owns the data? How will you ensure that the data is not accessed by Python (If the Vec buffer was allocated by C++) or C++ (if the Vec buffer was allocated by Python/NumPy) after the original buffer was deleted? You could potentially make it safe using further lifetime annotations, but those are missing from the example snippet.

[ThoreWietzke]

Thanks for your remarks.

In my application, Vec is an interface for a C library which uses raw buffers. The lifetime of the buffers is directly tied to Vec, so if Vec is deconstructed, the buffers are freed.

A more complete example would be

struct Vec
{
    Map<VectorXd> map;

    Vec() : map(NULL, 0) // initialize the map with an empty data pointer
    {
        allocate_memory(raw_buffer);  // memory for the interface gets allocated
        // placement new for reusing the allocated memory 
        new (&map) Map<VectorXd> (raw_buffer, size);  // size is known beforehand through the library
    } 

    ~Vec()
    {
        free_memory(raw_buffer);
    }

    // functions for interfacing with the library

    private:
        double* raw_buffer;
};

Therefore I think accessing the buffer should be safe.

[wjakob]

That takes care of read-only accesses. But what if the user assigns a NumPy array to the "map" property? Will this create a copy or a dangling reference to the NumPy array? (I am not sufficiently familiar with the behavior of operator= of Eigen::Map<>)

[ThoreWietzke]

As far as I see, the operator= of Eigen::Map is borrowed from the template matrix type of the map, e.g. Eigen::MatrixXXf. This is based on the Eigen::Dense class which implements operator= with a copy operation:
https://eigen.tuxfamily.org/dox/classEigen_1_1DenseBase.html#title69

So no dangling reference to the NumPy array should occur.

[wjakob]

Sounds all good then!