invoking std::function inside std::thread hangs

[samuelpmish]

Hi,

I've been using nanobind in a few projects and it's been working great so far. One project has a function that accepts a std::function as one of its arguments and spawns some number of threads that evaluate that function with different inputs. I'm interested in using nanobind to expose this function to python, but it seems to be hanging when calling the function from inside a std::thread.

For example:

NB_MODULE(example, m) {
  // invoke the function directly
  m.def("foo", [](const std::function<float(float)> & func) { 
    return func(4.2f);
  });

  // invoke the function from inside a std::thread
  m.def("bar", [](const std::function<float(float)> & func) { 
    float answer = 0;
    std::thread thr([&](){ answer = func(4.2f); });
    thr.join();
    return answer;
  });
}

When testing this with a simple python script

import example

def f(x):
    return x + 3

print("running foo(f)")
print(example.foo(f))
print("running bar(f)")
print(example.bar(f))
print("done")

I get the following output:

% python3 test.py 
running foo(f)
7.199999809265137
running bar(f)

That is, the direct evaluation of the python-to-std::function works, but using it from inside a std::thread causes the program to hang indefinitely.

Is this an inherent GIL limitation, a bug with nanobind or something else entirely?

Thank you,

Sam

[wjakob]

That's a GIL-related deadlock. bar holds the GIL, and then your thread is trying to re-enter it when calling the Python function func. If those two thread were completely decoupled, then they would simply wait for each other. But you call join() within bar. This should work if you drop the GIL within bar by adding nb::gil_scoped_release guard. However, note that you're not allowed to call other nanobind functions within that scope since they assume that the GIL is held.

[samuelpmish]

Ah, thank you for the quick response!