File handle left open when an nd2 file fails to read

[manthey]

• nd2 version: 0.5.1
• Python version: 3.9
• Operating System: Ubuntu 20.04

Description

I try to open the nd2 file downloaded from here: https://downloads.openmicroscopy.org/images/ND2/aryeh/b16_pdtB+y50__crop.nd2
It fails with TypeError: <lambda>() missing 6 required positional arguments: 'bitsPerComponentInMemory', 'bitsPerComponentSignificant', 'componentCount', 'heightPx', 'pixelDataType', and 'sequenceCount'.

If I catch that exception and do something else, I notice that there is an open file handle to the file I tried to load that never closes, even when all my local variables are deleted and gc.collect() is run. If I try to open a number of bad files, eventually, a lot of file handles are permanently open (cab be checked via /proc//fd, for instance).

What I Did

python test.py

# where test.py is
---
import gc
import time
import nd2

try:
    f = nd2.ND2File("b16_pdtB+y50__crop.nd2")
    print(f.sizes)
except Exception as exc:
    print exc
f = None
gc.collect()
time.sleep(1000)
---

ls /proc/<pid of python process>/fd

Naturally, it would be great to read all files, but on failure it would be nice to release the file handle. I tried explicitly calling f.close() and f._rdr.close() in the exception handler, but that didn't release it either.

[tlambert03]

thanks for the report @manthey! totally agree with the desired behavior. there's a couple things to address here.

I have that same OME nd2 file in my test files and it opens fine on my computer (macos)... So I'm having difficulty reproducing the leaked handle. I'm currently downloading your file too just to see if it has a different sha and/or behavior.

as for your example. I can reproduce the leaked file when using the f = nd2.ND2File(...); f = None syntax, but can't reproduce it when I use the recommended approach of f.close() or with nd2.ND2File() context manager.
While I certainly wouldn't recommend it to people, I can try to fix the case of simply clobbering the local name as well (and I definitely agree that if an exception prevents you from using close or the context manager, that a file shouldn't be leaked.

Anyway, give me a bit to see if I can find a file and/or system to reproduce the error-on-open leak. and I'll also look at handling the clobbered local followed by gc.collect as well

[tlambert03]

just a note the b16_pdtB file in your link has the same sha as my local one, and worked fine for me too... (f.sizes = {'T': 57, 'P': 12, 'Y': 1200, 'X': 1600}) will seek out a linux box to try as well

[tlambert03]

On Dec 7, 2022, at 9:58 AM, David Manthey ***@***.***> wrote: nd2 version: 0.5.1 Python version: 3.9 Operating System: Ubuntu 20.04 Description I try to open the nd2 file downloaded from here: https://downloads.openmicroscopy.org/images/ND2/aryeh/b16_pdtB+y50__crop.nd2 It fails with TypeError: <lambda>() missing 6 required positional arguments: 'bitsPerComponentInMemory', 'bitsPerComponentSignificant', 'componentCount', 'heightPx', 'pixelDataType', and 'sequenceCount'. If I catch that exception and do something else, I notice that there is an open file handle to the file I tried to load that never closes, even when all my local variables are deleted and gc.collect() is run. If I try to open a number of bad files, eventually, a lot of file handles are permanently open (cab be checked via /proc//fd, for instance). What I Did python test.py # where test.py is --- import gc import time import nd2 try: f = nd2.ND2File("b16_pdtB+y50__crop.nd2") print(f.sizes) except Exception as exc: print exc f = None gc.collect() time.sleep(1000) --- ls /proc/<pid of python process>/fd Naturally, it would be great to read all files, but on failure it would be nice to release the file handle. I tried explicitly calling f.close() and f._rdr.close() in the exception handler, but that didn't release it either. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[tlambert03]

Hey @manthey, I just released a new version of nd2. I think it should solve your script case of deleting the pointer, but since I can't reproduce the file open error (and I don't have other files that fail), could you try it out to see if you still have the leaked file handle on read error?

[manthey]

Thanks @tlambert03. I still see the problem on Ubuntu. Specifically, I think the file handle is staying open after whatever causes the failure in reading the attributes within the file there. I also just tried this on python 3.9.15 on an M1 mac, and there it works (and tells me I failed to close the handle before garbage collection).

In ordinary use, I am using close(). That doesn't affect whether the file handle stays open on Ubuntu, though. I also tried explicitly calling f._rdr.close() in the exception handler, but that didn't release it either.

And, I doubt it will be surprising, but the failure also occurs on Ubuntu using python 3.11.

[tlambert03]

ok! thanks for checking. I'll try it on ubuntu soon and give it a fix. (will ideally be able to fix both the leaked handle, but also the failed read itself... since that's definitely a supported file)

[manthey]

As a crude workaround for the file handle, if I guard these lines: https://github.com/tlambert03/nd2/blob/main/src/nd2/_sdk/latest.pyx#L78-L84 in a try/except, and then on exception do Lim_FileClose(self._fh) and self._is_open = 0 and then reraise, it does release the file handle.

[tlambert03]

thanks, yeah, I definitely can/will do that too...

but I really want to figure out why you're getting that lambda error! 😂 I just tried on my ubuntu (20.04.5, py311) and the file opened fine!
I'm having difficulty even determining where that error would happen (I don't have any lambdas in the code). do you have a full stack trace?

[manthey]

The full stack trace is:

<lambda>() missing 6 required positional arguments: 'bitsPerComponentInMemory', 'bitsPerComponentSignificant', 'componentCount', 'heightPx', 'pixelDataType', and 'sequenceCount'
Traceback (most recent call last):
  File "test.py", line 8, in <module>
    f = nd2.ND2File('b16_pdtB+y50__crop.nd2')
  File ".../env39/lib/python3.9/site-packages/nd2/nd2file.py", line 98, in __init__
    self._rdr = get_reader(
  File ".../env39/lib/python3.9/site-packages/nd2/_util.py", line 68, in get_reader
    return ND2Reader(
  File "src/nd2/_sdk/latest.pyx", line 48, in nd2._sdk.latest.ND2Reader.__cinit__
  File "src/nd2/_sdk/latest.pyx", line 79, in nd2._sdk.latest.ND2Reader.open
  File "src/nd2/_sdk/latest.pyx", line 115, in nd2._sdk.latest.ND2Reader.attributes.__get__
TypeError: <lambda>() missing 6 required positional arguments: 'bitsPerComponentInMemory', 'bitsPerComponentSignificant', 'componentCount', 'heightPx', 'pixelDataType', and 'sequenceCount'

I get this same error on two different Ubuntu 20.04 machines. But, I don't get it on the python:3.9-slim docker on one of those ubuntu machines.

[tlambert03]

interesting. I'm wondering if Lim_FileGetAttributes in the SDK itself is somehow failing on those systems.

if it's not too much trouble, could you test that for me by putting the following lines immediately after the opening of self._fh (around line 78 in latest.pyx)?

            string = Lim_FileGetAttributes(self._fh)
            print(string)
            Lim_FileFreeString(string)

[manthey]

Hmm... That print(string) causes a Seg fault.

Based on this working in a couple of different docker environments, I'm guessing it is somehow a library I have on my Ubuntu machines that is causing an issue.

[tlambert03]

nonetheless ... concerning! Do any files open for you on those systems? or nothing at all?

[tlambert03]

and it's the print statement huh... not the Lim_FileGetAttributes(self._fh) call?

[tlambert03]

just a thought... do you work with this library? https://github.com/nlohmann/json and possibly have a non-standard version?

[manthey]

Other nd2 files from that same directory work: (e.g., MeOh_high_fluo_003.nd2), plus lots I have from another lab work correctly.

I'm not aware of using that json library,

[manthey]

As one more interesting clue to failing to read some nd2 files in some environments and not others: When I use stock Ubuntu python 3.8 or python 3.9 installed from the deadsnakes ppa, I get the failure to read attributes. If I use pyenv to compile python 3.9 locally, that reads the nd2 file successfully. The only difference I have noticed is that using LD_TRACE_LOADED_OBJECTS=1 shows that the system/deadsnakes pythons both pull in libexpat, whereas the pyenv python does not.

[tlambert03]

very interesting! thanks so much. I'm going to open a new issue to track this linux error

[manthey]

Thank you.