-
Notifications
You must be signed in to change notification settings - Fork 653
streaming readdir() #931
Comments
I agree.
This is indeed a longstanding issue in node: nodejs/node-v0.x-archive#388
Neither do I, you're free to propose one :) It might be worthwhile to report a little bit more than just filenames. Windows can do this with |
Correct me if I'm wrong, but the I suppose that, with some kind of multi-call/streaming API, this would imply a |
opendir() / readdir() / closedir() is more portable. Either way, the kernel doesn't report anything besides the filename and the inode number. |
|
I can't speak for Windows but why don't you use readdir_r() on UNIX? That will require some changes though, I think. uv_fs_readdir_next() should take a list of uv_dirent_t structures and go through the thread pool. It should also copy out the d_name from the native struct dirent because that won't be stable over time. I would not recommend making uv_dirent_t an alias for struct dirent because the layout of that struct may change depending on compilation flags. |
Aha, I (or maybe @indutny) will look into that.
Sure. Thanks for the comments Ben. |
Ran into this problem this week (trying to process a directory with >1M files), and felt this pain, so would love to see this happen. I'm very ignorant on this topic, but nodejs/node-v0.x-archive#388 (comment) seems encouraging. =) |
Yeah, closing! |
The current
uv_fs_readdir
API returns all entries in a directory, at once, in achar*
ptr
field onuv_fs_t
. This is problematic in the case of a directory with lots of files allocating a large amount of memory on every request as well as for perf.We (for rust, but others as well I'm sure) would like some kind of streaming/paging approach to directory listing.
I have no idea what this API would look like. Thoughts?
The text was updated successfully, but these errors were encountered: