Official fs overlay

[arcanis]

With the addition of PRs like nodejs/node#48658 or nodejs/node#49593 virtual filesystem overlays are in an uncomfortable position where Node both doesn't respect anymore its own APIs (ie fs). We currently workaround that by patching some primitives from process.binding('fs'), but this is untenable, even at short term (nodejs/node#50687).

We often heard that "one of our goals is to allow mocking of modules, including of builtins; but only via loaders". That's fine to me, but given the recent acceleration of PRs aiming to shortcut fs it suggests now is the time to start discussing in details what such an overlay would look like that would satisfy everyone.

[targos]

How should we start the discussion? Depending on what we are talking about, this might interest the @nodejs/fs team, and people working on the permissions model.

Maybe we should try define first where this overlay would be placed?

There are several layers, currently:

• Native OS/system calls
• libuv
• C++ bindings
• Internal JS functions
• the public fs module

[arcanis]

The current way of patching fs lives between "C++ bindings" and "Internal JS functions" layers, but on the JS side. This model worked quite well so far, proving very flexible in the context of JS packages.

As for permissions, I suspect we'd want any hooking API to still live within the limits of whatever permissions were given to the process. We don't want to accidentally open a bypass, and implementing this layer in JS-land would prevent that (my understanding being that permissions are implemented at a lower level?).

[joyeecheung]

This sounds like a nodejs/node issue. We are also looking into it for single executable applications for example nodejs/single-executable#37