doc: add minutes for meeting 5 Sep 2022 #31
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| # Node.js Single-Executable Meeting 2022-09-05 | ||
|
|
||
| ## Links | ||
|
|
||
| * **Recording**: https://youtu.be/93cIMtkQaak | ||
| * **GitHub Issue**: https://github.com/nodejs/single-executable/discussions/19 | ||
|
|
||
| ## Present | ||
|
|
||
| * Anna Henningsen @addaleax | ||
| * Darshan Sen @RaisinTen | ||
| * Jean Burellier @sheplu | ||
| * Juan Cruz Viotti @jviotti | ||
| * Maël Nison @arcanis | ||
| * Tobias Nießen @tniessen | ||
|
|
||
| ## Agenda | ||
|
|
||
| * Document the complete list of requirements for the VFS [#28](https://github.com/nodejs/single-executable/pull/28) | ||
| * ; | ||
|
|
||
| * How are native addons going to interact with the VFS? [#29](https://github.com/nodejs/single-executable/discussions/29) | ||
| * ; | ||
|
|
||
| ## Notes | ||
|
|
||
| During this call, most of the discussions revolved around the problem of | ||
| designing a general Virtual File System that would fit every necessary use | ||
| case, ranging from compression support, native add-on interoperability, etc. | ||
|
|
||
| The key conclusion from the call is that designing a truly universal VFS that | ||
| checks all the boxes that we care about and the ones that are probably unknown | ||
| so far is not trivial. For this reason, we will attempt to make VFS | ||
| user-defined for the SEA MVP. That way, we will have a better sandbox to play | ||
| with VFS', see what people come up with and eventually bring back these | ||
| learnings to the SEA project without getting indefinitely stuck at the VFS | ||
| phase. | ||
|
|
||
| To accomplish this decoupling, we hope to: | ||
|
|
||
| - Provide hooking capabilities to Node.js to allow developers to extend | ||
| `require()`, `fs` and related key components of Node.js to support arbitrary | ||
| virtual file systems without monkey-patching these modules like i.e. | ||
| Electron, Yarn and PKG do at the moment. | ||
|
Comment on lines
+33
to
+36
There was a problem hiding this comment. I think it should be clearer that "Provide hooking capabilities to Node.js" doesn't actually involve changing Node.js in any way at this time - even if non-official, we already have the "hooking capabilities" we need (monkey-patching) to implement a userland SEA. Of course it won't the best in terms of quality / perfs, but since I think we agreed this userland idea was only to get a couple prototypes that could be used as basis for the discussion (and that it wasn't intended to be the final "deliverable"), it's good enough:
There was a problem hiding this comment. True. However, we would still need to hook into these things indefinitely (see the Bootstrapper component discussed in https://github.com/nodejs/single-executable/blob/main/blog/2022-08-05-an-overview-of-the-current-state.md#bootstrapper), even with a blessed implementation. And if that's the case, should we just solve the entire thing for once and for all? As we have seen with Yarn, PKG, Electron, etc a very common reason for monkey patching As far as I understand, even I guess @addaleax and @RaisinTen might be able to confirm or debunk :P Feels at least worth a look IMHO There was a problem hiding this comment.
The way I imagine it, once a format is selected, the support for reading it from a VFS would be integrated inside Node (similar to how Python supports importing modules from eggs without users having to hook anywhere). We'd then get the bootstrap "for free", since by running
I'd say it falls within the scope of https://github.com/nodejs/loaders, a separate WG working to add customization support to import/require. There was a problem hiding this comment.
That seems like a big stretch? By itself, fs hooking has nothing to do with module loading. There was a problem hiding this comment. The loaders team has talked about renaming to the "customization" team and expanding their purview. Might make sense to double-check with them. There was a problem hiding this comment.
Yeah, I think we can eventually aim for that, once we have a VFS that we are confident to support long term and that works for all the scenarios we care about. Still, the "native" implementation on Node.js does require some hooking (even if it's internal hooking). |
||
|
|
||
| - Fix remaining issues on the Resource Injector current implementation | ||
| (https://github.com/postmanlabs/postject) for injecting multiple sections of | ||
| data. One of these sections will contain the VFS byte-stream, and the other | ||
| one will contain the bootstrap script that can be injected to `teach` Node.js | ||
| about the VFS of choice. | ||
|
|
||
| Once we have a better grasp on what the ideal VFS should look like, we will | ||
| attempt to ship the SEA project with a blessed VFS implementation. | ||
|
|
||
| We also touched on what the definition of "Single Executable Application" | ||
| means. Does it mean we will go out of our way to package every Node.js | ||
| application as a single file (embedding native add-ons, etc on the executable | ||
| itself)? Does an app that consists of the main executable along with sibling | ||
| ".node" shared libraries still qualifies as a SEA? Clearly defining what an SEA | ||
| is (and is not) is key for identifying the core constrains for the ideal VFS | ||
| implementation later down the road. | ||
|
There was a problem hiding this comment. the “s” is for “single”, so imo unless it results in just one file, it doesn’t qualify. There was a problem hiding this comment. That was the discussion about. The project name doesn't say "single file applications", but "single executable applications". Therefore, the question is: does a Node.js CLI application compiled to a binary + a couple of Some files must live outside of the main binary to be used in all operating systems (like native add-ons), and as @zcbenz and @arcanis pointed out in other discussions, extracting them on the fly every single time can be both problematic and slow. If an app consisting of an executable + shared objects still qualifies as a SEA, then that can solve a lot of problems. Or to ask the reverse question: what do we gain from having a single file containing the entire thing vs a single executable + shared objects? There was a problem hiding this comment. It's much easier to distribute and manage. There was a problem hiding this comment. It's true that a single file its easier to distribute, however people rarely distribute standalone binary executables in the internet. Instead, even single executables are usually distributed on a ZIP, tarball, etc. One of the core reasons is that on i.e. macOS, the browser would strip the executable bit from the binary and users would need to be taught how to So my thinking is: if you would zip/tar the single binary anyway, then having a few files rather than a single one doesn't really make distribution much harder. Furthermore, having a single file representing an app with native add-ons on Windows is a no-go (#30 (reply in thread)), and it might be a hard constraint outside of our control. The current status is: if a Node.js app has native add-ons and targets Windows, shipping a single file is not possible. So we either accommodate our definition or say that apps with native add-ons cannot be archived as an SEA. |
||
|
|
||
| ## Action Items | ||
|
|
||
| - [ ] Make defining user-land VFS' feasible (given the plan outlined above) | ||
| - [ ] Define and document what SEAs really mean | ||
|
|
||
| ## Upcoming Meetings | ||
|
|
||
| * **Node.js Project Calendar**: <https://nodejs.org/calendar> | ||
|
|
||
| Click `+GoogleCalendar` at the bottom right to add to your own Google calendar. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems like a really problematic thing to consider (require especially) if it will work outside a SEA.