WASI IPC module

[skyne98]

Hey there, everyone!

As I very briefly described in WebAssembly/wasi-sdk#33, WASI at the moment seems to be lacking support for IPC communication. I think that this feature is essential for some applications, including drivers, daemons, window managers as well as applications talking to system-wide data buses.

It looks like there only about a dozen system calls required to make all IPC scenarios run correctly if we decide to follow the POSIX syscalls first.

So, is this possible? What do you think about the amount of effort it will take to implement support for it? The only things that sketch me out are mentions of synchronization primitives.

Thanks!

[sunfishcode]

On one level, talking to drivers, daemons, window managers, system-wide data buses directly is often non-portable, even if the IPC APIs themselves are relatively portable. It's one thing to expose Unix-domain socket functinonality, but another if what you really want to do is eg. Wayland protocol, because not all hosts have Wayland. So one option here is to look into more abstract interfaces to eg. display services. This is surely a complex task, but it may be preferable if the alternative is eg. emulating Wayland on hosts with non-Wayland display services.

On another, some use cases may want wasm primarily for the architecture independence, and less so the OS independence, and may be ok assuming eg. Linux and associated userspace services. With WASI's anticipated modular API design, we could plausibly consider such use cases, though it'll come down to priorities of the Subgroup.

[skyne98]

I think that I might be one of those people, who consider architectural independence more important, over the operating system independence. However, I understand that you cannot neglect all the other user groups that can invent interesting use-cases for WASI!

In my particular use-case, I am interested in using ZMQ on WASI (whose IPC interface is the fastest one and can really benefit from something like in-kernel WASI with fast syscalls). Also, another very interesting possibility to me is having a portable pure-rust wayland compositor based on Smithay. However, its implementation requires IPC for communications.

I think that the modular design might be very meaningful in such a case. I think that there should definitely be modules covering most of often-used POSIX system calls.

[dumblob]

Just few nitpicks.

I am interested in using ZMQ on WASI (whose IPC interface is the fastest one...

ZMQ is arguably not the fastest IPC (you don't have to go far - look at ZMQs successor nanomsg or the nanomsgs successor nng ).

I think that the modular design might be very meaningful in such a case. I think that there should definitely be modules covering most of often-used POSIX system calls.

Sure there will be modules, but I'm not convinced that these modules shall be part of WASI. There shall be a WASI-compatible POSIX module, but it wouldn't be part of WASI IMHO.

[skyne98]

@dumblob hey there! I didn't really mean that ZMQ is the fastest one, I meant that IPC is the fastest transfer medium for ZMQ itself. But thanks a lot for the recommendation, a will look into nanomsg!
Also, yeah, I think that having a POSIX module for WASI is a good idea.

[olanod]

What about APIs like the ones seen on the web, quite high level like broadcast channel or something like shared workers to let people decide on how to implement their communication channel or eventually have a D-Bus like API built on top of it?

[sunfishcode]

It's in scope. The first step is to examine some high-level design questions.

What do "browsing context" and "same origin" translate to in a non-Web context? Do the applications that wish to communicate all live in the same WebAssembly store?

[lygstate]

IPC are important for high performance work
For Windows we have mmap. anymous file mapping
Linux also have mmap. memfd
For Unix world we have
shm_open + mmap.

[skyne98]

It has been quite a while, looks like WASI has seen some significant progress. Has there been any progress on modules for architecture-independent UNIX-specific APIs?

Right now I am having a look at NNG again and documentation for IPC communication describes the need in UNIX domain sockets. Is it possible to use something like NNG via WASI nowadays, what do you think?

Again, thanks for the amazing work!

[sunfishcode]

There hasn't been anyone volunteering to advance Unix-specific APIs so far. That said, I expect it is an area that makes sense in general.

[skyne98]

What do you think is the most performant way for a WASI binary to talk to the outside world right now?

It looks like communication over TCP is partially supported, which also means WebSocket? Or maybe UDP?
Because in case any of those are available, it will make it possible for those binaries to communicate with the overall system, which is nice.

[sunfishcode]

Could you say more about what you're looking to do here? You mentioned WebSocket; are you looking for something that can run in a browser? Of course, you also mention UDP or TCP, which aren't available (as such) in browsers :-).

[skyne98]

The idea is to have universal WASI binaries which can communicate with the outside system via a system bus (similar DBUS). So I am looking for a way for those binaries to cross the isolated border.

[sunfishcode]

Could you be even more specific? 😃 Would these binaries be hardcoded to work with D-Bus specifically, or are you envisioning an abstraction that could be implemented on D-Bus or also on operating systems that don't use D-Bus but have comparable functionality?

What would be the sandboxing requirements for these binaries? With access to D-Bus or similar systems, would WASI need to be able to restrict the kinds of messages they can send, or who they can be sent to, to provide additional sandboxing?

D-Bus carries a lot of assumptions about desktop-like computing environments. Would these "universal" binaries similarly be limited to desktop-like use cases?

[lygstate]

Could you be even more specific? 😃 Would these binaries be hardcoded to work with D-Bus specifically, or are you envisioning an abstraction that could be implemented on D-Bus or also on operating systems that don't use D-Bus but have comparable functionality?

What would be the sandboxing requirements for these binaries? With access to D-Bus or similar systems, would WASI need to be able to restrict the kinds of messages they can send, or who they can be sent to, to provide additional sandboxing?

D-Bus carries a lot of assumptions about desktop-like computing environments. Would these "universal" binaries similarly be limited to desktop-like use cases?

I think for windows Named PIPE, and for unix, (Unix Sock) is enough.
And beyond this, cross process shared memory access is necessary:)
There is no need bound to D-Bus

[skyne98]

Yeah, D-BUS is just a comparison 🙂 My idea of "universal" binaries is more similar to UWP, where it mostly means architecture portability on top of one specific operating system or operating environment.

I used D-BUS as an analogy, because I want to discover the most high-bandwidth and low-latency way for this binary to exchange data (binary messages) with the outside system.

In my case the bus itself will handle all the access limitations and sandboxing. In theory, each binary needs access to only one socket or even just one TCP/UDP port, whatever is possible with WASI at the moment. And the communication should be both-ways.

Those binaries are indeed for a desktop operating system.

[sunfishcode]

The most straightforward way to get started would be to use stdin/stdout. That works in WASI today, and host environments can arrange for those to be TCP, UDP, or other kinds of sockets, or pipes, or other things.

[skyne98]

But I would guess that using stdin/stdout for communication compromises binary's ability to use it for terminal input/output? If that's true, that will mean that porting applications to this binary format will require rewriting everything that outputs into stdout.

[tsavola]

In my case the bus itself will handle all the access limitations and sandboxing. In theory, each binary needs access to only one socket or even just one TCP/UDP port, whatever is possible with WASI at the moment. And the communication should be both-ways.

I have done something similar with Gate. Gate programs can use WASI functions to do I/O through a single file descriptor. The fd number can be discovered via an environment variable (WASI). That fd is used to talk to Gate's custom service bus. See https://github.com/gate-computer/gate/blob/master/ABI.md for more details.

[skyne98]

@tsavola, thanks for an interesting idea! I need to look into communicating via simple file descriptors. I wonder if nng can use that for communication...

Also, you reminded me that I am still working with WASM modules and I can import functions from the outside environment to use as system calls!

[abrown]

(@tsavola, it was fun to read about your project!)