What is the difference between wasi-sdk and emscripten?

[miladfarca]

Hello,

I wanted to get more information on wasi-sdk and it's support on big endian platforms.

BE support was added to emscripten in this PR: emscripten-core/emscripten#13413
emscripten uses llvm/clang to generate wasm binaries. From my understanding it also compiles musl libc to wasm and glues it all together with a generated js file which can directly be called with Node js, and it all gets executed by V8 runtime. Everything is little endian enforced by wasm standards and V8 is hard coded to reverse bytes accordingly on BE machines.

What is the difference between the output of wasi-sdk and emscripten and where does it stand in terms of supporting BE machines as well as C runtime? Does it also make sure libc is LE enforced?

[miladfarca]

/cc @sbc100

[sunfishcode]

While Emscripten produces JS code to accompany the wasm files it produces, wasi-sdk doesn't produce any JS. Since wasi-sdk only produces wasm, it isn't affected by the issue that JS typed arrays are endianness-dependent in JS.

WebAssembly and WASI are little-endian, so implementations on big-endian platforms need to perform byte-swapping under the covers, and several of the major implementations indeed do this.

[miladfarca]

WebAssembly and WASI are little-endian

That't right. V8 is patched to reverse bytes on BE platforms when executing wasm itself.
We have also patched emscripten to handle Js typed arrays correctly on BE.

My only concern right now is related to runtime calls to functions outside of wasm, mainly libc.
If I'm not mistaken, libc is also compiled into wasm on emscripten so byte ordering will be covered by V8 at runtime.

How are such calls to libc or Wasi API handled by wasi-sdk? Are they compiled to wasm as well? I'm just wondering if we need to apply any patches to make sure LE ordering is enforced to any outside calls.

[sunfishcode]

In wasi-sdk, libc itself is implemented entirely in wasm. There are calls to WASI APIs which may be implemented outside of wasm, but the expectation is that the implementations of those APIs behave as-if they are little-endian. And in practice, the major implementations indeed do run on big-endian platforms and do the byte-swapping as needed.

[miladfarca]

Thanks for confirming. So how are WASI APIs outside of wasm compiled and handled?
i.e are they also Wasm binary files which get executed directly by V8, or are they generic binaries which could access memory in native BE order and mix byte orderings?

[sunfishcode]

To my knowledge, V8 today does not have an implementation of WASI. In engines which do, such as Wasmtime, WASI is typically implemented in native host code, rather than wasm code, and it performs bytes-swapping on big-endian platforms so that it presents a little-endian API to wasm.

[miladfarca]

I see, so by default it should handle bye swapping on BE to maintain LE ordering.

V8 today does not have an implementation of WASI.

So then how should wasi-sdk generated binaries get executed? They can't run directly on V8 and there needs to be an engine such as Wasmtime which would call V8 behind the scene?
Just wondering how Node.js would execute a generated wasi-sdk binary such as this:
nodejs/llhttp#93

[richardlau]

Just wondering how Node.js would execute a generated wasi-sdk binary such as this: nodejs/llhttp#93

Node.js includes https://github.com/nodejs/uvwasi as its WASI implementation.

[miladfarca]

Thanks for the link Richard,
Also thank you for the information @sunfishcode . Will close this issue now.