all: WebAssembly ("wasm") support

[bradfitz]

WebAssembly ("wasm") is similar to Native Client, but different notably in that other browsers plan to implement it.

http://webassembly.org/

This has been asked about a few times, so this is a tracking bug for it.

Whether we get it via cmd/compile, gccgo, or llvm-go, we can post updates here.

[minux]

@cherrymui and I discussed the possibility of a wasm GC port at length in the past few days: Our current conclusion is that there is no efficient way to implement setjmp/longjmp functionality currently, therefore it's not a viable target of a gc port. We need to wait until it gets real stack unwinding and exception handling support. All the other aspects looked fine, and we even designed a Go ABI under wasm (pass g and SP on wasm stack and everything else on emulated Go stack) and a way to modify the current MIPS backend to support wasm by emulating MIPS registers as wasm local variables. Perhaps GopherJS can support wasm easier.

[ernado]

From gopherjs/gopherjs#432:

However, it is not a technology that GopherJS can use. GopherJS compiles on the AST level, not on the machine code level.

WebAssembly could be used as a backend for the normal Go compiler

[4ad]

@minux The lack of threads or async i/o seems to be a bigger problem than the ABI workarounds we'd have to do because of wasm quirks.

[sbinet]

@minux, @cherrymui
could you post somewhere in more details what you did and achieved?
as you may know the go-interpreter community is considering writing an interpreter based on either LLVM or wasm bytecode.
(we may even have a GSoC student working on the wasm bytecode interpreter [1], so, just the "bytecode consumption" part, not the "bytecode production via some Go-based toolchain" one)

minimizing the amount of duplicated efforts and impedance mismatch between various components would be great.

[RaananHadar]

I highly recommend this as a strategic feature. Consider that in just a few years web programmers will flock in hordes to pick their language of choice to be compiled into web assembly. The sooner we join the race the better. Mozilla is pushing Rust hard as the web assembly language of choice...

[tiborvass]

I second what @RaananHadar said. It would be a pity if the spec didn't suit Go's needs, especially given how Go's runtime can be unique.

EDIT: Apologies for the MeToo nature of this comment :) Thanks for the pointer Brad.

[bradfitz]

Reminder: https://golang.org/wiki/NoMeToo

[vibhavp]

The WebAssembly stack machine context consists of linear memory and an operation stack. IIUC, saving the VM context can possibly be done by saving the:

1. Current PC, bytecode string
2. Linear memory, and the
3. Operation Stack

To a global list of saved contexts in the VM context struct, if any.

Thus, a setjump should simply restore these values, and continue with the interpreter loop as usual (This is similar to how the emacs bytecode VM implements signals: https://github.com/emacs-mirror/emacs/blob/master/src/bytecode.c#L785). An exception system can be implemented on the top of this, though I'm not sure how performant would this be.

Further, the webassembly spec mentions that the stack height at any moment in the bytecode is statically known, making all stack operations equivalent to operations on registers unique to each position on the stack. This paper describes a way to achieve such a stack -> register mapping for converting stack machine code to native code, allowing us to use setjmp/longjmp as usual.

Edit: What about using panic() and recover() with special WasmException{} values for signalling an exception? recover does the tough job of stack unwinding already, it shouldn't be hard to restore the interpreter loop after recovering from a caught exception.

[minux]

Yes, basically the discussion with @cherrymui resulted in similar findings. The initial design with @cherrymui is this: (this stage focus on getting $GOROOT/test/sieve.go to work, no async IO is considered.) we reuse the MIPS port, mapping the 31 MIPS registers as WASM local variables and only use WASM stack for intermediate results. We can't use WASM's function call mechanism for most Go functions because the lack of stack unwinding support. change the mips objlink (cmd/internal/obj) to emulate each MIPS instruction with respective WASM instructions. This makes setjmp/longjmp (aka runtime.gogo) implementation trivial and reuses a lot of existing effort. Go already needs to implement its own stack, separate of the WASM stack, because Go needs copyable stack, stack pointer maps and stack unwinding support, which are not available from WASM. (This strategy also aligns with the LLVM port, we can't use LLVM stack for almost the same reasons (unless we add a custom backend pass to LLVM proper to encode stack pointer maps).) We will probably need to use a big switch to thread indirect jumps? This is similar to NestedVM's approach. http://nestedvm.ibex.org/ If we decide to implement a native WASM backend, then we can do this: the Go ABI is: all parameters (including results) on (emulated) Go stack, at function entry, the following are on the WASM stack: emulated SP, g. The discussion happened more than one month ago, and I probably forgot some details at this point.

[vibhavp]

IIUC, you can completely discard the WASM stack (according to the spec). An extendable and copyable go stack is still something that needs to be investigated.

[minux]

Yes, the initial design I mentioned only uses the WASM stack for intermediates used in computation. All data is either on Go heap (linear memory) or pseudo registers in WASM local variables. I don't expect WASM spec to incorporate moving/segmented and copying stack. It's rarely used by mainstream languages targeted by WASM.

[SerkanSipahi]

@bradfitz @minux @vibhavp What is the current state of wasm in golang? Nothing has happened since March 13th ! will it be possible transform golang to wasm in the future? is there maybe any route-map of that?

[kanocz]

if somebody will decide to implement wasm support I can offer my time for non-global tasks

[bradfitz]

@SerkanSipahi, nobody is working on this. This bug is marked LongTerm. If something starts happening, you'll see updates here.

[joeblew99]

A good roadmap ?

We have DELVE and GDLV for debugging golang with a GUI. It works well too on all Desktops and Perf is excellent.
https://github.com/derekparker/delve
https://github.com/aarzilli/gdlv

Now, GDLV is based on NUCULAR which is shiny with some nice abstractions.
https://github.com/aarzilli/nucular

SO i was thinking this would be a good basis to do what has already been done here:

• Demo: http://shi-yan.github.io/AssortedWidgets/
• Code: https://github.com/shi-yan/AssortedWidgets

There are 5 WASM go libaries now too:
https://github.com/search?l=Go&q=webassembly&ref=simplesearch&type=Repositories&utf8=%E2%9C%93

[techtonik]

NaCl is shut down - https://blog.chromium.org/2017/05/goodbye-pnacl-hello-webassembly.html
So this should probably be cleaned up - https://github.com/golang/go/tree/master/misc/nacl

[ianlancetaylor]

I believe that NaCl is used by various parts of the Go ecosystem. That is, it's not used only by Chrome. And in any case that blog announcement is about PNaCl, which, although it provides functionality similar to NaCl, is actually a completely different product based on a different technology. So removing NaCl support from Go is premature at present.

Any in any case whether or not we remove NaCl from Go has nothing to do with whether we add support for Web Assembly.

[techtonik]

I would say that PNaCl is just an extension over NaCl to provide platform independent code https://developer.chrome.com/native-client/nacl-and-pnacl I asked about NaCl deprecation here - https://groups.google.com/d/topic/native-client-discuss/wgN2ketXybQ/discussion

If is possible to get list of those various parts of the Go ecosystem where NaCl is still used?
Going through it one by one will make transitioning to WebAssembly more interesting.

[4ad]

I would say that PNaCl is just an extension over NaCl

You can say whatever you want, but it doesn't mean you are right. They are completely different ISAs, NaCL is just the target ISA (x86/arm) with some restrictions, while PNaCl is a completely different ISA for an abstract machine.

Go has never supported PNaCL, and the Go NaCL port was never suitable for Chrome anyway. It had nothing to do with Chrome, it was not usable in the browser. Whether Chrome abandons NaCL or PNaCL has no relevance to Go whatsoever. Nada, nil, no relevance. How many more times this must be said? Of course, if NaCL is abandoned completely then Go will be forced at some point to abandon it too.

WebAssembly support in Go has absolutely nothing to do with NaCL. Go might or might not get WebAssembly support. If, or when it might get such support has no relevance to the state of the NaCL port.

[bradfitz]

Here is a very important part of the ecosystem where we use nacl: https://play.golang.org/p/MfJIq8wb5-

(that runs server-side, not nacl-in-a-browser)

[techtonik]

Whether Chrome abandons NaCL or PNaCL has no relevance to Go whatsoever. Nada, nil, no relevance.

And who will support sandbox loader and toolchain then? It is imported from Chrome project, which switched efforts to WebAssembly. Wouldn't it be wise to follow and see if Go playground can already be ported from NaCl sandbox to WebAssembly sandbox? It can also be run server-side.

[ianlancetaylor]

I think everybody is in favor of supporting WebAssembly.

The time to discuss moving away from NaCl is after WebAssembly is fully working. There's no point to discussing it before then.

[vine77]

@ianlancetaylor, can you clarify "after WebAssembly is fully working"?

At a glance, it appears webassembly.org says "the initial version of WebAssembly has reached cross-browser consensus" and wasm is available in current or future browser versions from all browser vendors.

[ianlancetaylor]

@vine77 I mean after WebAssembly is fully working for Go programs, at least as well as NaCl works today.

[agnivade]

@neelance - Now that callbacks are in, I believe basic support is complete. Shall we close this bug or did you have something else in mind ?

[neelance]

Yes, I think we can close this now. 🎉

[Jinex2012]

Great work!!! @neelance

[alecthomas]

This is awesome, great work @neelance!

[gopherbot]

Change https://golang.org/cl/120057 mentions this issue: doc/go1.11: mention GOOS/GOARCH values of WebAssembly port explicitly

[gopherbot]

Change https://golang.org/cl/120575 mentions this issue: cmd/dist: skip non-std tests on js/wasm

[matthewp]

Is it appropriate to open an issue on reducing the file size (if that is even possible)? Or is that being tracked elsewhere?

[bradfitz]

@matthewp, feel free to open a webassembly-specific size bug if you'd like. The general one is #6853.

[gopherbot]

Change https://golang.org/cl/120958 mentions this issue: net: re-implement built-in simulated network on JS and NaCl

[omac777]

Does the golang compiler have built-in control flow integrity(CFI)? I recently skimmed through the following to do with vulnerabilities in WASM prevented with CFI built-into the clang compiler:
https://www.fastly.com/blog/hijacking-control-flow-webassembly-program
https://github.com/trailofbits/clang-cfi-showcase/blob/master/cfi_vcall.cpp

[neelance]

Just skimmed it, but it seems to me as if the post says "WebAssembly is not a high level language such as JavaScript, but a low level language, so certain classes of bugs may apply if the source language (C or Go) allows them." No surprise there. This means that if Go would have such issues, they would also apply to other architectures, not only WebAssembly.

[randall77]

WebAssembly is mostly about defending the browser from nefarious WebAssembly code. It is not so much about providing guarantees within the WebAsssembly instance (although there is some of that). As @neelance said, it is up to the LanguageX->WebAssembly compilers to provide any required safety guarantees.
Go does not have CFI. I don't think it would be easy - at the point where a method starts, we've already lost the vtable. We've even lost the fact that a vtable was used.
Go is vulnerable to this exploit, in cases where there is any parallelism. For now, at least, the WebAssembly Go port (and WebAssembly, full stop) does not have parallelism so this exploit is only theoretical. The WebAssembly folks do plan to implement parallelism at some point, though.

[neelance]

@randall77 How does this affect plain linux/amd64? Is the risk somehow larger for WebAssembly?

[randall77]

The risk is the same regardless of architecture (with the exception that WebAssembly doesn't have threads yet, so it's actually zero for WebAssembly for the moment). Data races can be used to simulate unsafe, without importing unsafe.
It would be hard to exploit such a vulnerability. You either need to link in untrusted code, or somehow find and trigger a gadget in an existing binary that does a data race on an interface value and then invoke a method on the result.

[neelance]

Okay, so if your Go code has no data races, then it should be fine. Also, when running untrusted code with WebAssembly you can make sure that it can not escape the WebAssembly environment, even though it is able to have data races or simply use unsafe. Thanks for the interesting insights and sorry for this being slightly off-topic for this issue.

[omac777]

The more I examine this javascript usage from within golang, the more I perceive as an infection that will degrade the quality and the long-term maintenance of the golang code. Let me explain. Here is an example of a wonderful wasm golang app that does some impressive eye candy magic.

https://github.com/stdiopt/gowasm-experiments/blob/master/bouncy/main.go

The results are impressive. Delving furthing into the code that actually makes it happen though was disappointing because it require that every time you want to call something from the javascript side, it requires you to describe the name of the javascript function or the javascript value as a string AT EVERY TURN. As a result, there is no way for the golang compiler to check javascript correctness at compile time. It's "fly by prayer" at runtime. I would prefer to see golang functions/variables/types everywhere for long-term maintenance otherwise it defeats the purpose of using golang for wasm in my humble opinion. BTW I detest javascript and always have. It's too freestyle. Why is it that such a language has persisted in the web browsers? The answer escapes me.

Thank you for listening.

[geovanisouza92]

I agree that calling JS using strings isn't the better experience. Maybe we could generate Go interfaces from web platform specification IDL's. The downside is how to keep up with the spec, because depending on the browser it will run, the API's isn't available, while on the other side, new spec keep coming all the time as the web platform evolves.

[neelance]

@omac777 I agree with you. That's why I hope that there will be nice Go libraries around the JS low levels so most users will never have to touch syscall/js directly. Similar to the syscall package, it is ugly and almost no one uses it directly. ;-)

[dmitshur]

@omac777 For GopherJS, many people use high-level bindings for various browser APIs rather than the low-level js package directly. I suspect a similar approach will be most popular with Wasm too. I expect many of the GopherJS bindings will add support for Wasm in the same package, and new packages will be created as needed.

For reference, see https://github.com/gopherjs/gopherjs/wiki/Bindings, https://dmitri.shuralyov.com/talks/2016/Go-in-the-browser/Go-in-the-browser.slide#11 (and the following 4 slides), and dominikh/go-js-dom#57.

[guidorice]

Also we should consider there is a bunch of stuff on the webassembly roadmap:

https://webassembly.org/docs/future-features/

that will improve the syscall story quite a lot, in the long run, like

reference DOM and other Web API objects directly from WebAssembly code;
call Web APIs (passing primitives or DOM/GC/Web API objects) directly from WebAssembly without calling through JavaScript; and
efficiently allocate and manipulate GC objects directly from WebAssembly code.

Anyways, thanks @neelance et. al. for implementing the initial version. So awesome! 🥇

[fractalbach]

What if there was a Go implementation of the DOM?

Documents and scripts could be written and executed in Go, making use of the type system.

Then, code could be generated from the Go DOM.

[yookoala]

@fractalbach: If the WebAssembly's host binding is confirmed and implemented by the work group, you can actually do DOM manipulation through WebAssembly. Then it would be reasonable to have high level libraries to abstract DOM, Documents and scripts.

But before that, it is not quite as appealing to do.

[omac777]

Go WebAssembly: Binding structures to JS references

https://medium.com/@nlepage/go-webassembly-binding-structures-to-js-references-4eddd6fd4d23

The approach is appealing. The usage is similar to json marshallling/unmarshalling within structs. That makes it an easily transferable skill.

The one thing missing is how to apply the same approach to duck-typing? I would prefer not to have the function declarations within the structure, but outside of it standalone in the same manner as duck-typing. If the service exists, it can use it. That way it is easily modifiable without affecting the core struct holding the different desired non-function values. Duck-typing rocks!

Thank again @neelance for all your work. It's greatly appreciated.
Thank you Mr. Nicolas Lepage for your points of view. They really help to crystallize what could be a better path for interoperating with all this javascript noise for the interim until the direct wasm interfacing is complete.

[agnivade]

Hello everyone, this is a high traffic thread with a lot of people subscribed. Having a general discussion like this is not fruitful to those just concerned about the original issue - which is to have WebAssembly support for Go.

Please feel free to continue the discussion in appropriate forums - golang-nuts/golang-dev. Or if you have something specific in mind, please open a new issue.

Thank you.