Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all: WebAssembly ("wasm") support #18892

Closed
bradfitz opened this issue Feb 2, 2017 · 147 comments
Closed

all: WebAssembly ("wasm") support #18892

bradfitz opened this issue Feb 2, 2017 · 147 comments
Labels
arch-wasm WebAssembly issues NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@bradfitz
Copy link
Contributor

bradfitz commented Feb 2, 2017

WebAssembly ("wasm") is similar to Native Client, but different notably in that other browsers plan to implement it.

http://webassembly.org/

This has been asked about a few times, so this is a tracking bug for it.

Whether we get it via cmd/compile, gccgo, or llvm-go, we can post updates here.

@bradfitz bradfitz added this to the Unplanned milestone Feb 2, 2017
This was referenced Feb 2, 2017
@minux
Copy link
Member

minux commented Feb 2, 2017 via email

@ernado
Copy link
Contributor

ernado commented Feb 2, 2017

From gopherjs/gopherjs#432:

However, it is not a technology that GopherJS can use. GopherJS compiles on the AST level, not on the machine code level.

WebAssembly could be used as a backend for the normal Go compiler

@4ad
Copy link
Member

4ad commented Mar 3, 2017

@minux The lack of threads or async i/o seems to be a bigger problem than the ABI workarounds we'd have to do because of wasm quirks.

@sbinet
Copy link
Member

sbinet commented Mar 8, 2017

@minux, @cherrymui
could you post somewhere in more details what you did and achieved?
as you may know the go-interpreter community is considering writing an interpreter based on either LLVM or wasm bytecode.
(we may even have a GSoC student working on the wasm bytecode interpreter [1], so, just the "bytecode consumption" part, not the "bytecode production via some Go-based toolchain" one)

minimizing the amount of duplicated efforts and impedance mismatch between various components would be great.

@RaananHadar
Copy link

I highly recommend this as a strategic feature. Consider that in just a few years web programmers will flock in hordes to pick their language of choice to be compiled into web assembly. The sooner we join the race the better. Mozilla is pushing Rust hard as the web assembly language of choice...

@tiborvass
Copy link

tiborvass commented Mar 13, 2017

I second what @RaananHadar said. It would be a pity if the spec didn't suit Go's needs, especially given how Go's runtime can be unique.

EDIT: Apologies for the MeToo nature of this comment :) Thanks for the pointer Brad.

@bradfitz
Copy link
Contributor Author

Reminder: https://golang.org/wiki/NoMeToo

@vibhavp
Copy link

vibhavp commented Mar 13, 2017

The WebAssembly stack machine context consists of linear memory and an operation stack. IIUC, saving the VM context can possibly be done by saving the:

  1. Current PC, bytecode string
  2. Linear memory, and the
  3. Operation Stack

To a global list of saved contexts in the VM context struct, if any.

Thus, a setjump should simply restore these values, and continue with the interpreter loop as usual (This is similar to how the emacs bytecode VM implements signals: https://github.com/emacs-mirror/emacs/blob/master/src/bytecode.c#L785). An exception system can be implemented on the top of this, though I'm not sure how performant would this be.

Further, the webassembly spec mentions that the stack height at any moment in the bytecode is statically known, making all stack operations equivalent to operations on registers unique to each position on the stack. This paper describes a way to achieve such a stack -> register mapping for converting stack machine code to native code, allowing us to use setjmp/longjmp as usual.

Edit: What about using panic() and recover() with special WasmException{} values for signalling an exception? recover does the tough job of stack unwinding already, it shouldn't be hard to restore the interpreter loop after recovering from a caught exception.

@minux
Copy link
Member

minux commented Mar 13, 2017 via email

@vibhavp
Copy link

vibhavp commented Mar 13, 2017

IIUC, you can completely discard the WASM stack (according to the spec). An extendable and copyable go stack is still something that needs to be investigated.

@minux
Copy link
Member

minux commented Mar 13, 2017 via email

@SerkanSipahi
Copy link

@bradfitz @minux @vibhavp What is the current state of wasm in golang? Nothing has happened since March 13th ! will it be possible transform golang to wasm in the future? is there maybe any route-map of that?

@kanocz
Copy link

kanocz commented Apr 21, 2017

if somebody will decide to implement wasm support I can offer my time for non-global tasks

@bradfitz
Copy link
Contributor Author

@SerkanSipahi, nobody is working on this. This bug is marked LongTerm. If something starts happening, you'll see updates here.

@joeblew99
Copy link

joeblew99 commented Jun 3, 2017

A good roadmap ?

We have DELVE and GDLV for debugging golang with a GUI. It works well too on all Desktops and Perf is excellent.
https://github.com/derekparker/delve
https://github.com/aarzilli/gdlv

Now, GDLV is based on NUCULAR which is shiny with some nice abstractions.
https://github.com/aarzilli/nucular

SO i was thinking this would be a good basis to do what has already been done here:

There are 5 WASM go libaries now too:
https://github.com/search?l=Go&q=webassembly&ref=simplesearch&type=Repositories&utf8=%E2%9C%93

@techtonik
Copy link

NaCl is shut down - https://blog.chromium.org/2017/05/goodbye-pnacl-hello-webassembly.html
So this should probably be cleaned up - https://github.com/golang/go/tree/master/misc/nacl

@ianlancetaylor
Copy link
Contributor

I believe that NaCl is used by various parts of the Go ecosystem. That is, it's not used only by Chrome. And in any case that blog announcement is about PNaCl, which, although it provides functionality similar to NaCl, is actually a completely different product based on a different technology. So removing NaCl support from Go is premature at present.

Any in any case whether or not we remove NaCl from Go has nothing to do with whether we add support for Web Assembly.

@techtonik
Copy link

I would say that PNaCl is just an extension over NaCl to provide platform independent code https://developer.chrome.com/native-client/nacl-and-pnacl I asked about NaCl deprecation here - https://groups.google.com/d/topic/native-client-discuss/wgN2ketXybQ/discussion

If is possible to get list of those various parts of the Go ecosystem where NaCl is still used?
Going through it one by one will make transitioning to WebAssembly more interesting.

@4ad
Copy link
Member

4ad commented Jun 7, 2017

I would say that PNaCl is just an extension over NaCl

You can say whatever you want, but it doesn't mean you are right. They are completely different ISAs, NaCL is just the target ISA (x86/arm) with some restrictions, while PNaCl is a completely different ISA for an abstract machine.

Go has never supported PNaCL, and the Go NaCL port was never suitable for Chrome anyway. It had nothing to do with Chrome, it was not usable in the browser. Whether Chrome abandons NaCL or PNaCL has no relevance to Go whatsoever. Nada, nil, no relevance. How many more times this must be said? Of course, if NaCL is abandoned completely then Go will be forced at some point to abandon it too.

WebAssembly support in Go has absolutely nothing to do with NaCL. Go might or might not get WebAssembly support. If, or when it might get such support has no relevance to the state of the NaCL port.

@bradfitz
Copy link
Contributor Author

bradfitz commented Jun 7, 2017

Here is a very important part of the ecosystem where we use nacl: https://play.golang.org/p/MfJIq8wb5-

(that runs server-side, not nacl-in-a-browser)

@techtonik
Copy link

Whether Chrome abandons NaCL or PNaCL has no relevance to Go whatsoever. Nada, nil, no relevance.

And who will support sandbox loader and toolchain then? It is imported from Chrome project, which switched efforts to WebAssembly. Wouldn't it be wise to follow and see if Go playground can already be ported from NaCl sandbox to WebAssembly sandbox? It can also be run server-side.

@ianlancetaylor
Copy link
Contributor

I think everybody is in favor of supporting WebAssembly.

The time to discuss moving away from NaCl is after WebAssembly is fully working. There's no point to discussing it before then.

@vine77
Copy link

vine77 commented Jun 8, 2017

@ianlancetaylor, can you clarify "after WebAssembly is fully working"?

At a glance, it appears webassembly.org says "the initial version of WebAssembly has reached cross-browser consensus" and wasm is available in current or future browser versions from all browser vendors.

@ianlancetaylor
Copy link
Contributor

@vine77 I mean after WebAssembly is fully working for Go programs, at least as well as NaCl works today.

gopherbot pushed a commit that referenced this issue Jun 14, 2018
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates #18892
Updates #25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
Reviewed-on: https://go-review.googlesource.com/114197
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@agnivade
Copy link
Contributor

@neelance - Now that callbacks are in, I believe basic support is complete. Shall we close this bug or did you have something else in mind ?

@neelance
Copy link
Member

Yes, I think we can close this now. 🎉

@Jinex2012
Copy link

Great work!!! @neelance

@alecthomas
Copy link

This is awesome, great work @neelance!

@gopherbot
Copy link

Change https://golang.org/cl/120057 mentions this issue: doc/go1.11: mention GOOS/GOARCH values of WebAssembly port explicitly

gopherbot pushed a commit that referenced this issue Jun 22, 2018
It's more common to specify GOOS/GOARCH values in that order,
rather than the inverse. Fix the order.

Updates #18892.

Change-Id: I8551508599e019f6617dc007397b562c9926418d
Reviewed-on: https://go-review.googlesource.com/120057
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/120575 mentions this issue: cmd/dist: skip non-std tests on js/wasm

gopherbot pushed a commit that referenced this issue Jun 22, 2018
After the std tests, most of the rest of the tests aren't applicable
to js/wasm. (anything with -cpu=>1, cgo, etc)

Skip them all for now. We can incrementally re-enable them over time
as the js/wasm port is fleshed out. But let's get the builder column
black again so we can enable trybots and keep it black.

Updates #26014
Updates #26015
Updates #18892

Change-Id: I8992ed3888f598fa42273ce8646a32d62ce45b1d
Reviewed-on: https://go-review.googlesource.com/120575
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
@matthewp
Copy link

Is it appropriate to open an issue on reducing the file size (if that is even possible)? Or is that being tracked elsewhere?

@bradfitz
Copy link
Contributor Author

@matthewp, feel free to open a webassembly-specific size bug if you'd like. The general one is #6853.

@gopherbot
Copy link

Change https://golang.org/cl/120958 mentions this issue: net: re-implement built-in simulated network on JS and NaCl

@omac777
Copy link

omac777 commented Jul 3, 2018

Does the golang compiler have built-in control flow integrity(CFI)? I recently skimmed through the following to do with vulnerabilities in WASM prevented with CFI built-into the clang compiler:
https://www.fastly.com/blog/hijacking-control-flow-webassembly-program
https://github.com/trailofbits/clang-cfi-showcase/blob/master/cfi_vcall.cpp

@neelance
Copy link
Member

neelance commented Jul 3, 2018

Just skimmed it, but it seems to me as if the post says "WebAssembly is not a high level language such as JavaScript, but a low level language, so certain classes of bugs may apply if the source language (C or Go) allows them." No surprise there. This means that if Go would have such issues, they would also apply to other architectures, not only WebAssembly.

@randall77
Copy link
Contributor

WebAssembly is mostly about defending the browser from nefarious WebAssembly code. It is not so much about providing guarantees within the WebAsssembly instance (although there is some of that). As @neelance said, it is up to the LanguageX->WebAssembly compilers to provide any required safety guarantees.
Go does not have CFI. I don't think it would be easy - at the point where a method starts, we've already lost the vtable. We've even lost the fact that a vtable was used.
Go is vulnerable to this exploit, in cases where there is any parallelism. For now, at least, the WebAssembly Go port (and WebAssembly, full stop) does not have parallelism so this exploit is only theoretical. The WebAssembly folks do plan to implement parallelism at some point, though.

@neelance
Copy link
Member

neelance commented Jul 3, 2018

@randall77 How does this affect plain linux/amd64? Is the risk somehow larger for WebAssembly?

@randall77
Copy link
Contributor

The risk is the same regardless of architecture (with the exception that WebAssembly doesn't have threads yet, so it's actually zero for WebAssembly for the moment). Data races can be used to simulate unsafe, without importing unsafe.
It would be hard to exploit such a vulnerability. You either need to link in untrusted code, or somehow find and trigger a gadget in an existing binary that does a data race on an interface value and then invoke a method on the result.

@neelance
Copy link
Member

neelance commented Jul 3, 2018

Okay, so if your Go code has no data races, then it should be fine. Also, when running untrusted code with WebAssembly you can make sure that it can not escape the WebAssembly environment, even though it is able to have data races or simply use unsafe. Thanks for the interesting insights and sorry for this being slightly off-topic for this issue.

@omac777
Copy link

omac777 commented Jul 5, 2018

The more I examine this javascript usage from within golang, the more I perceive as an infection that will degrade the quality and the long-term maintenance of the golang code. Let me explain. Here is an example of a wonderful wasm golang app that does some impressive eye candy magic.

https://github.com/stdiopt/gowasm-experiments/blob/master/bouncy/main.go

The results are impressive. Delving furthing into the code that actually makes it happen though was disappointing because it require that every time you want to call something from the javascript side, it requires you to describe the name of the javascript function or the javascript value as a string AT EVERY TURN. As a result, there is no way for the golang compiler to check javascript correctness at compile time. It's "fly by prayer" at runtime. I would prefer to see golang functions/variables/types everywhere for long-term maintenance otherwise it defeats the purpose of using golang for wasm in my humble opinion. BTW I detest javascript and always have. It's too freestyle. Why is it that such a language has persisted in the web browsers? The answer escapes me.

Thank you for listening.

@geovanisouza92
Copy link

geovanisouza92 commented Jul 5, 2018

I agree that calling JS using strings isn't the better experience. Maybe we could generate Go interfaces from web platform specification IDL's. The downside is how to keep up with the spec, because depending on the browser it will run, the API's isn't available, while on the other side, new spec keep coming all the time as the web platform evolves.

@neelance
Copy link
Member

neelance commented Jul 5, 2018

@omac777 I agree with you. That's why I hope that there will be nice Go libraries around the JS low levels so most users will never have to touch syscall/js directly. Similar to the syscall package, it is ugly and almost no one uses it directly. ;-)

@dmitshur
Copy link
Contributor

dmitshur commented Jul 5, 2018

@omac777 For GopherJS, many people use high-level bindings for various browser APIs rather than the low-level js package directly. I suspect a similar approach will be most popular with Wasm too. I expect many of the GopherJS bindings will add support for Wasm in the same package, and new packages will be created as needed.

For reference, see https://github.com/gopherjs/gopherjs/wiki/Bindings, https://dmitri.shuralyov.com/talks/2016/Go-in-the-browser/Go-in-the-browser.slide#11 (and the following 4 slides), and dominikh/go-js-dom#57.

@guidorice
Copy link

Also we should consider there is a bunch of stuff on the webassembly roadmap:

https://webassembly.org/docs/future-features/

that will improve the syscall story quite a lot, in the long run, like

reference DOM and other Web API objects directly from WebAssembly code;
call Web APIs (passing primitives or DOM/GC/Web API objects) directly from WebAssembly without calling through JavaScript; and
efficiently allocate and manipulate GC objects directly from WebAssembly code.

Anyways, thanks @neelance et. al. for implementing the initial version. So awesome! 🥇

@fractalbach
Copy link

What if there was a Go implementation of the DOM?

Documents and scripts could be written and executed in Go, making use of the type system.

Then, code could be generated from the Go DOM.

@yookoala
Copy link

yookoala commented Jul 6, 2018

@fractalbach: If the WebAssembly's host binding is confirmed and implemented by the work group, you can actually do DOM manipulation through WebAssembly. Then it would be reasonable to have high level libraries to abstract DOM, Documents and scripts.

But before that, it is not quite as appealing to do.

@omac777
Copy link

omac777 commented Jul 6, 2018

Go WebAssembly: Binding structures to JS references

https://medium.com/@nlepage/go-webassembly-binding-structures-to-js-references-4eddd6fd4d23

The approach is appealing. The usage is similar to json marshallling/unmarshalling within structs. That makes it an easily transferable skill.

The one thing missing is how to apply the same approach to duck-typing? I would prefer not to have the function declarations within the structure, but outside of it standalone in the same manner as duck-typing. If the service exists, it can use it. That way it is easily modifiable without affecting the core struct holding the different desired non-function values. Duck-typing rocks!

Thank again @neelance for all your work. It's greatly appreciated.
Thank you Mr. Nicolas Lepage for your points of view. They really help to crystallize what could be a better path for interoperating with all this javascript noise for the interim until the direct wasm interfacing is complete.

@agnivade
Copy link
Contributor

agnivade commented Jul 6, 2018

Hello everyone, this is a high traffic thread with a lot of people subscribed. Having a general discussion like this is not fruitful to those just concerned about the original issue - which is to have WebAssembly support for Go.

Please feel free to continue the discussion in appropriate forums - golang-nuts/golang-dev. Or if you have something specific in mind, please open a new issue.

Thank you.

@golang golang locked as resolved and limited conversation to collaborators Jul 6, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-wasm WebAssembly issues NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests