Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: implement timers, JavaScript callbacks back into WebAssembly #25506

Closed
bradfitz opened this issue May 22, 2018 · 24 comments
Closed

runtime: implement timers, JavaScript callbacks back into WebAssembly #25506

bradfitz opened this issue May 22, 2018 · 24 comments
Labels
arch-wasm WebAssembly issues FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Milestone

Comments

@bradfitz
Copy link
Contributor

After some discussion last week with @cherrymui, @rsc, @aclements, et al, we decided that users would be really sad if Go 1.11 was released with WebAssembly support and then realized they couldn't really use it in a browser well, without working timers & callbacks back into Go.

@cherrymui did a little prototype of callbacks and it was enough to show a demo of a Go webassembly program updating images in the DOM and responding to button clicks.

So we decided it was worth spending a bit of time and making sure the wasm support is what users will expect for Go 1.11.

@neelance, what's your availability look like to finish that in the next month or two?

I see you updated your branch (https://github.com/neelance/go/commits/wasm-wip) 3 days ago with neelance@773eb46

Do you need any help? @cherrymui and perhaps @aclements could help if needed.

Please use this issue number in any related CLs. Thanks!

@bradfitz bradfitz added NeedsFix The path to resolution is known, but the work has not been done. release-blocker labels May 22, 2018
@bradfitz bradfitz added this to the Go1.11 milestone May 22, 2018
neelance added a commit to neelance/go that referenced this issue May 23, 2018
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
@gopherbot
Copy link

Change https://golang.org/cl/114197 mentions this issue: runtime, sycall/js: add support for callbacks from JavaScript

@neelance
Copy link
Member

Thanks for offering the help! Callbacks are already working quite well. I just cleaned up the commit some more and pushed the CL for initial feedback/review: https://go-review.googlesource.com/c/go/+/114197

Do we also want to use this to get support for http.Transport into this release? There is already a POC at https://github.com/johanbrandhorst/fetch.

@bradfitz
Copy link
Contributor Author

An http.DefaultTransport value implementing http.RoundTripper I assume? SGTM for Go 1.11. Let's do it.

If the author wants to send it via a PR or CL I can review. (PRs get converted into Gerrit these days)

@dmitshur
Copy link
Contributor

dmitshur commented May 23, 2018

I'm happy to offer help with implementing and/or reviewing here as well.

I'm the author of the original Fetch API-based http.RoundTripper implementation in GopherJS that https://github.com/johanbrandhorst/fetch is based on, so I'm very familiar with how it works.

I've reached out to @johanbrandhorst and I'll coordinate next steps (regarding http.DefaultTransport) with him.

@johanbrandhorst
Copy link
Member

@shurcooL I'll try and get something in as soon as I can. I'm thinking we should implement the XHR transport as well and maybe make that the default as even among browsers that support WASM, they don't all support Fetch out of the box.

@neelance
Copy link
Member

fetch can be polyfilled and services like https://polyfill.io/ make it trivially easy to do so efficiently. I nowadays always prefer the newer (stable) APIs + polyfills.

@bradfitz
Copy link
Contributor Author

@neelance, in general I agree with that, but note that we want Go WebAssembly to work for users without the users needing to write or deal with any JavaScript. So unless that polyfill is part of some Go JS library, I'd rather we just fall back to an XHR RoundTripper if fetch is unavailable.

@neelance
Copy link
Member

@bradfitz This is a slippery slope. What about all the ES6 features that wasm_exec.js is using?

@neelance
Copy link
Member

For example fetch is supported by older browsers than async/await.

@bradfitz
Copy link
Contributor Author

@johanbrandhorst, @neelance, which browsers support WebAssembly & the ES6 features we're depending on but do NOT support Fetch?

If zero, then obviously we don't need XHR support.

https://caniuse.com/#feat=fetch suggests everything supports Fetch that already supports wasm.

@neelance
Copy link
Member

Oh yes, working on WebAssembly such much made me take it for granted. ;-) Yes, it looks like all browsers that support WebAssembly also support Fetch. No XHR needed.

@neelance
Copy link
Member

Same goes for async/await and the other ES6 features.

@dmitshur
Copy link
Contributor

As far as I know, all modern browsers support the basic Fetch API by now, but not all of them support streams, which are required for streaming the response body. E.g., see https://caniuse.com/#search=Streams. Firefox has javascript.options.streams and dom.streams.enabled flags that can enable it.

I think it should still be possible to use Fetch for those browsers, and avoid using Streams when they're not available.

neelance added a commit to neelance/go that referenced this issue May 24, 2018
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
@bradfitz bradfitz added the arch-wasm WebAssembly issues label May 24, 2018
@johanbrandhorst
Copy link
Member

Great, that sounds fine. I'll put together a PR as soon as I can (I want to try the PR workflow).

@gopherbot
Copy link

Change https://golang.org/cl/114515 mentions this issue: net/http: add js/wasm compatible DefaultTransport

gopherbot pushed a commit that referenced this issue May 30, 2018
Adds a new Transport type for the js/wasm target that uses the
JavaScript Fetch API for sending HTTP requests. Support for
streaming response bodies is used when available, falling back
to reading the entire response into memory at once.

Updates #25506

Change-Id: Ie9ea433a1a2ed2f65b03c6cc84a16e70c06fcf5c
GitHub-Last-Rev: 6df6467
GitHub-Pull-Request: #25550
Reviewed-on: https://go-review.googlesource.com/114515
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@johanbrandhorst
Copy link
Member

With https://golang.org/cl/114515 merged, we didn't quite finish the discussion opened in https://go-review.googlesource.com/c/go/+/114515#129. I'd like to revisit and implement that discussion as the existing implementation may break some use cases. How can we identify if a test binary is running?

@bradfitz
Copy link
Contributor Author

@johanbrandhorst, I can think of hacky ways to do it, but I'm not sure we have a supported way. For this application in particular I think we can look at os.Args[0] and see if it's "node".

@johanbrandhorst
Copy link
Member

That actually makes a lot of sense since Fetch isn't supported natively by Node anyway. I'll get a CL up.

@gopherbot
Copy link

Change https://golang.org/cl/115495 mentions this issue: net/http: use fake Transport network when running in Node

gopherbot pushed a commit that referenced this issue May 31, 2018
Replaces the existing local loopback check with a check to see
whether the program is being interpreted by Node. This means
tests that are run with Node will use the fake network while still
allowing users who are using js/wasm to talk to local networks.

Updates #25506

Change-Id: I8bc3c6808fa29293b7ac5f77b186140c4ed90b51
GitHub-Last-Rev: 43d26af
GitHub-Pull-Request: #25663
Reviewed-on: https://go-review.googlesource.com/115495
Reviewed-by: Agniva De Sarker <agniva.quicksilver@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
neelance added a commit to neelance/go that referenced this issue Jun 3, 2018
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
@neelance
Copy link
Member

neelance commented Jun 4, 2018

The net/http package does not compile any more:

src/net/http/roundtrip_js.go:23:10: undefined: t
src/net/http/roundtrip_js.go:200:11: undefined: errClosed
src/net/http/roundtrip_js.go:258:11: undefined: errClosed

Please fix.

neelance added a commit to neelance/go that referenced this issue Jun 4, 2018
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
@johanbrandhorst
Copy link
Member

My bad, I'll sort it.

@gopherbot
Copy link

Change https://golang.org/cl/116076 mentions this issue: net/http: fix build errors on js/wasm target

gopherbot pushed a commit that referenced this issue Jun 4, 2018
The in-progress WASM port does not yet have sufficient automatic
testing performed against it, so these errors slipped through when
adding the new Fetch API backed http.Roundtripper.

Updates #25506

Change-Id: I84c5832452e3e6067a02d926f67d01aaca66b837
GitHub-Last-Rev: 064062b
GitHub-Pull-Request: #25714
Reviewed-on: https://go-review.googlesource.com/116076
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
neelance added a commit to neelance/go that referenced this issue Jun 4, 2018
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
neelance added a commit to neelance/go that referenced this issue Jun 12, 2018
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
neelance added a commit to neelance/go that referenced this issue Jun 13, 2018
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
neelance added a commit to neelance/go that referenced this issue Jun 13, 2018
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
neelance added a commit to neelance/go that referenced this issue Jun 13, 2018
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
neelance added a commit to neelance/go that referenced this issue Jun 14, 2018
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
gopherbot pushed a commit that referenced this issue Jun 14, 2018
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates #18892
Updates #25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
Reviewed-on: https://go-review.googlesource.com/114197
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@agnivade
Copy link
Contributor

@neelance - Just checking, is there anything else pending on this that you wanted to do ?

@neelance
Copy link
Member

I think we can close this issue. I only want to add some minor additions to the API for convenience (e.g. an easy way to create a promise).

@golang golang locked and limited conversation to collaborators Jun 18, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-wasm WebAssembly issues FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Projects
None yet
Development

No branches or pull requests

6 participants