Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: provide some way to bundle raw JavaScript during wasm compilation #27626

Closed
flimzy opened this issue Sep 11, 2018 · 10 comments
Closed

cmd/go: provide some way to bundle raw JavaScript during wasm compilation #27626

flimzy opened this issue Sep 11, 2018 · 10 comments
Labels
arch-wasm WebAssembly issues FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@flimzy
Copy link
Contributor

flimzy commented Sep 11, 2018

What version of Go are you using (go version)?

Go 1.11

Does this issue reproduce with the latest release?

Yes

What did you expect to see?

GopherJS supports the option to bundle raw JavaScript in the compiled output by including files named *.inc.js in the Go source directory. Go's wasm support doesn't offer anything comparable, as far as I am aware.

What did you see instead?

I would like to see the functional equivalent included when compiling to a wasm target.

I say functionally equivalent, because identical might be dangerous, in that it could make isomorphic GopherJS/GoWASM packages difficult, if there are ever incompatibilities. It is likely that one might need a particular *.inc.js file for GopherJS support, and a separate one for Go/WASM support. And at least at present, build tags in these files are broken for GopherJS (see gopherjs/gopherjs#468), so unless that is resolved, or some other mutual-exclusion scheme is provided, the simplest path forward may be to use an entirely different convention for Go/WASM.

No doubt, many lessons from CGO can also be applied here, but this is an area where I have no direct experience.

@flimzy flimzy changed the title cmd/compile: Provide some way to include raw JavaScript during wasm compilation cmd/compile: Provide some way to bundle raw JavaScript during wasm compilation Sep 11, 2018
@ianlancetaylor ianlancetaylor changed the title cmd/compile: Provide some way to bundle raw JavaScript during wasm compilation cmd/compile: provide some way to bundle raw JavaScript during wasm compilation Sep 12, 2018
@ianlancetaylor ianlancetaylor changed the title cmd/compile: provide some way to bundle raw JavaScript during wasm compilation cmd/go: provide some way to bundle raw JavaScript during wasm compilation Sep 12, 2018
@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 12, 2018
@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Sep 12, 2018
@ianlancetaylor
Copy link
Contributor

This seems like an area where it would help to see a specific proposal.

@ianlancetaylor ianlancetaylor added the arch-wasm WebAssembly issues label Sep 12, 2018
@myitcv
Copy link
Member

myitcv commented Sep 12, 2018

Drive-by thought.

Is there some trick that can be performed here using eval in JavaScript? You'd want to avoid any sort of string encoding conversions, i.e. have pre-compiled the correct encoding within the .wasm output, but feels like it would be possible?

That way we could avoid the go tool needing to do anything special; a go generate-r could do the bundling and encoding, with an init()-time function then being responsible for calling eval.

@flimzy
Copy link
Contributor Author

flimzy commented Sep 12, 2018

Is there some trick that can be performed here using eval in JavaScript?

If I'm not mistaken, this would require that any such programs be executed with a potentially dangerous content security policy.

@myitcv
Copy link
Member

myitcv commented Sep 12, 2018

@flimzy seems to work unless I'm missing something really obvious?

cd $(mktemp -d)
cat <<EOD > main.go
package main

import (
	"syscall/js"
)

func init() {
	js.Global().Call("eval", "(typeof window !== 'undefined' ? window : global).banana = (function() { console.log(\"hello, world\") })")
}

func main() {
	js.Global().Call("banana")
}
EOD
GOOS=js GOARCH=wasm go build -o main.wasm main.go 
$GOROOT/misc/wasm/go_js_wasm_exec main.wasm

gives:

hello, world

(as it does in the browser)

@flimzy
Copy link
Contributor Author

flimzy commented Sep 12, 2018

@myitcv: I believe the default CSP is wide-open, allowing eval. But if you tighten down security, you'll typically disallow the unsafe-eval option, which would prohibit that, I believe.

@myitcv
Copy link
Member

myitcv commented Sep 12, 2018

@flimzy got it, thanks. In which case, doesn't that preclude the loading of any JavaScript from within a WASM context?

@flimzy
Copy link
Contributor Author

flimzy commented Sep 12, 2018

@myitcv I don't know the answer to that.

@flimzy
Copy link
Contributor Author

flimzy commented Sep 12, 2018

I've just read the CGO introduction, to familiarize myself with the approach taken there. Based on that, my more specific proposal would be the following:

A new special import package (akin to import "C") would be added. Name to be determined, but for the purpose of this proposal, let's assume we use import "WASM".

When the Go tool sees this special import is used in a package, then any *.js files in the same directory will be included in the wasm output. At runtime, this code is to be executed before package initialization (that is to say, the Go code should be able to safely assume that all included *.js files have been successfully executed).

The contents of each *.js file should be wrapped in a context to provide access to a special this variable. Exmaple:

(function() {
    // Contents of a single *.js file here
}).call(goWASM)

This goWASM variable would then be accessible from Go via the WASM package.

This allows the included *.js files to define variables or functions which can be accessed from Go, without polluting the global namespace.

A simple example:

main.js:

this.a = 123;
this.b = function(x) { console.log(x) };

main.go:

package main

import "WASM"

func main() {
    this := WASM.This // WASM.This is just an instance of js.Value; its contents defined by foo.js above
    a := this.Get("a").Int() // int(123)
    this.Call("b", "some text") // logs "some text" to the console        
}

Build constraints for the *.js files ought to be followed, as per usual rules.

This differs from GopherJS's implementation in two key ways:

  1. *.js files would not be included automatically, but rather only if the WASM package is imported. This would be more in-line with CGO's behavior.
  2. Each *.js file would be executed with access to a package-specific this variable, exposed to Go via WASM.This. In GopherJS, this was the global object, requiring any special variables to pollute the global namespace.

A few additional thoughts I haven't fully considered:

  • The import package name (WASM above) and the single exported symbol (WASM.This) should be considered. No doubt better alternatives for one or both exist.
  • Execution order of *.js files, when there are multiples, may be important, so should probably be defined.
  • This could break compatibility with GopherJS (as mentioned in the original issue description above). To resolve this, go could ignore *.inc.js files, as a special case.

@neelance
Copy link
Member

The Go compiler generates wasm binaries. The only way to include JavaScript code in a wasm binary is to use eval, but it is incompatible with a secure CSP. If you accept this, then you can use eval right now, there is no extra support necessary in the Go compiler.

I see no simple way on how something like the *.inc.js files of GopherJS can be possible with wasm. The logic you describe above might be possible, but I think it should not be part of the Go compiler itself. The best way would probably be to use require with some existing JS bundler and maybe have this JS bundler scan for .js files in the Go tree.

@neelance
Copy link
Member

neelance commented Jan 6, 2019

CGO consumes C code, but still emits a single machine code binary. The same is not possible with JavaScript, since it can not be compiled to WebAssembly. The Go compiler only emits a WebAssembly binary, which may or may not run on a JavaScript host. Also emitting a JavaScript file would be the job of a web application bundler, not the Go compiler itself. Closing this as out of scope for the compiler itself. An implementation with eval is still possible on the application level.

@neelance neelance closed this as completed Jan 6, 2019
@golang golang locked and limited conversation to collaborators Jan 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-wasm WebAssembly issues FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants