Similar Issues

• cmd/test2json: 'exec format error' for os/exec when exec is set to go_js_wasm_exec #32442
• path/filepath: WebAssembly uses *nix conventions on Windows #43768
• x/pkgsite/internal/tests/deps: TestCmdPkgsiteDeps failing on js/wasm due to lack of `exec` support #64942
• cmd/go: GOOS=js GOARCH=wasm go build -o [non-empty parent dir] ./... fails on Windows host #37472
• os: invalid tmp dir for GOARCH=wasm, GOOS=js when GOHOSTOS=windows #27306

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

It looks like an intentional decision in CL 470676 that the module toolchain isn't complete to run all tests, as it doesn't keep the "test" directory either:

https://cs.opensource.google/go/go/+/master:src/cmd/distpack/pack.go;l=206-214;drc=dbe03e4831206d7311e4423c3a988fc48beda2bd

The scripts in the misc/wasm directory can be fetched from the source or binary toolchain distributions.

I agree with this, but misc/wasm is not used only for testing, on the contrary, it is a mandatory runtime file.
Of course we could choose to parse the VERSION file and get the corresponding version from the internet, but that requires additional complexity. And these files are not large enough to make a significant impression on the size of the final archive.

@dmitshur - Just chiming in here. wasmbrowsertest needs access to the wasm_exec.js to run the wasm runtime. Not having the file included inside GOROOT is a problem for the tool to run the tests.

Downloading the file during running a test isn't a great experience either because one should be able to run tests without any internet connection.

Wondering if this decision can be revisited and the misc/wasm directory can be included? As mentioned before, the file sizes are fairly small and should not make any impact size wise.

cc @rsc for a decision as well.

misc files should not be included in toolchains, full stop. misc is by definition a dumping ground.

If there are required supporting files that should be included in toolchains, they should be moved somewhere else. Perhaps lib/wasm.

Did this move needs a proposal?

If there are required supporting files that should be included in toolchains, they should be moved somewhere else. Perhaps lib/wasm.

Happy to do that.

cc @neelance for alignment on a decision. Richard, please see the thread above for some context.

Moving wasm_exec.js sounds good to me. Please submit a CL if you can Agniva.

Oh, there was a thumbs up from Richard. Never saw that until now. 😄 Will do.

Change https://go.dev/cl/604696 mentions this issue: misc/wasm: move wasm runtime files to lib/wasm

Change https://go.dev/cl/604119 mentions this issue: WebAssembly: update support file location

Change https://go.dev/cl/604835 mentions this issue: all: add support for lib/wasm folder

I'm a little concerned that this will make any posts explaining how to use Wasm with Go outdated. We'll at the very least want to update the official wasi blog post (https://go.dev/blog/wasi) and Wasm wiki (https://go.dev/wiki/WebAssembly) to coincide with the release of this change. There might be other places too. I'm not sure how to best communicate this change to the Go Wasm community at large though.

Maybe we could keep a soft link in misc pointing to lib/wasm?

If that's something we can support, maybe, but I don't know if we have precedence for that sort of thing, or even capability for all releases.

I agree that unless it's strictly necessary it doesn't need to be in the toolchain. Only wasm_exec.js and wasm_exec_node.js are strictly necessary to execute the Go-compiled Wasm. The exec wrappers are only necessary to execute the code in specific environments. They are not necessary in the browser, for example. Adding these files would be a convenience, not a necessity.

I'm still confused about how go toolchain users can access these exec wrappers.

The exec wrappers are just conveniences. Users can write their own exec wrappers if they need something. Not every user will need an exec wrapper to execute the compiled code.

So we need to updates many document to indicate that these scripts are not avaiable while using GoToolChain?

Uses of export PATH=$PATH:$(go env GOROOT)/misc/wasm go run will not work reliably when using other Go toolchains, yeah. I think that's probably fine.

Hey all, just wanted to check in on this. Thank you @Zxilly for dilligently addressing the review comments.

@cherrymui - It sounds like adding further documentation updates for users depending on the default exec wrappers should be acceptable. Can we move forward with the changes then?

I moved this issue to NeedsDecision to better reflect its latest state, given the discussion here and on CL 604696. To expand on that: it seems there's agreement that the wasm_exec.js and wasm_exec_node.js files should be included in toolchain zips, and that the wasm_exec.html example should stay in misc/wasm (and not be included in toolchain zips). The remaining question is about the go_*_wasm_exec programs that help with running GOARCH=wasm code via go run and testing it via go test.

There are two paths forward being currently discussed, as I understand them:

1. Leave the exec programs out of toolchain zips. Update documentation that refers to those exec programs to clearly say that they're not supported with downloaded toolchains (https://go.dev/doc/toolchain#download), only with local toolchains (e.g., GOTOOLCHAIN=local) or toolchains found in path (e.g., GOTOOLCHAIN=local+path).

2. Make the exec programs available in toolchain zips, in the lib/wasm directory.

Let's get to a decision here, and then the CL can be updated to reflect that. Thanks.

Thanks Dmitri. Sounds to me like approach 1 is just going to make life a bit more painful for folks working with downloaded toolchains? What exactly are we gaining by not including the exec wrappers? I thought the original issue was more about renaming misc/wasm to lib/wasm.

To me, it feels like we have little to lose, but more to gain by including those files. But will defer to others here.

I chose option 2 because GoToolchain is more of an implicit mechanism. If a user doesn't know about GoToolchain, it will be difficult for him to understand why files present in his Go installation are not accessible.

It seems most people think it helpful to include the exec wrappers. I think that is fine.

One difference for the exec wrappers are that, unlike most things included in the release which are ready to be used on production, the exec wrappers are mostly intended for local uses. I guess it is not much of a problem given the nature of the exec wrappers? And perhaps the convenience of using the exec wrapper with automatic toolchain switch overweighs.

Yep exactly. IMO, it's an overall net gain.

Based on discussion above, it sounds like there's agreement: for Go 1.24, the .js files are moved to lib/wasm, as well as the exec scripts. The .html file isn't moved. It's a bit unusual to have executables in a lib directory but it shouldn't cause problems (and if there's a good reason to find a better long term location for them, that can still happen in a future issue). This makes it possible to refer to files in $(go env GOROOT)/lib/wasm directory for the currently selected Go toolchain even after using https://go.dev/doc/toolchain#download.

Let's give this a bit more time and move it to NeedsFix if there aren't new concerns.

I'm a little concerned that this will make any posts explaining how to use Wasm with Go outdated. We'll at the very least want to update the official wasi blog post (https://go.dev/blog/wasi) and Wasm wiki (https://go.dev/wiki/WebAssembly) to coincide with the release of this change. There might be other places too. I'm not sure how to best communicate this change to the Go Wasm community at large though.

Once this change lands, we can address the documentation concern by creating a 1.24 release blocking issue to update the documentation along with the release (bloc post, wiki, etc).

What's more concerning is that this doesn't work right now

Filed #69175 to track the remaining work of updating relevant documentation.

why not just include the exec js without breaking all the blog and makefiles out there by changing the location? (ie stay where it’s been but only pick the needed files instead of all of misc/)