Hello @artspb, thank you for reporting this issue and welcome to the Go project!

So I believe that os/exec is not supported on js/wasm and perhaps we could document this on that guide that you shared at https://github.com/golang/go/wiki/WebAssembly#executing-webassembly-with-nodejs
However, perhaps let's add a task for Go1.14 to add build tags to all the tools that invoke os/exec so:

+build !js,!wasm,!nacl

since os/exec doesn't fly for nacl as well?

Kindly paging @neelance too.

I don't expect cmd/test2json is supported under WebAssembly. We should either document that, or decide if it's worth trying to find a way to make it work (it can't use os/exec then, etc.).

I've never used test2json before so I'm not familiar with its details, but its documentation notes that:

Note that test2json is only intended for converting a single test binary's output. To convert the output of a "go test" command, use "go test -json" instead of invoking test2json directly.

@artspb Can you check if using go test -json approach can work for your needs?

There are some basic misunderstandings here. You cannot use a test2json ELF binary to execute a wasm test. Both needs to be compiled to wasm.

Adding $(go env GOROOT)/misc/wasm to $PATH doesn't help.

Yes, this is only for go commands to find the right exec binary for its tasks.

Here is what you should be doing -

$../misc/wasm/go_js_wasm_exec ./test2json -t  ./test.wasm -test.run=Simple
{"Time":"2019-06-11T09:25:16.313+05:30","Action":"output","Output":"test2json: pipe: not implemented on js\n"}
{"Time":"2019-06-11T09:25:16.315+05:30","Action":"fail","Elapsed":0.005}

Note that test2json is itself compiled to wasm.

go test -json also works and is the better way to run tests to avoid the os/exec issue.

Output from some sample tests -

$PATH=~/play/gosource/go/misc/wasm:$PATH GOOS=js GOARCH=wasm gotip test -json -run=Simple
{"Time":"2019-06-11T09:34:32.87285878+05:30","Action":"run","Package":"github.com/agnivade/shimmer","Test":"TestSimple"}
{"Time":"2019-06-11T09:34:32.872994865+05:30","Action":"output","Package":"github.com/agnivade/shimmer","Test":"TestSimple","Output":"=== RUN   TestSimple\n"}
{"Time":"2019-06-11T09:34:32.875067881+05:30","Action":"output","Package":"github.com/agnivade/shimmer","Test":"TestSimple","Output":"--- PASS: TestSimple (0.00s)\n"}
{"Time":"2019-06-11T09:34:32.875649484+05:30","Action":"pass","Package":"github.com/agnivade/shimmer","Test":"TestSimple","Elapsed":0}
{"Time":"2019-06-11T09:34:32.875680497+05:30","Action":"output","Package":"github.com/agnivade/shimmer","Output":"PASS\n"}
{"Time":"2019-06-11T09:34:32.903534819+05:30","Action":"output","Package":"github.com/agnivade/shimmer","Output":"ok  \tgithub.com/agnivade/shimmer\t3.036s\n"}
{"Time":"2019-06-11T09:34:32.903572952+05:30","Action":"pass","Package":"github.com/agnivade/shimmer","Elapsed":3.036}

I think documenting this in the wiki is the best way here.

Adding build tags may be fine, but it already gives a nice error when we try to run os/exec under wasm -
{"Time":"2019-06-11T09:25:16.313+05:30","Action":"output","Output":"test2json: pipe: not implemented on js\n"}

I think documenting this in the wiki is the best way here.

@agnivade Would you be up for that? You seem to have the clearest understanding of the problem + solution. 😄

Thank you, everyone, for your responses.

Can you check if using go test -json approach can work for your needs?

@dmitshur Unfortunately, this approach doesn't work for us as we want to debug with delve and produce JSON output at the same time. From what I know, there's no other solution but test2json. You can find more details in #23033.

Note that test2json is itself compiled to wasm.

@agnivade It doesn't sound like a scalable approach. But I found out that inverting order of arguments somehow solves the problem. When I run the following command everything works as expected.

GOOS=js GOARCH=wasm /usr/local/Cellar/go/1.12.5/libexec/bin/go tool test2json /usr/local/Cellar/go/1.12.5/libexec/misc/wasm/go_js_wasm_exec /Users/artspb/go/src/github.com/artspb/wasm-test-example/main.wasm -test.v -test.run '^Test_main$'

Am I missing something or is it the most convenient way to run a test binary so far except go test -json?

You are debugging a wasm binary using delve ? Is that even possible ?

#23033 was closed because there wasn't sufficient clarity on what the OP wanted.

But I found out that inverting order of arguments somehow solves the problem.

Ah yea, it's a roundabout way. What you are doing is essentially running the test with go_js_wasm_exec and converting the output to json. This is equivalent to -

~/go/misc/wasm/go_js_wasm_exec test.wasm -test.v 2>&1 | go tool test2json

If this is what you want, then go ahead. Although, I don't understand why go test -json does not work for you.

You are debugging a wasm binary using delve ? Is that even possible ?

Not yet (see go-delve/delve#1325) but it'd be great to be ready.

If this is what you want, then go ahead.

Not really. What I want is to run tests in a platform-agnostic way. Ideally, I should only provide GOOS and GOARCH and Go should figure out the rest for me. But it seems to be impossible at the moment.

Although, I don't understand why go test -json does not work for you.

The whole situation is very well described here. Long story short, it's impossible to run dlv test -json to have a debug session and JSON-output at the same time. Please correct me if I'm wrong.

You are misunderstanding how go tool <x> operates. go tool <x> will always pick up x from the platform in which the go binary was compiled in. So if you run go tool compile, and go is a linux/amd64 binary, then it will always pickup compile from pkg/tool/linux_amd64/compile. Regardless of what GOOS and GOARCH values you supply.

Example -

GOOS=js GOARCH=wasm gotip tool -n vet foo.go
/home/agniva/play/gosource/go/pkg/tool/linux_amd64/vet foo.go

So then coming to your example, if you are doing go tool test2json, it will invoke test2json for linux/amd64 and thereby hit the exec format error.

This is by design. If you have suggestions, you are welcome to open a new issue.

To conclude, this is not a wasm specific issue. Even a binary compiled in darwin won't work with a go toolchain in linux/amd64. The easy way is to use go test -json, but since that does not work for you, you have to unfortunately jump some hoops.

Again, there is no bug here. Everything is working as expected.

Thank you for the information.

The easy way is to use go test -json

Still, one has to provide exec to run WASM tests, am I right? Is there a way to eliminate this step if GOOS and GOARCH are provided?

Yes you have to. GOOS and GOARCH are just the target values. But how does the compiler know the file path to your exec binary ? It can be anywhere. Again, this is not a wasm issue. You have to set the exec param whenever you are doing cross-platform tests.

Either set the PATH to go_js_wasm_exec, or set the -exec flag.

Fair enough. Thank you again for your patience explaining how it works.

You are debugging a wasm binary using delve ? Is that even possible ?

Possibly interestingly, that's an area I've been investigating recently from a "how to get this working" point of view. 😉

Delve has a number of backends that can be run, with the specific one for a given situation depending on the OS and what the debugging user requests: https://github.com/go-delve/delve/blob/master/service/debugger/debugger.go#L156-L171

To add support for Wasm, a backend for Wasm would need adding. I've been investigating modifying the Life VM so it could fit that task, though the wagon VM looks like it may be a better choice (being investigated).

eg User runs Delve in a session (via GoLand/IntelliJ or whatever), Delve spins up the Life or wagon wasm VM in the background with the given Wasm loaded.

While not completely simple, it appears do-able for wasi style Wasm. eg stuff that doesn't interact with a browser.

No idea yet how browser dev tools work, Wasm that interacts with a browser... is an unknown (for now).

Might be possible to get the modified Life or Wagon VM to communicate with a browser debug session, but that's just pure concept guessing without any research at all behind it. eg might not actually work.

No idea (yet), and am instead looking into other aspects of things first. 😉

That's great @justinclift.

I will close the issue as this was a misunderstanding of how go tool test2json operates and not related to wasm. Trying to use os/exec in wasm already gives a nice error - "pipe: not implemented on js".

I am not sure what else should I add in the wiki to explain further than this. There are a lot of things that don't work in wasm, should we then mention all of those ? Not sure. If anyone has a better idea, please feel free to update the wiki.