@neelance this might interest you as well.

The buffer of the typed array is the whole WebAssembly memory. You take a slice of that buffer and then pass it to postMessage. I'm not sure how postMessage is supposed to work on buffers, but it is most likely the culprit. I guess it serializes the whole buffer, not only the slice. You should try to actually create a second buffer with only the data you need.

@neelance - Feel free to comment if you want to prioritize it for 1.11.

If this is really due to postMessage, then it is working as intended from Go's point of view. postMessage shouldn't be used that way. @leidegre needs to evaluate if this is really the case, then we can close.

I should probably have emphasized this a bit more but the issue is more subtle than postMessage. For example, if you do console.log on a slice, you run into the same issue and I suspect that there are additional edge cases lurking around the corner. If you trigger this behavior your whole app goes up in the air.

For example, take this program as an example

package main

func main() {
	var x []byte
	println(x)
}

You run that 3 or 4 times and the process hosting the page hard faults with an out of memory error and it is not limited to []byte an println([]int{}) is just as dangerous and please note that this behavior is not limited to WebWorker processes.

@neelance what do you think people will be doing with this, as soon as it's in the offical Go release? I think, at the very least, that this warrants a disclaimer. People will probably run into inexplicable crashes and having some guidance on this behavior accompanying the release, could be a good idea. Save some headaches.

I should also emphasize again that this workaround, where the slice is first explicitly copied into a memory segment managed by the JavaScript GC works really well. It doesn't have the same problem when passed to for example console.log or postMessage

[worker.js]

// I changed the name from slice to copy, becuase it is copying a portion of the ArrayBuffer
function copy({ byteOffset, byteLength, buffer }) {
  return buffer.slice(byteOffset, byteOffset + byteLength)
}

[worker.go]

var x []byte
x = append(x, 2)
x = append(x, 3)
x = append(x, 5)
y := js.TypedArrayOf(x)
z := global.Call("copy", y)
y.Release()

global.Call("postMessage", z) // from WebWorker postMessage

Additionally, the reason why I'm doing this, like this is that I'm having my Go program do expensive crypto stuff, stuff that isn't supported by any other major npm crypto package that I know of. Being able to leverage the Go standard library has tremendous value in this case. Also, there doesn't seem to be an obvious way to build a dynamic object in Go and have that pass the boundary, i.e. postMessage so I opted to pass already serialized JSON and use JSON.Parse on the browser receiving side. Since the json.Marshal Go function return []byte, error it was just natural to do it this way. If there is a standard way of serializing data to JSON in the library, I suspect, people will use it but you end up with []byte not string, if you convert this value to string before you send it, you avoid the issue entirely but if you don't you end up running out of memory very quickly.

You run that 3 or 4 times and the process hosting the page hard faults with an out of memory error and it is not limited to []byte an println([]int{}) is just as dangerous and please note that this behavior is not limited to WebWorker processes.

I can not reproduce this on Chrome, Firefox or Safari.

@neelance that's new. What build where you using? I can test on additional systems. Also I will put up my repo somewhere for you to try.

@neelance is this Windows specific?

Exact code I used

https://gist.github.com/leidegre/2fe06162eae09837ab533d6acf5d90e6

Git clone/download as zip then npx serve . goto http://localhost:5000/wasm_exec.html

Click Run button a couple of times and I get "Aw, Snap".

Chrome Version 67.0.3396.99 (Official Build) (64-bit) (cohort: Stable)

@neelance I promise you, I'm not making this up

@neelance watch the "Committed" value after I open the devtools. I managed to crash another processes on the system as well because I ran out of virtual memory.

Firefox 61.0 (64-bit)

I can observe similar behavior on FF linux. There is a linear growth as you click on the Run button

The 2 snapshots above have been taken respectively after the first click and a after few hundred clicks.

I did a quick test with Debian (4.9.82-1+deb9u3) and Chromium 66

As I click run, the virtual memory expands to the point I hit my limit, which is 16 GiB, at that point I get an out of memory error and subsequent attempts to run the program fail with the error message shown in the log.

I'm not trying to point out a flaw with the JS/WASM approach, simply that this is an issue that people will run into and it will cause all kinds of weird faults, which running out of virtual memory does. I think we need to be mindful of when values mapped in memory in Go, traverse a boundary like this. A simple solution is to introduce copying so that the ArrayBuffer matches perfectly the value where needed would work. I think this is something the JS/WASM backend can do.

Change https://golang.org/cl/122296 mentions this issue: misc/wasm: free up memory on exit

Unfortunately I still can't reproduce this on my OS X machine, Chrome 67.0.3396.99.

I just used Chrome's memory profiler to look into how many WebAssembly instances are allocated at any given time. The CL above makes it so this number does not go above 1 on my machine. Would you mind looking into the memory profiler on your machine to see if it shows something different?

I believe that It happens on your machine, but I am still not convinced that this is related to the JS/Go boundary. Have you tried a fully empty program, without any println instruction?

@neelance I have not tried a fully empty program with any println code. My belief is that that wouldn't cause an issue. Overall, I'm not having any problems with the JS/WASM tooling, things work really well but I'm purposely avoiding the things that cause the linear memory map to be copied.

This is 100% workable, it's just that if you happen to touch these builtin functions or pass any kind of slice over a JS boundary where copying happens you get this huge memory hit which will cause all kinds of crazy. All this is, is a nasty edge case that the user may run into.

To the extent that we can prevent the user from running into this issue, I think it would be prudent to do so. Unfortunately, I do not know how you could possibly detect that the slice you are about to pass is going to be copied in the most unfortunate manner possible.

I guess you could deal with this in the js.Value type, somehow, but that would imply that this copying should take place all the time you cross a JS/Go boundrary, regardless of where it is actually needed. Until a better solution is available, I'm perfectly fine with leaving this issue as-is or closing as "working as intended" but with some action to include this in a known issue release note, or maybe a blog post. It's an edge case, a nasty one that I just know is gonna put people off and the kind of thing that I've not had to deal with when working with Go in the past. It's very Go unlike, even if it's JavaScript and the whole browser ecosystem's fault.

@neelance maybe we should bring this issue to the browser people?

On Win10/64 bit with Edge everything works fine, no crash after a lot of clicks.

Here my Browser version:
Microsoft Edge 41.16299.492.0
Microsoft EdgeHTML 16.16299
with enabled about:flags "Enable experimental Javascript features"

For Chrome 67 I can reproduce the crash (after 4 clicks).

@leidegre Are you still able to reproduce this with the latest Chromium?

@neelance not with latest build of Chromium no, but I can reliably crash Chrome 69 and 70.

So, it is a browser thing?

• Version 72.0.3585.0 (Developer Build) (64-bit) works.
• Version 70.0.3538.67 (Official Build) (64-bit) does not work

Yes, it is. Is it okay to close this?

Sure, but until this is fixed in all browser people will be crashing hard. Hopefully they'll find this.