-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test/bench/go1: benchmark is noisy on node/wasm #33749
Comments
any better way to check if my optimzation on WASM works as expected ? |
On what kind of machine, under what conditions? (Did you use CC @mvdan (for his related recent talks on benchmarking), @dr2chase (general interest in benchmarking), @neelance (WASM) |
On top of what @bcmills said: Is the machine idle? Have you tried randomizing the order of the benchmark executions? Is the machine throttling during any of the benchmarks? |
I'm not sure what we can do on the Go side. The WASM bytecode is run by V8 (or any other execution engine), which has a complex JIT system. |
my go1 test condition:
Actually I use this environment for go1's AMD64/i386 benchmark tests, whose results are OK. |
I agree that there isn't much that can be done on the Go side. The entire execution is run by V8 which can optimize or de-optimize code at will. We can run using perflock on a quiet machine and hope for the best. Perhaps also run with Node 10 to check if things are any better. @beenshi - Did you want anything else to be done on this issue ? |
Have you tried the way you suggested? Or even the result can be shown? Though we can do less on the go side, I expect to see a solid way to verify WASM backend's performance. |
I haven't tried with perflock, but on some optimization CLs, I have indeed noticed performance improvements with the Go1 benchmarks. Although I confess I haven't tried running the test multiple times with the same Go version. But what I was trying to say is that, personally, I don't see an actionable item that can be done in the code to address this. |
This does not seem to be actionable. I'm closing it for now. Maybe in the future some non-V8 runtime will give more stable results. |
Run the go1 benchmark twice, with the same go version.
(with node 8.16.0 and the recent master branch of go)
The text was updated successfully, but these errors were encountered: