New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
math: make benchmark iterations more data-dependent #33441
Comments
I think we should be careful doing this, at least when measuring the speed of operations that are only a couple of instructions:
One possible solution is to add a small array of input values (with a length of a power of 2 to avoid division instructions) and iterate over those for throughput benchmarks. The cost of the indexing and load should be low. Then add a separate latency benchmark for operations where we are particularly interested in latency (probably only the single instruction operations such as FMA, Sqrt, Abs, Copysign etc.). |
This CL is probably the better solution, more robust as runtime.KeepAlive is basically guaranteed not to get optimizted out: https://go-review.googlesource.com/c/go/+/188437 |
If there's no boxing that happens with runtime.KeepAlive, then @nsajko's CL is probably a reasonable option. Especially because as @mundaym points out, we would have to first differentiate and duplicate the benchmarks based on the metric we care about (latency vs throughput), and then define additional input values for each of those benchmarks to avoid reaching a steady state. That seems like a lot of work for little return. |
As discussed in CL 127458, benchmarks should be updated to make each iteration dependent on the result of the previous. This reduces the chance of work done in the loop being optimized away, as well feeding the benchmarks a larger input space. For instance, the FMA benchmark was changed from
to
The text was updated successfully, but these errors were encountered: