Interesting. cc @aclements @mpvl for thoughts. Tentatively marking this Go 1.8.

What is the API you are looking for?

ping @benburkert

I think it would mostly consist of a (*testing.B).Report(testing.BenchmarkResult) method for reporting a sub benchmark. That plus an additional Name field on testing.BenchmarkResult should satisfy my needs:

https://play.golang.org/p/QavnfPoTUr

Oh, this isn't at all what I thought you were proposing. I thought you wanted custom metrics (e.g., metrics other than the standard ns/op, allocs/op, etc). That certainly what I want. :)

Can you explain why sub-benchmarks don't already solve this?

Oh, this isn't at all what I thought you were proposing. I thought you wanted custom metrics (e.g., metrics other than the standard ns/op, allocs/op, etc). That certainly what I want. :)

That would be a nice ability as well. For my immediate need i'm using the sub-benchmark's name to specify the custom metric:

// BenchmarkThing-8         20000000        80.4 ns/op
// BenchmarkThing/P99.9-8   20000000        80.4 ns/op

But an API for specifying a custom unit would be preferable:

// BenchmarkThing-8         20000000        80.4 ns/op      80.4 P99.9-ns/op

Which could be done with an extra method on BenchmarkResult: https://play.golang.org/p/_ITU8Ta0RZ

Can you explain why sub-benchmarks don't already solve this?

I'm building on sub-benchmarks but using the reflection package to get at name and iteration values. And even with reflection there's no way to get at the io.Writer used for output.

If I understand correctly, it seems like what you really want is a way of reporting custom metrics and doing this through sub-benchmarks/extra benchmark lines is just a hack to get around the lack of custom metrics. If so, we should add custom metrics, not add mechanism to support the hack.

If I understand correctly, it seems like what you really want is a way of reporting custom metrics and doing this through sub-benchmarks/extra benchmark lines is just a hack to get around the lack of custom metrics.

yes, that's what i'm wanting, sorry for the confusion.

Closing this in favor of #26037, which covers the same ground but appears to be moving forward.