Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/test2json: benchmark --- BENCH output not handled #23346

Closed
zolotov opened this issue Jan 5, 2018 · 4 comments
Closed

cmd/test2json: benchmark --- BENCH output not handled #23346

zolotov opened this issue Jan 5, 2018 · 4 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.10

Comments

@zolotov
Copy link
Contributor

zolotov commented Jan 5, 2018

While running benchmarks, the output is poor-structured. Would be nice to have start and passed events. Skip and Failed events work fine.

~/dev/go/bin/go test -bench=. -json benchmark_test.go

{"Time":"2018-01-05T12:38:47.961012155+03:00","Action":"output","Package":"command-line-arguments","Output":"goos: darwin\n"}
{"Time":"2018-01-05T12:38:47.961389758+03:00","Action":"output","Package":"command-line-arguments","Output":"goarch: amd64\n"}
{"Time":"2018-01-05T12:38:47.96164686+03:00","Action":"output","Package":"command-line-arguments","Output":"BenchmarkFoo-4   \t2000000000\t         0.00 ns/op\n"}
{"Time":"2018-01-05T12:38:47.961683819+03:00","Action":"output","Package":"command-line-arguments","Output":"--- BENCH: BenchmarkFoo-4\n"}
{"Time":"2018-01-05T12:38:47.961696276+03:00","Action":"output","Package":"command-line-arguments","Output":"\tbenchmark_test.go:8: My benchmark\n"}
{"Time":"2018-01-05T12:38:47.96170952+03:00","Action":"output","Package":"command-line-arguments","Output":"\tbenchmark_test.go:8: My benchmark\n"}
{"Time":"2018-01-05T12:38:47.961714862+03:00","Action":"output","Package":"command-line-arguments","Output":"\tbenchmark_test.go:8: My benchmark\n"}
{"Time":"2018-01-05T12:38:47.961719489+03:00","Action":"output","Package":"command-line-arguments","Output":"\tbenchmark_test.go:8: My benchmark\n"}
{"Time":"2018-01-05T12:38:47.961723798+03:00","Action":"output","Package":"command-line-arguments","Output":"\tbenchmark_test.go:8: My benchmark\n"}
{"Time":"2018-01-05T12:38:47.961731677+03:00","Action":"output","Package":"command-line-arguments","Output":"\tbenchmark_test.go:8: My benchmark\n"}
{"Time":"2018-01-05T12:38:47.961738298+03:00","Action":"output","Package":"command-line-arguments","Output":"PASS\n"}
{"Time":"2018-01-05T12:38:47.962676143+03:00","Action":"output","Package":"command-line-arguments","Output":"ok  \tcommand-line-arguments\t0.009s\n"}
{"Time":"2018-01-05T12:38:47.962695242+03:00","Action":"pass","Package":"command-line-arguments","Elapsed":0.009}

package benchmark

import (
	"testing"
)

func BenchmarkFoo(b *testing.B) {
	b.Log("My benchmark")
}
@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 5, 2018
@ianlancetaylor ianlancetaylor added this to the Go1.10 milestone Jan 5, 2018
@ianlancetaylor
Copy link
Contributor

CC @rsc

@rsc
Copy link
Contributor

rsc commented Jan 5, 2018

The test2json event definition has no answer for benchmarks generally. Benchmark timing output (the leading BenchmarkFoo line that benchstat reads) probably will be treated as ordinary top-level output here.

The --- BENCH: output that shows b.Log output we can definitely process and json-ify as if it were a test. I'll do that.

@rsc rsc mentioned this issue Jan 5, 2018
Closed
@rsc rsc changed the title cmd/test2json: benchmark start and passed events are missing cmd/test2json: benchmark --- BENCH output not handled Jan 5, 2018
@rsc
Copy link
Contributor

rsc commented Jan 5, 2018

Filed #23352 for benchmark start events.

@gopherbot
Copy link

Change https://golang.org/cl/86396 mentions this issue: cmd/test2json: fix processing of --- BENCH: output

@golang golang locked and limited conversation to collaborators Jan 5, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.10
Development

No branches or pull requests
4 participants
@rsc @zolotov @ianlancetaylor @gopherbot