cmd/go: test -bench implies "chatty" #18010
Comments
quentinmit
added
the
NeedsDecision
|
Possibly related to #17603? |
|
Doesn't seem related to me. On Nov 22, 2016 19:34, "Josh Bleecher Snyder" notifications@github.com
|
|
Could you reproduce this on any of the standard packages?
And which version of Go are you using?
|
|
What do you mean by "the standard packages"? What does this have to do with the package being tested?
|
|
Because I can't reproduce this in any of the std packages that I tried,
so I suspect that maybe it has something to do with the package
being tested.
Are you suggesting that difference in output is the extra "PASS"
at the beginning of the output? If that's case, it's working as intended
as if -bench is passed, even if none of the benchmark matches,
the testing package will still use the output format intended for
benchmarks.
|
|
@minux I think he means stdout/stderr produced by tests, which is hidden by |
|
Also, what is the reason for the inconsistency between |
|
I still don't understand what this issue is about.
$ cat a_test.go
package pkg
import (
"fmt"
"testing"
)
func TestA(t *testing.T) {
fmt.Println("Tstdout")
println("Tstderr")
t.Log("Tlog")
}
func BenchmarkA(t *testing.B) {
fmt.Println("Bstdout")
println("Bstderr")
t.Log("Blog")
}
$ go171 test
Tstdout
Tstderr
PASS
ok pkg 0.001s
$ go171 test -bench -
Tstdout
Tstderr
PASS
ok pkg 0.001s
|
|
@minux your example is exactly the problem. if you run |
|
@tamird but he did just there and it also prints. |
|
Oh, it seems the behaviour is not so simple. ...so if you specify a package name, stdout/stderr are swallowed, but if you don't, they are not. this is so inconsistent as to be crazy. |
|
OK, here are all four combinations:
$ go171 test pkg
ok pkg 0.001s
$ go171 test pkg -bench -
Tstdout
Tstderr
PASS
ok pkg 0.001s
$ go171 test
Tstdout
Tstderr
PASS
ok pkg 0.001s
$ go171 test -bench -
Tstdout
Tstderr
PASS
ok pkg 0.002s
I think this is all working as intended.
When -bench is not given,
1. with package name(s), go test only shows test summary
2. without package name, go test assume you're developing that package,
and want full output, so it doesn't suppress stdout/stderr.
When -bench is given, go test will always show stdout/stderr, even if it
doesn't match any of the benchmarks (this is because the benchmark
statistics is produced by the testing package, not by the go command,
so if the Go command wants to show you the benchmark result, it can't
hide the other output from the test.)
|
|
Working as intended by whom? None of this is documented, and none of it makes sense. Why does supplying a package name suppress stdout? "go test assume you're developing that package" this is a rationalization, or something, because it's certainly not documented, and doesn't make an iota of sense.
That sounds like an implementation detail, not a sensible reason that for this behaviour. Again, it is not documented. |
|
undocumented, perhaps, but it has always been this way since Go 1
so it's very unlikely to change.
|
|
why? it doesn't seem to me that any of this is covered by the Go 1 guarantee. |
|
What do you want to change?
Tools are not covered by Go 1 guarantee, but we should at least
preserve existing behavior as much as possible (in case people
have existing programs that parse such output).
|
|
As the issue description states, I propose that the |
|
+1 I think the behavior currently in the code was actually not intended by the designer of this code, and I think anyone using the benchmarking code will welcome fixing this issue that has been there since Go 1. |
|
Like @minux wrote above, this is working as intended (I designed and wrote the code, and this is what I intended). If running benchmarks, we stream the output from the benchmarks to make sure they are seen at all but also that they are seen as they happen. The only option we've come up with is to make go test -bench run the binary twice, once for tests (quietly) and once for benchmarks (not quietly). That seems a bit odd, and a different special case (init code would run multiple times, and so on). |
|
There are conflicting goals here, of course, but since this has been this way from Go 1, I'm inclined to keep things as is. |
Consider the following invocations:
the two should ostensibly produce the same result, but the latter does not omit the output of successful tests as the former does. This does not appear to be documented anywhere.
A somewhat related issue was previously filed (#10713) and closed with e9827f6, even though the behaviour appears to be global depending on the value of the
-benchflag rather than based on the type of receiver ofLogandLogf.The text was updated successfully, but these errors were encountered: