Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: test mixes stderr and stdout #25241

Open
andreimatei opened this issue May 3, 2018 · 4 comments
Open

cmd/go: test mixes stderr and stdout #25241

andreimatei opened this issue May 3, 2018 · 4 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@andreimatei
Copy link
Contributor

7badae8 appears to have made go test mix stderr and stdout from tests, which has made go 1.10 a regression for CockroachDB's benchmarks.
This issue is related to #18010 :
Not only is the benchmark output not swallowed, but stderr and stdout are now mixed, so, if our binary produces logs (on stderr, as ours does), it's now impossible to see the benchmark results or use tools like benchstat.
In particular, I believe benchstat fails to work because the mixing of stderr and stdout doesn't even use line buffering - so the benchmark measurements lines are mixed with log lines and benchstat can't recognize them any more.

I generally concur with the folks on #18010 arguing that the swallowing/not swallowing of output is inconsistent (and there's no control over it as far as I can tell). Additionally, now the mixing of stdout and stderr is really a problem. The argument for not doing anything about go test -bench not swallowing output was that "we should at least preserve existing behavior as much as possible", but it would appear that existing behavior was changed in a quite significant way by this mixing. So I would kindly ask for the swallowing decision to be re-evaluated.

What version of Go are you using (go version)?

go version go1.10 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

OS X/ amd64

cc @benesch

@agnivade agnivade changed the title go test mixes stderr and stdout, making benchmark results impossible to see cmd/go: test mixes stderr and stdout, making benchmark results impossible to see May 4, 2018
@agnivade agnivade added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label May 4, 2018
@agnivade agnivade added this to the Go1.11 milestone May 4, 2018
@odeke-em
Copy link
Member

/cc @bradfitz and @rsc

@zamsden
Copy link

zamsden commented Jul 29, 2018

This is horrible and next to unusable. Users should be able to opt-in to getting this output, not forced in.

@rsc
Copy link
Contributor

rsc commented Aug 10, 2018

I'm sorry your benchmarks print so much to standard error.
They might give more reliable timing results if they didn't do that, parsing aside.
"Horrible and next to unusable" strikes me as a bit of an exaggeration.
As far as I can tell only CockroachDB is running into this.

I'm sorry we didn't get to it in Go 1.11.

@rsc rsc modified the milestones: Go1.11, Go1.12 Aug 10, 2018
@rsc rsc modified the milestones: Go1.12, Go1.13 Nov 20, 2018
@bcmills bcmills changed the title cmd/go: test mixes stderr and stdout, making benchmark results impossible to see cmd/go: test mixes stderr and stdout Jan 18, 2019
@bcmills
Copy link
Contributor

bcmills commented Jan 18, 2019

#24929 may also affect this behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

7 participants