Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

proposal: testing: add -vf (verbose when fail) flag to go test command #59201

Closed
vladopajic opened this issue Mar 23, 2023 · 4 comments
Closed

proposal: testing: add -vf (verbose when fail) flag to go test command #59201

vladopajic opened this issue Mar 23, 2023 · 4 comments
Labels
Proposal
Milestone
Proposal

Comments

@vladopajic
Copy link

vladopajic commented Mar 23, 2023 •
edited

Problem

For large golang codebase running test can produce overwhelming test results. Developers may not care about output of all tests unless test has failed.

Solution

With new option for go test command, flag -vf (verbose when fail), developer would be able to run tests, but with verbose output of tests only if it has failed. This would reduce total test result output only to bare minimum.

Open questions

  1. For test functions which want to produce additional output it would be helpful to expose io.Writer in testing.T struct which could be used to collect all outputs of tests in order to conditionally print entire test outputs.
// Consider having this test case in which
// output will be always written to stdout regardless 
// if test has failed or not.
func TestFoo(t *testing.T) {
    fmt.Printf("running test")
    
    ...
}

// In order to control outputs of tests, they would have to use 
// exposed `t.Writer` which would collect outputs of tests. 
// `go test` tool would then be able to decided weather data collected by
// this `t.Writer` would need to be outputted to stdout.
func TestFoo(t *testing.T) {
    fmt.Fprintf(t.Writer, "running test")

    ...
}

// Since most codebases already use loggers, it would be quite trivial to
// create logger with `t.Writer` and pass it where necessary.
func TestFoo(t *testing.T) {
    log := logger.New().WithWriter(t.Writer)
    log.Debug("running test")

    ...
}
@gopherbot gopherbot added this to the Proposal milestone Mar 23, 2023
@apparentlymart
Copy link

I like the idea of this, but I believe the current implementation of testing.T requires knowing whether the test is "verbose" before it produces any output.

For example, the t.Log family of functions writes into the "chatty printer" only if it's non-nil when the log function is called:

go/src/testing/testing.go

Lines 1024 to 1034 in e0c6958

if c.chatty != nil {
if c.bench {
// Benchmarks don't print === CONT, so we should skip the test
// printer and just print straight to stdout.
fmt.Print(c.decorate(s, depth+1))
} else {
c.chatty.Printf(c.name, "%s", c.decorate(s, depth+1))
}
return
}

c.chatty seems to be set only if running in "chatty mode", which seems to be the internal name for -v.

I think what you are proposing would require a change in approach where all of the functions must buffer log information etc in memory and discard it if the test runs to completion without failing, which might be reasonable but seems quite different than the current approach where (again, assuming I'm reading the code correctly) the log functions seem to just write directly to the output stream when verbose mode is enabled.

Do you have a different idea of how this would be implemented, or is buffering in memory what you were assuming?

(If I've misunderstood how the testing.T implementation behaves in my shallow reading of it then I apologize.)

@ianlancetaylor
Copy link
Contributor

It's already the case that any messages printed through t.Log and friends are printed if the test fails. This is true even though testing.Verbose() returns false. So I don't see any need for t.Writer. Also, see #22513.

So I'm not clear on what this proposal is for. Can you give a small example? Thanks.

@vladopajic
Copy link
Author

Thanks @ianlancetaylor, you are right. I haven't noticed t.Log before, which could be utilized instead of t.Writer. Also running tests without -v seems to be already doing what was proposed here.

@vladopajic vladopajic closed this as not planned Won't fix, can't repro, duplicate, stale Mar 24, 2023
@miparnisari
Copy link

@ianlancetaylor is there a way of having go test -v not print test names?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Proposal
Projects
None yet
Milestone
Proposal
Development

No branches or pull requests
5 participants
@apparentlymart @ianlancetaylor @vladopajic @miparnisari @gopherbot