Here is a smaller repro:

$ cat go.mod
module foo.bar
$ cat f_test.go
package foo

import (
        "os"
        "testing"
)

func TestExit(t *testing.T) {
        os.Exit(0)
}

func TestFatal(t *testing.T) {
        t.Fatal()
}
$ go test
ok      foo.bar 0.001s
$ go test -v
=== RUN   TestExit
ok      foo.bar 0.001s

I agree that this can be confusing, since go test appears happy. It's only go test -v that looks off, since none of the tests actually succeed and finish.

Perhaps go test could error if a test binary exited too soon. For example, if we do go test -c and run the test binary, we get no output at all instead of a PASS or a FAIL. But I'm not sure if we could implement that without introducing even more confusing edge cases. For example, what should happen if a test binary panics?

/cc @bcmills @ianlancetaylor

A perhaps simple fix would be for go test to complain if a test binary prints absolutely nothing. I think that should never happen.

@mvdan: That fix sounds like the right one.

A test binary can provide an arbitrary func main, right? Why does it need to print anything on success?

Are you talking about TestMain? It prints a result (PASS) as well.
I suppose you could implement TestMain without calling m.Run on its argument, but that seems not supported.

I did some experimenting and it looks like I was just mistaken. Carry on!

Change https://golang.org/cl/184457 mentions this issue: cmd/go: fail if a test binary succeeds with no output

From the CL:

A few TestMain funcs in the standard library needed fixing, as they
returned without printing anything as a means to skip testing the entire
package

How concerning is this for breaking user tests? If you had to fix up stdlib tests, it's likely that many tests in user code are going to break.

@randall77 #31969 mentions a TestMain that doesn't invoke m.Run (though the docs of testing ask not to do that). Also, as @mvdan's CL demonstrates, this is done in the stdlib: https://github.com/golang/go/blob/master/src/cmd/internal/goobj/goobj_test.go#L33

Historically, it has been acceptable to break user programs that were doing the opposite of what is documented. For example, see #31859. We want to avoid this breakage whenever possible, but in some cases it's the best option.

I think this is one of those cases. The fix will uncover package tests that were succeeding by accident, which was the motivation of this bug. And after all, fixing the broken tests was a two-line change, and I think that's a reasonable thing to ask of the users who weren't following the docs.

Change https://golang.org/cl/200104 mentions this issue: doc/go1.14: note that tests without any output now fail

Change https://golang.org/cl/200106 mentions this issue: Revert "cmd/go: fail if a test binary exits with no output"

I'm reverting this CL due to #34791. We can un-revert it if we decide that it is acceptable to break the assumptions of #18153, or if we figure out a way to implement this change without breaking those assumptions.

Thanks for taking care of this @bcmills. I do fundamentally disagree with the premise of #18153. For example, it makes tests brittle, and requires them to be run with a real terminal, and no arguments:

$ cat f_test.go
package foo

import (
        "os"
        "testing"

        "golang.org/x/crypto/ssh/terminal"
)

func TestFoo(t *testing.T) {
        if !terminal.IsTerminal(int(os.Stdout.Fd())) {
                t.Fatal("not a terminal!")
        }
}
$ go test
PASS
ok      test.tld/foo    0.002s
$ go test | cat
--- FAIL: TestFoo (0.00s)
    f_test.go:12: not a terminal!
FAIL
exit status 1
FAIL    test.tld/foo    0.002s
$ go test .
--- FAIL: TestFoo (0.00s)
    f_test.go:12: not a terminal!
FAIL
FAIL    test.tld/foo    0.002s
FAIL

go test -v with no arguments fails, too.

Moreover, it makes it very hard (or impossible, I suspect) to inspect or modify the output in any way, like we needed to do here.

Do you reckon a proposal is necessary to make a proper decision here?

I realise that the original idea was to color test output, and not necessarily make tests fail or pass depending on stdout being a terminal. Still, I think it's wrong to change the test's behavior depending on whether it happens to be run on a terminal. If a user really wants color on their test outputs, they could use a test flag, or an environment variable like $TERM != "dumb". At least in that case, you wouldn't get the inconsistencies I showed above.

No change in consensus. Accepted.

This issue is currently labeled as early-in-cycle for Go 1.15. That time is now, so friendly ping. If it no longer needs to be done early in cycle, that label can be removed.

Change https://golang.org/cl/222243 mentions this issue: cmd/go: make go test -json report failures for panicking/exiting tests

Change https://golang.org/cl/222658 mentions this issue: [release-branch.go1.14] cmd/go: make go test -json report failures for panicking/exiting tests

This issue is currently labeled as early-in-cycle for Go 1.16.
That time is now, so this is a friendly ping so the issue is looked at again.

Change https://golang.org/cl/250977 mentions this issue: cmd/go, testing, os: fail test that calls os.Exit(0)

Change https://golang.org/cl/251262 mentions this issue: testing: restore os.Exit(0) after every call to (*M).Run

Change https://golang.org/cl/283873 mentions this issue: cmd/test2json: document passing -test.paniconexit0