Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/test2json: Fails to mark a test as failed when using os.Exit #43263

Closed
dlsniper opened this issue Dec 18, 2020 · 4 comments
Closed

cmd/test2json: Fails to mark a test as failed when using os.Exit #43263

dlsniper opened this issue Dec 18, 2020 · 4 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@dlsniper
Copy link
Contributor

What version of Go are you using (go version)?

$ go version
go version go1.16beta1 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Florin\AppData\Local\go-build
set GOENV=C:\Users\Florin\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=D:\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=D:\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=D:\go-sdks\go1.16beta1
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=D:\go-sdks\go1.16beta1\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.16beta1
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=D:\GoLandProjects\awesomeProject37\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\Florin\AppData\Local\Temp\go-build898034260=/tmp/go-build -gno-record-g
cc-switches

What did you do?

I used the following test scenario

package main

import (
	"os"
	"testing"
)

func TestDemo(t *testing.T) {
	t.Log("This is a demo")
	os.Exit(0)
}

When I run the command below, the test fails, as documented in the release notes

go test -v ./...
=== RUN   TestDemo
    main_test.go:9: This is a demo
--- FAIL: TestDemo (0.00s)
panic: unexpected call to os.Exit(0) during test [recovered]
        panic: unexpected call to os.Exit(0) during test

goroutine 19 [running]:
testing.tRunner.func1.2(0x9801e0, 0x9d7090)
        D:/go-sdks/go1.16beta1/src/testing/testing.go:1144 +0x345
testing.tRunner.func1(0xc000158300)
        D:/go-sdks/go1.16beta1/src/testing/testing.go:1147 +0x4b6
panic(0x9801e0, 0x9d7090)
        D:/go-sdks/go1.16beta1/src/runtime/panic.go:965 +0x1c7
os.Exit(0x0)
        D:/go-sdks/go1.16beta1/src/os/proc.go:68 +0x74
awesomeProject37.TestDemo(0xc000158300)
        D:/GoLandProjects/awesomeProject37/main_test.go:10 +0x7d
testing.tRunner(0xc000158300, 0x9b4e90)
        D:/go-sdks/go1.16beta1/src/testing/testing.go:1194 +0xef
created by testing.(*T).Run
        D:/go-sdks/go1.16beta1/src/testing/testing.go:1239 +0x2b3
FAIL    awesomeProject37        0.270s
FAIL

However, when I run:

go test -c -o demo.exe ./...
go tool test2json -t demo.exe -test.v
{"Time":"2020-12-18T13:09:04.2031136+02:00","Action":"run","Test":"TestDemo"}
{"Time":"2020-12-18T13:09:04.2047196+02:00","Action":"output","Test":"TestDemo","Output":"=== RUN   TestDemo\n"}
{"Time":"2020-12-18T13:09:04.2052725+02:00","Action":"output","Test":"TestDemo","Output":"    main_test.go:9: This is a demo\n"}
{"Time":"2020-12-18T13:09:04.2166141+02:00","Action":"pass","Test":"TestDemo","Elapsed":0.207}

The test passes.

What did you expect to see?

I expect to see a test failure when using go test -c too.

What did you see instead?

I see the tests passing.

@ianlancetaylor ianlancetaylor changed the title go/test2json: Fails to mark a test as failed when using os.Exit cmd/test2json: Fails to mark a test as failed when using os.Exit Dec 18, 2020
@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 18, 2020
@ianlancetaylor ianlancetaylor added this to the Go1.17 milestone Dec 18, 2020
@dlsniper
Copy link
Contributor Author

@ianlancetaylor can we please prioritize this to Go 1.16 instead? This is a new feature added to Go 1.16 and everyone using test2json will have different behavior than those that don't use it. For example, this is a problem for those using GoLand to run tests, as we rely on the test2json command to produce correct data to display to end-users.

@ianlancetaylor
Copy link
Contributor

The fix is to also use -test.paniconexit0, as in

go tool test2json -t demo.exe -test.v -test.paniconexit0

I'll update the docs. Invoking go tool test2json directly is an unusual case.

@gopherbot
Copy link

Change https://golang.org/cl/283873 mentions this issue: cmd/test2json: document passing -test.paniconexit0

@bcmills
Copy link
Contributor

bcmills commented Jan 14, 2021

This is for #29062.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

4 participants