Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/playground: a test that terminates itself with a signal is spuriously reported as passing #46823

Open
bcmills opened this issue Jun 18, 2021 · 2 comments
Open

x/playground: a test that terminates itself with a signal is spuriously reported as passing #46823

bcmills opened this issue Jun 18, 2021 · 2 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@bcmills
Copy link
Contributor

bcmills commented Jun 18, 2021

What version of Go are you using (go version)?

$ go version
go version devel go1.17-45f251ad6 Thu Jun 17 21:58:54 2021 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/usr/local/google/home/bcmills/.cache/go-build"
GOENV="/usr/local/google/home/bcmills/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/tmp/tmp.BvIzRvpmLB/.gopath/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/tmp/tmp.BvIzRvpmLB/.gopath"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/google/home/bcmills/sdk/gotip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/google/home/bcmills/sdk/gotip/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="devel go1.17-45f251ad6 Thu Jun 17 21:58:54 2021 +0000"
GCCGO="/usr/local/google/home/bcmills/bin/gccgo"
AR="ar"
CC="gcc"
CXX="c++"
CGO_ENABLED="1"
GOMOD="/tmp/tmp.BvIzRvpmLB/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2893288510=/tmp/go-build -gno-record-gcc-switches"

What did you do?

  1. Wrote the program in https://play.golang.org/p/Natjl4SGrTr into a file main_test.go in a local package.
  2. Ran go test .
  3. Compared the result with the output reported on https://play.golang.org/p/Natjl4SGrTr.

What did you expect to see?

I expected the Playground to produce roughly the same result as go test ., which reports the test as failing:

$ go test .
2021/06/18 13:32:15 Huh‽
FAIL    example.com/m   0.013s
FAIL

What did you see instead?

The output from the Playground is:

=== RUN   TestLogPrintf
2009/11/10 23:00:00 Huh‽

All tests passed.

CC @golang/release
@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 18, 2021
@bcmills bcmills added this to the Backlog milestone Jun 18, 2021
@prattmic
Copy link
Member

It looks like the playground looks only for "--- FAIL" in the output to determine the tests failed. https://cs.opensource.google/go/x/playground/+/master:sandbox.go;l=382

Since the test binary exited early, it wouldn't print "--- FAIL". We should probably use a non-zero exit code to indicate failure as well. I assume this is what the go command does?

@bcmills
Copy link
Contributor Author

bcmills commented Jun 18, 2021

Yes. (See previously #40132, which I fixed by adding cmd/internal/test2json.(*Converter).Exited.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
2 participants
@prattmic @bcmills