Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

testing: silly typo in subtests causes hard-to-understand error message #21175

Closed
barnex opened this issue Jul 26, 2017 · 4 comments
Closed

testing: silly typo in subtests causes hard-to-understand error message #21175

barnex opened this issue Jul 26, 2017 · 4 comments
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Milestone
Go1.10

Comments

@barnex
Copy link

barnex commented Jul 26, 2017

What did you do?

Run go test on this (wrong) test file. The subtest is wrongfully referring to the outer testing.T:

package main

import "testing"

func TestPanic(t *testing.T) {
        t.Run("", func(typo *testing.T) {
                t.Fatalf("...") // wrong, should be typo.Fatalf, not t.Fatalf
        })
}

What did you expect to see?

Perhaps a somewhat informative error message. At least not a panic(nil) leading to a long debug session.

What did you see instead?

A very confusing panic(nil):

> go test
panic: test executed panic(nil) or runtime.Goexit

goroutine 6 [running]:
testing.tRunner.func1(0xc4200941e0)
        /home/arne/bin/go/src/testing/testing.go:712 +0x2d2
runtime.Goexit()
        /home/arne/bin/go/src/runtime/panic.go:366 +0x147
testing.(*common).FailNow(0xc4200940f0)
        /home/arne/bin/go/src/testing/testing.go:544 +0x39
testing.(*common).Fatalf(0xc4200940f0, 0x53665f, 0xd, 0x0, 0x0, 0x0)
        /home/arne/bin/go/src/testing/testing.go:588 +0x83
_/home/arne/tmp.TestPanic.func1(0xc4200941e0)
        /home/arne/tmp/panic_test.go:7 +0x5c
testing.tRunner(0xc4200941e0, 0xc42000e400)
        /home/arne/bin/go/src/testing/testing.go:747 +0xd0
created by testing.(*T).Run
        /home/arne/bin/go/src/testing/testing.go:790 +0x2de
exit status 2
FAIL    _/home/arne/tmp 0.003s

Does this issue reproduce with the latest release (go1.8.3)?

Yes. Also with go1.9rc1.

System details

go version go1.9rc1 linux/amd64
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/arne"
GORACE=""
GOROOT="/home/arne/bin/go"
GOTOOLDIR="/home/arne/bin/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build514931481=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOROOT/bin/go version: go version go1.9rc1 linux/amd64
GOROOT/bin/go tool compile -V: compile version go1.9rc1
uname -sr: Linux 4.4.0-83-generic
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.2 LTS
Release:	16.04
Codename:	xenial
/lib/x86_64-linux-gnu/libc.so.6: GNU C Library (Ubuntu GLIBC 2.23-0ubuntu9) stable release version 2.23, by Roland McGrath et al.
gdb --version: GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
@ALTree
Copy link
Member

ALTree commented Jul 26, 2017 •
edited

What did you see instead?
A very confusing panic(nil)

What did you expect to see?
At least not a panic(nil)

The full error message is not

test executed panic(nil)

it's

test executed panic(nil) or runtime.Goexit

the part after the or is the relevant one.

Your erroneous Fatalf call causes a FailNow call, which triggers a runtime.Goexit on the external goroutine. The error message is (correctly) informing you of this (the fact that runtime.Goexit was called on the external goroutine before it was done with its job).

What's really misleading here is the fact that the error message refers to a runtime.Goexit call, which the user has probably no way to trace back to t.FailNow without reading the testing.go source code.

Suggestion: amend the FailNow doc by mentioning the fact that it calls runtime.Goexit, i.e. from this:

FailNow marks the function as having failed and stops its execution.

to this:

FailNow marks the function as having failed and stops its execution by calling runtime.Goexit.

This way a user reading about a crash caused by runtime.Goexit will have a way to trace the issue back to a bad t.FailNow call without having to read the testing package source code.

@ALTree ALTree changed the title testing: Silly typo in subtests causes panic(nil) testing: silly typo in subtests causes hard-to-understand error message Jul 26, 2017
@ALTree ALTree added this to the Go1.10 milestone Jul 26, 2017
@ALTree ALTree added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Jul 26, 2017
@ALTree
Copy link
Member

ALTree commented Jul 26, 2017

Oh, and the SkipNow doc, too. It calls runtime.Goexit, like FailNow.

@cznic
Copy link
Contributor

cznic commented Jul 26, 2017

Related #17421.

@gopherbot
Copy link

Change https://golang.org/cl/52770 mentions this issue: testing: explain how SkipNow and FailNow stop execution

@devdinu devdinu mentioned this issue Mar 13, 2018
Closed
@golang golang locked and limited conversation to collaborators Aug 9, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Projects
None yet
Milestone
Go1.10
Development

No branches or pull requests
4 participants
@barnex @cznic @ALTree @gopherbot