Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: inconsistent behaviors between running tests directly and after compiling the code first #59879

Closed
rainydew opened this issue Apr 28, 2023 · 3 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@rainydew
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.20.3 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/baozhenchen/Library/Caches/go-build"
GOENV="/Users/baozhenchen/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/baozhenchen/.gvm/pkgsets/go1.20.3/global/pkg/mod"
GONOPROXY="git.garena.com"
GONOSUMDB="git.garena.com"
GOOS="darwin"
GOPATH="/Users/baozhenchen/.gvm/pkgsets/go1.20.3/global"
GOPRIVATE="git.garena.com"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/baozhenchen/.gvm/gos/go1.20.3"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/Users/baozhenchen/.gvm/gos/go1.20.3/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.20.3"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/baozhenchen/gits/go-tester/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/6r/b8pq0wys5gg6j9l59fx0_gfc0000gq/T/go-build2381900583=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

with this code

package gotest_parallel_bug

import (
	"sync"
	"testing"
	"time"
)

func TestParallelBug(t *testing.T) {
	wg := &sync.WaitGroup{}

	t.Run("a", func(t *testing.T) {
		wg.Add(1)
		defer wg.Done()
		t.Parallel()
		time.Sleep(10 * time.Second)
	})

	t.Run("b", func(t *testing.T) {
		wg.Add(1)
		defer wg.Done()
		t.Parallel()
		time.Sleep(10 * time.Second)
	})

	time.Sleep(1 * time.Second)
	wg.Wait()

	t.Run("c", func(t *testing.T) {
		wg.Add(1)
		defer wg.Done()
		t.Parallel()
		time.Sleep(10 * time.Second)
	})

	t.Run("d", func(t *testing.T) {
		wg.Add(1)
		defer wg.Done()
		t.Parallel()
		time.Sleep(10 * time.Second)
	})

	wg.Wait()
}

first I ran it directly

go test parallel_test.go -v

i see the Go process was hang and no response, need ctrl+C to exit

=== RUN   TestParallelBug
=== RUN   TestParallelBug/a
=== PAUSE TestParallelBug/a
=== RUN   TestParallelBug/b
=== PAUSE TestParallelBug/b
^CFAIL  command-line-arguments  21.867s

and then I ran it by compiling it first

go test -c -o parallel.test parallel_test.go
./parallel.test -test.v

i see Go panics

=== RUN   TestParallelBug
=== RUN   TestParallelBug/a
=== PAUSE TestParallelBug/a
=== RUN   TestParallelBug/b
=== PAUSE TestParallelBug/b
fatal error: all goroutines are asleep - deadlock!

goroutine 1 [chan receive]:
testing.(*T).Run(0xc000007860, {0x112b703?, 0x246962a686ad3?}, 0x1134820)
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/testing/testing.go:1630 +0x405
testing.runTests.func1(0x1214e20?)
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/testing/testing.go:2036 +0x45
testing.tRunner(0xc000007860, 0xc000106c88)
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/testing/testing.go:1576 +0x10b
testing.runTests(0xc00011a0a0?, {0x120bdb0, 0x1, 0x1}, {0x0?, 0x100c00010c8f8?, 0x0?})
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/testing/testing.go:2034 +0x489
testing.(*M).Run(0xc00011a0a0)
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/testing/testing.go:1906 +0x63a
main.main()
        _testmain.go:47 +0x1aa

goroutine 6 [semacquire]:
sync.runtime_Semacquire(0xc10b033f74909e88?)
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/runtime/sema.go:62 +0x27
sync.(*WaitGroup).Wait(0x3b9aca00?)
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/sync/waitgroup.go:116 +0x4b
command-line-arguments.TestParallelBug(0x0?)
        /Users/baozhenchen/gits/go-tester/notescripts/gotest_parallel_bug/parallel_test.go:27 +0xe5
testing.tRunner(0xc000007a00, 0x1134820)
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/testing/testing.go:1576 +0x10b
created by testing.(*T).Run
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/testing/testing.go:1629 +0x3ea

goroutine 7 [chan receive]:
testing.(*T).Parallel(0xc000007d40)
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/testing/testing.go:1384 +0x225
command-line-arguments.TestParallelBug.func1(0x0?)
        /Users/baozhenchen/gits/go-tester/notescripts/gotest_parallel_bug/parallel_test.go:15 +0x7a
testing.tRunner(0xc000007d40, 0xc00006a630)
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/testing/testing.go:1576 +0x10b
created by testing.(*T).Run
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/testing/testing.go:1629 +0x3ea

goroutine 8 [chan receive]:
testing.(*T).Parallel(0xc0001301a0)
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/testing/testing.go:1384 +0x225
command-line-arguments.TestParallelBug.func2(0x0?)
        /Users/baozhenchen/gits/go-tester/notescripts/gotest_parallel_bug/parallel_test.go:22 +0x7a
testing.tRunner(0xc0001301a0, 0xc00006a6a0)
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/testing/testing.go:1576 +0x10b
created by testing.(*T).Run
        /Users/baozhenchen/.gvm/gos/go1.20.3/src/testing/testing.go:1629 +0x3ea

What did you expect to see?

The behaviors should be the same between running tests directly and after compiling the code first

What did you see instead?

The behaviors are different. This troubled me when running the same code between some IDEs like Goland (which will compile it first) and CI tools like Jenkins (which will run code directly)

@rainydew rainydew changed the title affected/package: testing, inconsistent behaviors between running tests directly and after compiling the code first testing: inconsistent behaviors between running tests directly and after compiling the code first Apr 28, 2023
@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 28, 2023
@dmitshur dmitshur added this to the Backlog milestone Apr 28, 2023
@dmitshur
Copy link
Contributor

CC @bcmills, @matloob.

@ianlancetaylor
Copy link
Contributor

The difference is that when you use go test, it passes an extra argument -test.timeout=10m0s to the generated test program. This is documented as the default timeout used by go test (see the documentation of -timeout in https://pkg.go.dev/cmd/go#hdr-Testing_flags).

If you pass that same option when you run the test binary yourself, then it will also not report a deadlock. The reason there isn't a deadlock in that case is that there is a goroutine that is going to wake up in 10 minutes and say "test timed out". So the program is not deadlocked.

@ianlancetaylor ianlancetaylor closed this as not planned Won't fix, can't repro, duplicate, stale Apr 28, 2023
@rainydew
Copy link
Author

rainydew commented Apr 29, 2023

The difference is that when you use go test, it passes an extra argument -test.timeout=10m0s to the generated test program. This is documented as the default timeout used by go test (see the documentation of -timeout in https://pkg.go.dev/cmd/go#hdr-Testing_flags).

If you pass that same option when you run the test binary yourself, then it will also not report a deadlock. The reason there isn't a deadlock in that case is that there is a goroutine that is going to wake up in 10 minutes and say "test timed out". So the program is not deadlocked.

thanks~ (I just didn't realize test binary disables the timeout totally by default)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

3 participants