Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: race TestScript failures on linux/amd64 #30550

Closed
mvdan opened this issue Mar 3, 2019 · 3 comments
Closed

cmd/go: race TestScript failures on linux/amd64 #30550

mvdan opened this issue Mar 3, 2019 · 3 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Milestone
Go1.13

Comments

@mvdan
Copy link
Member

mvdan commented Mar 3, 2019

Seems to happen almost every run on my machine. go test -race -run TestScript/vendor seems to work as well, which is good since the entire run takes a long time.

$ go version
go version devel +fc42cf8b8c Sun Mar 3 10:33:18 2019 +0000 linux/amd64
$ go env
GOARCH="amd64"
GOBIN="/home/mvdan/go/bin"
GOCACHE="/home/mvdan/go/cache"
GOEXE=""
GOFLAGS="-ldflags=-w"
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/mvdan/go"
GOPROXY=""
GORACE=""
GOROOT="/home/mvdan/tip"
GOTMPDIR=""
GOTOOLDIR="/home/mvdan/tip/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build154362150=/tmp/go-build -gno-record-gcc-switches"

--- FAIL: TestScript (0.08s)
    --- FAIL: TestScript/mod_vendor_build (4.94s)
        script_test.go:188:
            # initial conditions: using sampler v1.3.0, not listed in go.mod. (1.279s)
            # update to v1.3.1, now indirect in go.mod. (1.309s)
            # vendoring can but should not need to make changes. (1.091s)
            # go list -mod=vendor (or go build -mod=vendor) must not modify go.mod.
            # golang.org/issue/26704 (1.253s)
            > go list -mod=vendor
            [stdout]
            m
            [stderr]
            ==================
            WARNING: DATA RACE
            Read at 0x00c0001a4000 by goroutine 9:
              cmd/go/internal/modload.(*mvsReqs).Required.func1()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:859 +0x572
              cmd/go/internal/par.(*Cache).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:128 +0x12f
              cmd/go/internal/modload.(*mvsReqs).Required()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:854 +0x11f
              cmd/go/internal/mvs.buildList.func1()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:88 +0x17a
              cmd/go/internal/par.(*Work).runner()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:101 +0x3c3

            Previous write at 0x00c0001a4000 by goroutine 8:
              cmd/go/internal/modload.(*mvsReqs).Required.func1()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:870 +0x4dc
              cmd/go/internal/par.(*Cache).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:128 +0x12f
              cmd/go/internal/modload.(*mvsReqs).Required()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:854 +0x11f
              cmd/go/internal/mvs.buildList.func1()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:88 +0x17a
              cmd/go/internal/par.(*Work).runner()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:101 +0x3c3

            Goroutine 9 (running) created at:
              cmd/go/internal/par.(*Work).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:67 +0xf9
              cmd/go/internal/mvs.buildList()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:86 +0x3d8
              cmd/go/internal/modload.(*loader).load()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:73 +0x1a2
              cmd/go/internal/modload.ImportPaths()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:161 +0x473
              cmd/go/internal/load.ImportPaths()
                  /home/mvdan/tip/src/cmd/go/internal/load/pkg.go:1888 +0xa0
              cmd/go/internal/load.PackagesAndErrors()
                  /home/mvdan/tip/src/cmd/go/internal/load/pkg.go:1839 +0x76
              cmd/go/internal/load.Packages()
                  /home/mvdan/tip/src/cmd/go/internal/load/pkg.go:1820 +0x6a
              cmd/go/internal/list.runList()
                  /home/mvdan/tip/src/cmd/go/internal/list/list.go:424 +0x3934
              main.main()
                  /home/mvdan/tip/src/cmd/go/main.go:219 +0xb21

            Goroutine 8 (running) created at:
              cmd/go/internal/par.(*Work).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:67 +0xf9
              cmd/go/internal/mvs.buildList()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:86 +0x3d8
              cmd/go/internal/modload.(*loader).load()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:73 +0x1a2
              cmd/go/internal/modload.ImportPaths()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:161 +0x473
              cmd/go/internal/load.ImportPaths()
                  /home/mvdan/tip/src/cmd/go/internal/load/pkg.go:1888 +0xa0
              cmd/go/internal/load.PackagesAndErrors()
                  /home/mvdan/tip/src/cmd/go/internal/load/pkg.go:1839 +0x76
              cmd/go/internal/load.Packages()
                  /home/mvdan/tip/src/cmd/go/internal/load/pkg.go:1820 +0x6a
              cmd/go/internal/list.runList()
                  /home/mvdan/tip/src/cmd/go/internal/list/list.go:424 +0x3934
              main.main()
                  /home/mvdan/tip/src/cmd/go/main.go:219 +0xb21
            ==================
            Found 1 data race(s)
            [exit status 66]
            FAIL: testdata/script/mod_vendor_build.txt:19: unexpected command failure

    --- FAIL: TestScript/mod_getmode_vendor (4.38s)
        script_test.go:188:
            > env GO111MODULE=on
            > go get -m rsc.io/quote@v1.5.1
            [stderr]
            go: finding rsc.io/quote v1.5.1
            go: finding rsc.io/sampler v1.3.0
            go: finding golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
            > go mod vendor
            [stderr]
            go: downloading rsc.io/quote v1.5.1
            go: extracting rsc.io/quote v1.5.1
            go: downloading rsc.io/sampler v1.3.0
            go: extracting rsc.io/sampler v1.3.0
            go: downloading golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
            go: extracting golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
            > env GOPATH=$WORK/empty
            > env GOPROXY=file:///nonexist
            > go list -mod=vendor
            [stdout]
            x
            > go list -mod=vendor -m -f '{{.Path}} {{.Version}} {{.Dir}}' all
            [stdout]
            x  $WORK/gopath/src
            golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c $WORK/gopath/src/vendor/golang.org/x/text
            rsc.io/quote v1.5.1 $WORK/gopath/src/vendor/rsc.io/quote
            rsc.io/sampler v1.3.0 $WORK/gopath/src/vendor/rsc.io/sampler
            [stderr]
            ==================
            WARNING: DATA RACE
            Read at 0x00c0001b8000 by goroutine 27:
              cmd/go/internal/modload.(*mvsReqs).Required.func1()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:859 +0x572
              cmd/go/internal/par.(*Cache).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:128 +0x12f
              cmd/go/internal/modload.(*mvsReqs).Required()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:854 +0x11f
              cmd/go/internal/mvs.buildList.func1()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:88 +0x17a
              cmd/go/internal/par.(*Work).runner()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:101 +0x3c3

            Previous write at 0x00c0001b8000 by goroutine 26:
              cmd/go/internal/modload.(*mvsReqs).Required.func1()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:870 +0x4dc
              cmd/go/internal/par.(*Cache).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:128 +0x12f
              cmd/go/internal/modload.(*mvsReqs).Required()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:854 +0x11f
              cmd/go/internal/mvs.buildList.func1()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:88 +0x17a
              cmd/go/internal/par.(*Work).runner()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:101 +0x3c3

            Goroutine 27 (running) created at:
              cmd/go/internal/par.(*Work).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:67 +0xf9
              cmd/go/internal/mvs.buildList()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:86 +0x3d8
              cmd/go/internal/modload.(*loader).load()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:73 +0x1a2
              cmd/go/internal/modload.ReloadBuildList()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:288 +0x91
              cmd/go/internal/modload.LoadBuildList()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:281 +0x38
              cmd/go/internal/modload.listModules()
                  /home/mvdan/tip/src/cmd/go/internal/modload/list.go:43 +0x47
              cmd/go/internal/modload.ListModules()
                  /home/mvdan/tip/src/cmd/go/internal/modload/list.go:20 +0x6b
              cmd/go/internal/list.runList()
                  /home/mvdan/tip/src/cmd/go/internal/list/list.go:388 +0x54e
              main.main()
                  /home/mvdan/tip/src/cmd/go/main.go:219 +0xb21

            Goroutine 26 (running) created at:
              cmd/go/internal/par.(*Work).Do()
                  /home/mvdan/tip/src/cmd/go/internal/par/work.go:67 +0xf9
              cmd/go/internal/mvs.buildList()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:86 +0x3d8
              cmd/go/internal/modload.(*loader).load()
                  /home/mvdan/tip/src/cmd/go/internal/mvs/mvs.go:73 +0x1a2
              cmd/go/internal/modload.ReloadBuildList()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:288 +0x91
              cmd/go/internal/modload.LoadBuildList()
                  /home/mvdan/tip/src/cmd/go/internal/modload/load.go:281 +0x38
              cmd/go/internal/modload.listModules()
                  /home/mvdan/tip/src/cmd/go/internal/modload/list.go:43 +0x47
              cmd/go/internal/modload.ListModules()
                  /home/mvdan/tip/src/cmd/go/internal/modload/list.go:20 +0x6b
              cmd/go/internal/list.runList()
                  /home/mvdan/tip/src/cmd/go/internal/list/list.go:388 +0x54e
              main.main()
                  /home/mvdan/tip/src/cmd/go/main.go:219 +0xb21
            ==================
            Found 1 data race(s)
            [exit status 66]
            FAIL: testdata/script/mod_getmode_vendor.txt:9: unexpected command failure

The tests just hang after that (10m timeout), and I haven't tried to see if a longer timeout would give any different result. /cc @bcmills
@mvdan mvdan added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. modules labels Mar 3, 2019
@bcmills bcmills self-assigned this Mar 4, 2019
@bcmills
Copy link
Contributor

bcmills commented Mar 4, 2019

Argh. I've been wanting to better-encapsulate access to the build list anyway, so this will give me some properties to fix in the process.

Thanks for the report.

@bcmills bcmills added this to the Go1.13 milestone Mar 4, 2019
@gopherbot
Copy link

Change https://golang.org/cl/170597 mentions this issue: cmd/go/internal/lockedfile: add a sync.Mutex to lockedfile.Mutex

@gopherbot
Copy link

Change https://golang.org/cl/170598 mentions this issue: cmd/go/internal/modload: fix aliasing bug in (*mvsReqs).Required with -mod=vendor

gopherbot pushed a commit that referenced this issue Apr 3, 2019
cmd/go/internal/lockedfile: add a sync.Mutex to lockedfile.Mutex
8c896aa 
The compiler (and race detector) don't interpret locking a file as a
synchronization operation, so we add an explicit (and redundant)
sync.Mutex to make that property clear.

The additional synchronization makes it safe to parallelize the tests
in cmd/go/internal/modfetch/coderepo_test.go, which cuts the wall time
of that test by around 50%.

Updates #30550

Change-Id: Ief3479020ebf9e0fee524a4aae5568697727c683
Reviewed-on: https://go-review.googlesource.com/c/go/+/170597
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@golang golang locked and limited conversation to collaborators Apr 2, 2020
@rsc rsc unassigned bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Projects
None yet
Milestone
Go1.13
Development

No branches or pull requests
3 participants
@mvdan @bcmills @gopherbot