cmd/go: go test -coverpkg=./... -race does not rebuild sync/atomic #18486

hirochachacha · 2017-01-01T11:02:54Z

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

go version devel +e776975 Sat Dec 31 18:54:27 2016 +0000 linux/amd64

What operating system and processor architecture are you using (`go env`)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/travis/gopath"
GORACE=""
GOROOT="/home/travis/.gimme/versions/go"
GOTOOLDIR="/home/travis/.gimme/versions/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build901867199=/tmp/go-build"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

create an empty package https://github.com/hirochachacha/empty
add .travis.yml

language: go
go:
    - 1.5
    - 1.6
    - 1.7
    - tip
script:
- go test -coverpkg=./... -race -v

then, I see a build error. https://travis-ci.org/hirochachacha/empty

$ go test -coverpkg=./... -race -v
# github.com/hirochachacha/empty
/tmp/go-build559712700/github.com/hirochachacha/empty/_obj/empty.go:3: can't find import: "sync/atomic"

What did you expect to see?

no errors.

What did you see instead?

an error.

I could not reproduce it on my mac and linux.
However, I can reproduce it on a docker container.

docker run -it quay.io/travisci/travis-go /bin/bash

and,

su - travis
GIMME_OUTPUT=$(gimme tip) && eval "$GIMME_OUTPUT"
export GOPATH=$HOME/gopath
export PATH=$HOME/gopath/bin:$PATH
go get github.com/hirochachacha/empty
cd $HOME/gopath/src/github.com/hirochachacha/empty
go test -coverpkg=./... -race -v

I have no clue. It may be travis's issue? or something I am missing.

Thanks.

The text was updated successfully, but these errors were encountered:

davecheney · 2017-01-01T11:10:17Z

1.7 is go 1.7, not 1.7.4. if you want the latest in the 1.7 series, use 1.7.x. Repeat for other versions.

…

On Sun, 1 Jan 2017, 22:02 hirochachacha ***@***.***> wrote: Please answer these questions before submitting your issue. Thanks! What version of Go are you using (go version)? go version devel +e776975 Sat Dec 31 18:54:27 2016 +0000 linux/amd64 What operating system and processor architecture are you using (go env)? GOARCH="amd64" GOBIN="" GOEXE="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOOS="linux" GOPATH="/home/travis/gopath" GORACE="" GOROOT="/home/travis/.gimme/versions/go" GOTOOLDIR="/home/travis/.gimme/versions/go/pkg/tool/linux_amd64" GCCGO="gccgo" CC="gcc" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build901867199=/tmp/go-build" CXX="g++" CGO_ENABLED="1" PKG_CONFIG="pkg-config" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" What did you do? If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best. 1. create an empty package https://github.com/hirochachacha/empty 2. add .travis.yml language: go go: - 1.5 - 1.6 - 1.7 - tip script: - go test -coverpkg=./... -race -v 1. then, I see a build error. https://travis-ci.org/hirochachacha/empty $ go test -coverpkg=./... -race -v # github.com/hirochachacha/empty /tmp/go-build559712700/github.com/hirochachacha/empty/_obj/empty.go:3 <http://github.com/hirochachacha/empty/tmp/go-build559712700/github.com/hirochachacha/empty/_obj/empty.go:3>: can't find import: "sync/atomic" What did you expect to see? no errors. What did you see instead? an error. I could not reproduce it on my mac and linux. However, I can reproduce it on a docker container. docker run -it quay.io/travisci/travis-go /bin/bash and, su - travis GIMME_OUTPUT=$(gimme tip) && eval "$GIMME_OUTPUT" export GOPATH=$HOME/gopath export PATH=$HOME/gopath/bin:$PATH go get github.com/hirochachacha/empty cd $HOME/gopath/src/github.com/hirochachacha/empty go test -coverpkg=./... -race -v I have no clue. It may be travis's issue? or something I am missing. Thanks. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#18486>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA3151XHQ-evzajX5dZm3ZK8IeOAYks5rN4ffgaJpZM4LYsfN> .

hirochachacha · 2017-01-01T11:15:09Z

@davecheney Good to know, thanks! But the problem is caused on the tip.

JayNakrani · 2017-01-01T17:57:14Z

-coverpkg=./... and -race will work alone, but both of the flags together fail.

$ go test -coverpkg=./... -v
?   	github.com/hirochachacha/empty	[no test files]

$ go test -race -v
?   	github.com/hirochachacha/empty	[no test files]

$ go test -coverpkg=./... -race -v
# github.com/hirochachacha/empty
/tmp/go-build463635544/github.com/hirochachacha/empty/_obj/empty.go:3: can't find import: "sync/atomic"

$ cat /tmp/go-build595576066/github.com/hirochachacha/empty/_obj/empty.go -n
     1	package empty
     2	
     3	import _cover_atomic_ "sync/atomic"
     4	
     5	var _ = _cover_atomic_.AddUint32
     6	
     7	var GoCover_0 = struct {
     8		Count     [0]uint32
     9		Pos       [3 * 0]uint32
    10		NumStmt   [0]uint16
    11	} {
    12		Pos: [3 * 0]uint32{
    13		},
    14		NumStmt: [0]uint16{
    15		},
    16	}

I've isolated the issue to, cmd/compile not being able to compile a program importing std-lib with -race. It can compile it without -race though.

Smaller repro case,

$ cat /tmp/p.go 
package p

import _ "sync/atomic"

$ go tool compile -o /dev/null /tmp/p.go 
# ... succeeds ...

$ go tool compile -o /dev/null -race /tmp/p.go 
/tmp/p.go:3: can't find import: "sync/atomic"

Looking into why cmd/compile fails to compile it with -race.

davecheney · 2017-01-01T18:32:58Z

Is there a GOROOT/pkg/linux_amd64_race/sync/atomic.a file?

JayNakrani · 2017-01-01T18:40:01Z

Nope. And that's the issue. findpkg() is adding _race to the import path. I am not sure what the correct behavior should be for std-imports. Should it be importing non-instrumented version of std libs?

davecheney · 2017-01-01T18:41:04Z

Yes, only instrument the package under test.

JayNakrani · 2017-01-01T18:59:32Z

Okay, thanks. I'll tackle this in 1.9.
Because this is not a regression from 1.7, the fix can probably wait for 1.9.

hirochachacha · 2017-01-01T23:00:47Z

I think this is a regression. please see https://travis-ci.org/hirochachacha/empty.

hirochachacha · 2017-01-02T01:27:12Z

I think this is a regression. please see https://travis-ci.org/hirochachacha/empty.

I understand.

gimme 1.7 download binary package which have "pkg/linux_amd64_race" directory.
gimme tip download source code and run ./make.bash.

./make.bash doesn't create "pkg/$GOOS_$GOARCH_race". (while ./all.bash create the directory)
So this is not a platform specific issue nor a regression. I can reproduce it on my mac.

go test should create "pkg/$GOOS_$GOARCH_race" if it is missing?

davecheney · 2017-01-02T02:17:41Z

Yeah, thus is a bug, i suspect on 1.7 as well, but we don't notice it because the released versions of go have the race runtime packages built ahead of time. Imo go test cover should add an explicit dependency on sync/atomic which will cause it to be built in race mode. It's probably not hard to fix and if a test in cmd/go can be written I think we should try to close this for 1.8

…

On Mon, 2 Jan 2017, 12:27 hirochachacha ***@***.***> wrote: I think this is a regression. please see https://travis-ci.org/hirochachacha/empty. I understand. gimme 1.7 download binary package which have "pkg/linux_amd64_race" directory. gimme tip download source code and run ./make.bash. ./make.bash doesn't create "pkg/$GOOS_$GOARCH_race". (while ./all.bash create the directory) So this is not a platform specific issue nor a regression. I can reproduce it on my mac. go test should create "pkg/$GOOS_$GOARCH" if it is missing? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#18486 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA73EWJewJ6z-KJOyea_Z17Fg7HjMks5rOFJxgaJpZM4LYsfN> .

JayNakrani · 2017-01-03T17:31:26Z

Yes, only instrument the package under test.

I thought about this and it doesn't add up. When compiling with -race, it has to import race instrumented versions of all packages. Without that, it wouldn't catch race between a function in user's package and another function in std-lib.

/cc @dvyukov, to confirm.

If that's the case, then cmd/compile is working as expected; Although, it could give better error message mentioning that it was looking for race/msan instrumented version of the package.

davecheney · 2017-01-03T19:59:25Z

Sorry, I was not clear. I should have said "only instrument for test coverage"

…

On Wed, 4 Jan 2017, 04:31 Dhananjay Nakrani ***@***.***> wrote: Yes, only instrument the package under test. I thought about this and it doesn't add up. When compiling with -race, it has to import race instrumented versions of all packages. Without that, it wouldn't catch race between a function in user's package and another in std-lib. /cc @dvyukov <https://github.com/dvyukov>, to confirm. If that's the case, then cmd/compile is working as expected; Although, it could give better error message mentioning that it was looking for race/msan instrumented version of the package. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#18486 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA-VOu2Wvos6FHXx79NSgw5RavwhIks5rOoXvgaJpZM4LYsfN> .

JayNakrani · 2017-01-03T21:24:59Z

Ah, okay. Sent out the CL for better error.

After golang.org/cl/34798:

$ compile -o /dev/null -race /tmp/p.go
/tmp/p.go:3: can't find race-instrumented version of import: "fmt"

gopherbot · 2017-01-03T21:25:13Z

CL https://golang.org/cl/34798 mentions this issue.

gopherbot · 2017-01-04T15:35:19Z

CL https://golang.org/cl/34830 mentions this issue.

hirochachacha mentioned this issue Jan 2, 2017

should use ./all.bash instead of ./make.bash for installation from source? travis-ci/gimme#77

Closed

rsc added this to the Go1.8 milestone Jan 4, 2017

rsc changed the title ~~'go test -coverpkg=./... -race' on the empty package fails on travis CI.~~ cmd/go: go test -coverpkg=./... -race does not rebuild sync/atomic Jan 4, 2017

gopherbot closed this as completed in 3fa53f1 Jan 4, 2017

starius mentioned this issue May 27, 2017

cmd/go: go get -race can't find "sync/atomic" #20512

Closed

golang locked and limited conversation to collaborators Jan 4, 2018

gopherbot added the FrozenDueToAge label Jan 4, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: go test -coverpkg=./... -race does not rebuild sync/atomic #18486

cmd/go: go test -coverpkg=./... -race does not rebuild sync/atomic #18486

hirochachacha commented Jan 1, 2017

davecheney commented Jan 1, 2017 via email

hirochachacha commented Jan 1, 2017

JayNakrani commented Jan 1, 2017 •

edited

davecheney commented Jan 1, 2017

JayNakrani commented Jan 1, 2017

davecheney commented Jan 1, 2017

JayNakrani commented Jan 1, 2017

hirochachacha commented Jan 1, 2017

hirochachacha commented Jan 2, 2017 •

edited

davecheney commented Jan 2, 2017 via email

JayNakrani commented Jan 3, 2017 •

edited

davecheney commented Jan 3, 2017 via email

JayNakrani commented Jan 3, 2017

gopherbot commented Jan 3, 2017

gopherbot commented Jan 4, 2017

cmd/go: go test -coverpkg=./... -race does not rebuild sync/atomic #18486

cmd/go: go test -coverpkg=./... -race does not rebuild sync/atomic #18486

Comments

hirochachacha commented Jan 1, 2017

What version of Go are you using (go version)?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

davecheney commented Jan 1, 2017 via email

hirochachacha commented Jan 1, 2017

JayNakrani commented Jan 1, 2017 • edited

davecheney commented Jan 1, 2017

JayNakrani commented Jan 1, 2017

davecheney commented Jan 1, 2017

JayNakrani commented Jan 1, 2017

hirochachacha commented Jan 1, 2017

hirochachacha commented Jan 2, 2017 • edited

davecheney commented Jan 2, 2017 via email

JayNakrani commented Jan 3, 2017 • edited

davecheney commented Jan 3, 2017 via email

JayNakrani commented Jan 3, 2017

gopherbot commented Jan 3, 2017

gopherbot commented Jan 4, 2017

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

JayNakrani commented Jan 1, 2017 •

edited

hirochachacha commented Jan 2, 2017 •

edited

JayNakrani commented Jan 3, 2017 •

edited