Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/compile: "can't find import" error due to race during concurrent build #22423

Closed
bassam opened this issue Oct 24, 2017 · 5 comments
Closed

cmd/compile: "can't find import" error due to race during concurrent build #22423

bassam opened this issue Oct 24, 2017 · 5 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@bassam
Copy link

bassam commented Oct 24, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.9.1 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/rook/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

on a machine with 16 cores (like a c4.4xlarge instance in EC2) I run the rook build system as follows:

git clone https://github.com/rook/rook
make distclean
build/run make -j16 build.all V=1 | tee build.log

What did you expect to see?

I expect the build to succeed every time.

What did you see instead?

Instead, I see intermittent errors that look like a race in the go build process. For example:

# github.com/rook/rook/vendor/k8s.io/api/autoscaling/v1
vendor/k8s.io/api/autoscaling/v1/types.go:20:2: can't find import: "github.com/rook/rook/vendor/k8s.io/api/core/v1"
mkdir -p /home/rook/go/pkg/linux_arm64_static/github.com/rook/rook/vendor/k8s.io/api/core/
cp $WORK/github.com/rook/rook/vendor/k8s.io/api/core/v1.a /home/rook/go/pkg/linux_arm64_static/github.com/rook/rook/vendor/k8s.io/api/core/v1.a

Note that the cp command happened after the can't find import error.
@bassam bassam mentioned this issue Oct 24, 2017
Closed
@ALTree ALTree changed the title "can't find import" error due to race during concurrent build cmd/compile: "can't find import" error due to race during concurrent build Oct 24, 2017
@ALTree ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 24, 2017
@bassam
Copy link
Author

bassam commented Oct 24, 2017

Note that our build system runs concurrent go build from the makefile. Not sure if this is causing .a packages to be removed form the file system and resulting in errors like the can't import

@ALTree
Copy link
Member

ALTree commented Oct 24, 2017 •
edited

Easy to test if there is an issue in your makefile: force sequential compilation in the Go compiler. Does the error go away?

@bassam
Copy link
Author

bassam commented Oct 24, 2017

@ALTree with export GO19CONCURRENTCOMPILATION=0 and after 5 builds I've not seen the build error. Note however that we are running concurrent go build commands for the same platform and they might be attempting to build similar import packages. Is this a supported scenario?

Looking at the copyFile function I see that it attempts to remove the file before copying. I can't imagine this being safe to do when concurrent builds of go build are running and import the same packages.

I'm going to change our makefile to only run concurrently across build platforms, and sequentially within a build platform. I'll report back here.

@ianlancetaylor
Copy link
Contributor

Running concurrent go build commands in the same GOPATH is clearly unsafe. Don't do that.

@bassam
Copy link
Author

bassam commented Oct 24, 2017

@ianlancetaylor yes I see why now. Fixed our makefile to avoid that. Thanks!

@bassam bassam closed this as completed Oct 24, 2017
@golang golang locked and limited conversation to collaborators Oct 24, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@bassam @ianlancetaylor @ALTree @gopherbot