Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go build rebuilds standard library if PWD contained a symlink at toolchain build time #20630

Closed
imarko opened this issue Jun 9, 2017 · 5 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.

Comments

@imarko
Copy link

imarko commented Jun 9, 2017

What version of Go are you using (go version)?

go version devel +504b8d15d6 Fri Jun 9 15:16:35 2017 +0000 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/user/go"
GORACE=""
GOROOT="/tmp/go-build/go"
GOTOOLDIR="/tmp/go-build-symlinked/go/pkg/tool/linux_amd64"
GCCGO="/usr/bin/gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build462810260=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

(note that GOROOT and GOTOOLDIR don't match)

What did you do?

cd /tmp
mkdir go-build
ln -s go-build go-build-symlinked
cd go-build-symlinked
git clone https://github.com/golang/go
cd /tmp/go-build-symlinked/go/src
./make.bash
echo 'package main\nimport "fmt"\nfunc main() {fmt.Println("hello world")}' >| /tmp/hello.go
/tmp/go-build-symlinked/go/bin/go build -v /tmp/hello.go

What did you expect to see?

command-line-arguments

(meaning that only hello.go gets rebuilt)

What did you see instead?

runtime/internal/sys
runtime/internal/atomic
runtime
errors
internal/race
sync/atomic
internal/cpu
unicode/utf8
math
sync
io
syscall
strconv
reflect
time
internal/poll
os
fmt
command-line-arguments

indicating that all stdlib packages were rebuilt, resulting in a much slower build.

I bisected this to the following change:

b8c7fdd

https://go-review.googlesource.com/42533

This issue probably won't affect many people, you need to build the toolchain yourself and the build path must contain a symlink somewhere. But when it does happen the symptoms are rather confusing: very slow builds for no apparent reason.

I personally encountered this because I build the toolchain in a workspace that is symlinked into my homedir from another filesystem.

@bradfitz
Copy link
Contributor

bradfitz commented Jun 9, 2017

Are you sure you're not just hitting the known issue that go build doesn't cache intermediate build results but go install does? I'm pretty sure that's what this is.

That issue is known and being worked on for Go 1.10. For now, just remember to run go test -i or go install before doing incremental work. Or just avoid go build.

@bradfitz
Copy link
Contributor

bradfitz commented Jun 9, 2017

(Caching work is #4719)

@bradfitz bradfitz changed the title go build rebuilds standard library if PWD contained a symlink at toolchain build time cmd/go: go build rebuilds standard library if PWD contained a symlink at toolchain build time Jun 9, 2017
@bradfitz bradfitz added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Jun 9, 2017
@imarko
Copy link
Author

imarko commented Jun 9, 2017

Brad,

I think there is more to it than that. I wouldn't expect the runtime and standard library packages rebuilt even during the first build. And it only happens in the symlinked toolchain path (at toolchain build time) situation, and only starting from b8c7fdd

I can work around this very easily for my own use (by avoiding building the Go toolchain in a symlinked path) but wanted to raise this in case others encounter it.

@rsc
Copy link
Contributor

rsc commented Nov 1, 2017

I hadn't seen this issue yet, but I think that https://go-review.googlesource.com/74250 fixed it. Can you please check, @imarko?

@imarko
Copy link
Author

imarko commented Nov 1, 2017

@rsc I just checked and it turns out that this was already fixed with https://go-review.googlesource.com/48550 back in July. Current master is good too so I think this can be closed.

@imarko imarko closed this as completed Nov 1, 2017
@golang golang locked and limited conversation to collaborators Nov 1, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

4 participants