cmd/go: go install -a <pkg> attempts to rebuild the stdlib #22469
Comments
To clarify - do you mean on all cases, or just on those where |
|
Only where the goroot is not writeable (notably the case on many machines that get installed via OS package managers). Works fine with a writeable home directory (such as on my machine where I built from source) |
|
For myself, I found this issue using the official macOS pkg file for Go 1.9.2 |
|
Is this a change in 1.9.2, or has it always been so? (I would expect Does |
|
I've seen similar bug reports going back 4 years, so it's not new, AFAICT. -a rebuilding and reinstalling the stdlib may be expected, depending on your assumptions, but it's still going to Just Not Work in a very large number of cases... and is almost assuredly not what most people intend when they type -i is not the right functionality. That installs dependencies, not the binary itself. I have a fix for my specific use case, by effectively copying what |
|
This is working as intended. See the long comment starting at https://golang.org/src/cmd/go/internal/load/pkg.go#L1213 . The need to use Closing because the current behavior is intentional. |
|
@ianlancetaylor I think that any flag which is guaranteed to fail a large percentage of the time (not sure what the stats are on go installs... but maybe 50% or more?) is a mistake, whether or not it's working as intended. Intending that a flag will fail with an obscure error is bad UX. Ideally, IMO, the flag should not rebuild the stdlib. At a minimum, the error message should be cleaned up to make it more clear what's going on, and that this is expected. |
|
It fails about 0% of time on a developer machine with Go installed from source. What should replace the -a flag on those machines? |
|
If I want to rebuild the stdlib, I'd go to my go source tree and re-run make.bash. But I've only wanted/needed to to do that exactly once in the past 5 years (outside of explicitly changing go versions). |
|
make.bash takes longer to run.
If one works on a patch of the compiler and/or stdlib, -a can be useful. I understand your experience differs, but you should consider that other people are doing different things as well and taking away useful features they use is not really an option. |
|
@natefinch The |
|
Change https://golang.org/cl/75850 mentions this issue: |
This CL makes "go install" behave the way many users expect: install only the things named on the command line. Future builds still run as fast, thanks to the new build cache (CL 75473). To install dependencies as well (the old behavior), use "go install -i". Actual definitions aside, what most users know and expect of "go install" is that (1) it installs what you asked, and (2) it's fast, unlike "go build". It was fast because it installed dependencies, but installing dependencies confused users repeatedly (see for example #5065, #6424, #10998, #12329, "go build" and "go test" so that they could be "fast" too, but that only created new opportunities for confusion. We also had to add -installsuffix and then -pkgdir, to allow "fast" even when dependencies could not be installed in the usual place. The recent introduction of precise content-based staleness logic means that the go command detects the need for rebuilding packages more often than it used to, with the consequence that "go install" rebuilds and reinstalls dependencies more than it used to. This will create more new opportunities for confusion and will certainly lead to more issues filed like the ones listed above. CL 75743 introduced a build cache, separate from the install locations. That cache makes all operations equally incremental and fast, whether or not the operation is "install" or "build", and whether or not "-i" is used. Installing dependencies is no longer necessary for speed, it has confused users in the past, and the more accurate rebuilds mean that it will confuse users even more often in the future. This CL aims to end all that confusion by not installing dependencies by default. By analogy with "go build -i" and "go test -i", which still install dependencies, this CL introduces "go install -i", which installs dependencies in addition to the things named on the command line. Fixes #5065. Fixes #6424. Fixes #10998. Fixes #12329. Fixes #18981. Fixes #22469. Another step toward #4719. Change-Id: I3d7bc145c3a680e2f26416e182fa0dcf1e2a15e5 Reviewed-on: https://go-review.googlesource.com/75850 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: David Crawshaw <crawshaw@golang.org>
|
@ianlancetaylor is absolutely right: this was working as intended. If he hadn't replied first, I'd have told you the same thing (as I have for years, for example in #6424). But it was also clearly not working as expected by many users - this same bug has been filed at least half a dozen times. I had originally planned to stop after content-based staleness and save the build cache for Go 1.11, but the expectation mismatch here was only going to be made worse by content-based staleness, which (correctly) rebuilds dependencies more than before, and "go install" would have installed them. To head that off, I pulled in all the caching we'd planned for Go 1.11 and then made "go install" no longer be defined to install dependencies. (The caching eliminates any performance hit from that semantic change.) My post on golang-dev has more details. Thanks for raising the issue. Hopefully no big unforeseen problem arises, and we can keep the new meaning for Go 1.10. |
|
Thanks, Russ. I read the post on golang-dev and it looks awesome. Can't
wait to see it in action.
…On Fri, Nov 3, 2017, 11:38 PM Russ Cox ***@***.***> wrote:
@ianlancetaylor <https://github.com/ianlancetaylor> is absolutely right:
this was working as intended. If he hadn't replied first, I'd have told you
the same thing (as I have for years, for example in #6424
<#6424>). But it was also clearly not
working as expected by many users - this same bug has been filed at least
half a dozen times.
I had originally planned to stop after content-based staleness and save
the build cache for Go 1.11, but the expectation mismatch here was only
going to be made worse by content-based staleness, which (correctly)
rebuilds dependencies more than before, and "go install" would have
installed them. To head that off, I pulled in all the caching we'd planned
for Go 1.11 and then made "go install" no longer be defined to install
dependencies. (The caching eliminates any performance hit from that
semantic change.) My post on golang-dev
<https://groups.google.com/forum/#!topic/golang-dev/qfa3mHN4ZPA> has more
details.
Thanks for raising the issue. Hopefully no big unforeseen problem arises,
and we can keep the new meaning for Go 1.10.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#22469 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ADCcyO83tPtUhXrjOt4HdzCxUXOGu5Z9ks5sy9wigaJpZM4QJO_4>
.
|
Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (
go version)?1.9.2 OSX
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env)?macOS 10.12.6
$ go version
go version go1.9.2 darwin/amd64
$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/jake/Projects/Labs/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/h2/whwtkkx168n8333397n69m2r0000gn/T/go-build684654678=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config
What did you do?
go install -a -ldflags=-X "github.com/magefile/mage/mage.timestamp=2017-10-27T09:39:17-05:00" -X "github.com/magefile/mage/mage.commitHash=accd8ef" -X "github.com/magefile/mage/mage.gitTag=v2.0.1-3-gaccd8ef" github.com/magefile/mage
go install runtime/internal/sys: open /usr/local/go/pkg/darwin_amd64/runtime/internal/sys.a: permission denied
Thoughts
I see a lot of comments about this specific problem in old issues. It seems like go install -a is trying to rebuild the standard library on machines with a goroot that is not in a user-editable directory.
My build script uses
go install -abecausego installwill no-op if you have already built the code without any changes, but in this case, my install script is building with ldflags, so the binary that is being output will be different than it would be without the ldflags, even though nothing has changed on disk.go install -ashould be smart enough not to rebuild the stdlib.The text was updated successfully, but these errors were encountered: