Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: automatic toolchain upgrades make the combination of go version and go install <package>@<version> confusing #66518

Open
dominikh opened this issue Mar 25, 2024 · 7 comments
Open

cmd/go: automatic toolchain upgrades make the combination of go version and go install <package>@<version> confusing #66518

dominikh opened this issue Mar 25, 2024 · 7 comments
Labels
GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@dominikh
Copy link
Member

dominikh commented Mar 25, 2024 •
edited

Go version

that's a good question!

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/dominikh/.cache/go-build'
GOENV='/home/dominikh/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/dominikh/prj/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/dominikh/prj'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/dominikh/prj/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-amd64'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/dominikh/prj/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-amd64/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.1'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/dominikh/bla/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3151298505=/tmp/go-build -gno-record-gcc-switches'

What did you do?

As the author of Staticcheck, I tell my users that to support Go 1.22 code, they have to build Staticcheck with Go 1.22. To assure me that they did, they show me this:

$ go version
go version go1.22.1 linux/amd64

$ go install honnef.co/go/tools/cmd/staticcheck@latest

The problem is that the user's local toolchain is Go 1.21.8, but the module they're currently in is specifying go 1.22.1 in its go.mod. Running go version downloads and runs the newer toolchain, as does go install ., but go install honnef.co/[...] does not.

This is problematic in a multitude of ways.

  • I don't know of any straightforward way to get the local toolchain's version without having to change the current working directory
  • Users wouldn't even know to use it if they aren't aware of this footgun
  • This interaction is just a symptom; the actual problem is that users have no good intuition for which version of Go they're actually using. In fact, this behavior confused 3 of 3 experienced Go users that I showed it to, as well as a dozen users of mine who struggled to build Staticcheck with Go 1.22. Most of them misdiagnosed the problem as the build cache not being invalidated correctly, because they thought they were building Staticcheck with 1.22 when they were in fact using their older, local toolchain.

That go install behaves this way is documented in go help install as

If the arguments have version suffixes (like @latest or @v1.0.0), "go install"
builds packages in module-aware mode, ignoring the go.mod file in the current
directory or any parent directory, if there is one. This is useful for
installing executables without affecting the dependencies of the main module.

This, however, predates the automatic upgrades. In my experience, users who are aware of automatic upgrades think of it as "First, go downloads the specified toolchain, then it runs the subcommand", which is true in most, but not all cases.

The issue I am filing is really a mix of two user reports, one from the perspective of a tool maintainer, and one from the perspective of a user. What they have in common is confusion over the effective Go version.

To avoid any confusion I've included a sample shell session demonstrating the current behavior.


chulak ~ ^$ go version
go version go1.21.8 linux/amd64
chulak ~ ^$ mkdir bla
chulak ~ ^$ cd bla
chulak ~/bla ^$ go mod init example.com
go: creating new go.mod: module example.com
chulak ~/bla ^$ go get go@1.22.1
go: updating go.mod requires go >= 1.22.1; switching to go1.22.1
go: upgraded go 1.21.8 => 1.22.1
chulak ~/bla ^$ go version
go version go1.22.1 linux/amd64
chulak ~/bla ^$ echo "package main\nfunc main(){}" >main.go
chulak ~/bla ^$ rm $GOPATH/bin/example.com
chulak ~/bla ^$ go install
chulak ~/bla ^$ go version $GOPATH/bin/example.com
/home/dominikh/prj/bin/example.com: go1.22.1
chulak ~/bla ^$ rm $GOPATH/bin/staticcheck 
chulak ~/bla ^$ go install honnef.co/go/tools/cmd/staticcheck@v0.4.7
chulak ~/bla ^$ go version $GOPATH/bin/staticcheck 
/home/dominikh/prj/bin/staticcheck: go1.21.8
@seankhliao
Copy link
Member

go env has a read only env GOVERSION=.
perhaps go env should get a new env GOLOCALVERSION, and go version get a second line prefixes with local: ?

@seankhliao seankhliao added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go labels Mar 25, 2024
@thediveo
Copy link

thediveo commented Mar 25, 2024 •
edited

go env has a read only env GOVERSION=. perhaps go env should get a new env GOLOCALVERSION, and go version get a second line prefixes with local: ?

To me, that would make it even more confusing. Please don't try to fix the opaque toolchain behavior by bolting on even more complexity.

@matloob
Copy link
Contributor

matloob commented Apr 4, 2024

I would expect go install package@version to use the toolchain in the go.mod file of the module version being installed. In this case it looks like the latest released version of go-tools is 0.4.7 whose go.mod declares 1.19. Since it's older that 1.21 there's no switching and it uses the local toolchain.

I see that your latest go.mod declares 1.22. When that's released users who build @latest will use go1.22, even if they were running 1.21 (and, let's say their current directory was in a 1.23 module). Is that not what you'd expect to happen with toolchain switching?

To put this another way, I think what makes this the most confusing is that we're crossing the 1.21 toolchain switching version boundary here with a 1.19 module, and a local 1.21 toolchain. This should be less confusing once we're on the other side of that boundary.

@dominikh
Copy link
Member Author

dominikh commented Apr 4, 2024

But if we assume that tools support the last two versions of Go, then the latest release of my module will have a go directive specifying an older version of Go than the latest. Users who are using a newer version of Go would still need to build with that newer version, which brings us back to this issue.

The behavior you describe is generally fine for most software written in Go, but not tools that operate on other Go code.

@matloob
Copy link
Contributor

matloob commented Apr 4, 2024

Hm I think I still need to understand something. Now that 1.21 and 1.22 are the last two versions of Go, can we assume that everyone (that we support) has a version of Go that's capable of switching to a newer toolchain and build the tool using that newest toolchain? Will your tool work with 1.21 code if it's built with 1.22?

@dominikh
Copy link
Member Author

dominikh commented Apr 4, 2024

Will your tool work with 1.21 code if it's built with 1.22?

Yes, but not vice versa.

Is your point that with automatic tool switching, there is no reason to support older versions of Go, and that go-tools should always specify the latest version of Go in its go.mod? Because I think that's problematic.

  • Updating the Go version can change the semantics of the code (cf. the for range loop variable scope change.)
  • It forces everyone who doesn't use automatic upgrades (such as Linux distributions packaging my tools) to support the newer version of Go.
  • If a new version of Go releases and I don't update my go.mod and make a new release in time, we're back to this very issue.

I don't want to be forced to depend on a newer version of Go for users to have an easier time go installing it.

@findleyr
Copy link
Contributor

First of all, I agree that the newly versioned world we live in is confusing, and I don't think we've done enough to mitigate that confusion. For example: the fact that go version depends on the directory from which it is run, yet this is not mentioned by go help version. Nor is it mentioned that go version can fail if the version in the go.mod file is a language version and not a toolchain version. IMO, it may have made more sense for go version to print its local version, and let go list -m go to be the way to get the toolchain version.

However, with respect to forcing toolchain upgrades:

In other words, I am hopeful that in the future most tools can upgrade their toolchain requirements frequently. This will allow for faster iteration with the standard library, and will make it more likely that fixes or new APIs get upstreamed rather than worked around. In another world, it would have been nice for go/parser, go/types, etc. to live in a separate module, but it's too late for that, and forward compatibility provides a similar outcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@dominikh @thediveo @seankhliao @matloob @findleyr