Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: go list reports net as "stale" right after ./make.bash (but only inside GOROOT/src) #44725

Closed
dmitshur opened this issue Mar 1, 2021 · 5 comments
Closed

cmd/go: go list reports net as "stale" right after ./make.bash (but only inside GOROOT/src) #44725

dmitshur opened this issue Mar 1, 2021 · 5 comments
Labels
FrozenDueToAge GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Go1.17

Comments

@dmitshur
Copy link
Contributor

dmitshur commented Mar 1, 2021 •
edited

What version of Go are you using (go version)?

$ go version
go version go1.16 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/dmitshur/Library/Caches/go-build"
GOENV="/Users/dmitshur/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/dmitshur/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/dmitshur/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/zb/5p8cwfhj29gf_m8vdy8ylmlr00jwcj/T/go-build203663307=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I tried building Go from source at Go 1.16 and 1.15.8, comparing whether the net package is reported as stale or not right after a ./make.bash.

What did you expect to see?

I expected that net is reported not stale after ./make.bash with both Go versions.

What did you see instead?

net is reported as not stale when using 1.15.8 both inside GOROOT/src and outside:

$ cd $(mktemp -d)  # Or equivalent that doesn't involve symlinks.
$ git clone https://go.googlesource.com/go && cd go
$ git checkout go1.15.8
$ cd src && ./make.bash && cd ..
$ export PATH="$(pwd)/bin:$PATH"
$ go list -f '{{.ImportPath}}: {{if .Stale}}stale; reason={{.StaleReason}}{{else}}not stale{{end}}' net
net: not stale
$ cd src
$ go list -f '{{.ImportPath}}: {{if .Stale}}stale; reason={{.StaleReason}}{{else}}not stale{{end}}' net
net: not stale

net is reported as not stale when using 1.16 outside GOROOT/src, but stale inside:

$ cd $(mktemp -d)  # Or equivalent that doesn't involve symlinks.
$ git clone https://go.googlesource.com/go && cd go
$ git checkout go1.16
$ cd src && ./make.bash && cd ..
$ export PATH="$(pwd)/bin:$PATH"
$ go list -f '{{.ImportPath}}: {{if .Stale}}stale; reason={{.StaleReason}}{{else}}not stale{{end}}' net
net: not stale
$ cd src
$ go list -f '{{.ImportPath}}: {{if .Stale}}stale; reason={{.StaleReason}}{{else}}not stale{{end}}' net
net: stale; reason=stale dependency: golang.org/x/net/dns/dnsmessage

I ran into this while investigating #44714 with @toothrot, and we both thought this was worth reporting so that it can be investigated as needed.

(While writing up this issue report, I realized it makes a difference whether the go list command is run in GOROOT/src or outside, which is not something I realized earlier.)

CC @jayconrod, @bcmills, @matloob.
@dmitshur dmitshur added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go labels Mar 1, 2021
@bcmills bcmills added this to the Unplanned milestone Mar 2, 2021
@bcmills
Copy link
Contributor

bcmills commented Mar 2, 2021

This is possibly a side-effect of the way the vendored std module dependencies work in module mode.

When you are inside GOROOT/src, the module dependencies of std have their actual, external-module identities. (That makes it possible to run commands like go mod tidy and go mod vendor.)

However, when you move outside of GOROOT/src, those vendored dependencies can no longer be ordinary module dependencies: their versions are fixed to what is shipped in std and the packages no longer retain their module identities. For example, the vendored copy of golang.org/x/net/dns/dnsmessage turns into the package vendor/golang.org/x/net/dns/dnsmessage.

When that happens, the package itself becomes stale, because its identity (and therefore its debug information, and perhaps its export metadata) has changed.

@bcmills
Copy link
Contributor

bcmills commented Mar 2, 2021 •
edited

This probably changed as of CL 251159, which removed special-case code for go get and go mod … within std.

That special-case forced the vendored std module dependencies to their vendor/… paths even when running the go command within GOROOT/src.

@jayconrod, @matloob: do you think we should restore that special-case (so that the dependencies of packages in std are identical whether inside or outside of the std module), or keep it as it is in 1.16 (so that the dependencies reported for those packages for general go commands match the paths used in go get and go mod tidy commands)?

@jayconrod
Copy link
Contributor

I'd lean toward restoring the special case (I hate saying that). It's pretty confusing that these packages have multiple identities. It's taken me longer than I'd like to admit to re-read your comments and the CL to work out what's going on.

@bcmills
Copy link
Contributor

bcmills commented Mar 2, 2021

Yeah, I agree. I have a CL in progress. 😁

@bcmills bcmills self-assigned this Mar 2, 2021
@gopherbot
Copy link

Change https://golang.org/cl/297869 mentions this issue: cmd/go: resolve std-vendored dependencies as std packages except in 'go get' and 'go mod'

@bcmills bcmills modified the milestones: Unplanned, Go1.17 Mar 2, 2021
@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Mar 9, 2021
@bcmills bcmills mentioned this issue Mar 10, 2021
Open
This was referenced May 20, 2021
Closed
Closed
@golang golang locked and limited conversation to collaborators Mar 9, 2022
@rsc rsc unassigned bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Go1.17
Development

No branches or pull requests
4 participants
@jayconrod @dmitshur @bcmills @gopherbot