Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: module cache can cache inappropriate commits #29099

Closed
rogpeppe opened this issue Dec 4, 2018 · 5 comments
Closed

cmd/go: module cache can cache inappropriate commits #29099

rogpeppe opened this issue Dec 4, 2018 · 5 comments
Labels
FrozenDueToAge GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Go1.13

Comments

@rogpeppe
Copy link
Contributor

rogpeppe commented Dec 4, 2018

What version of Go are you using (go version)?

$ go version
go version devel +22dbc96d8a Tue Nov 27 09:26:16 2018 +0000 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/rog/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/rog/src/go"
GOPROXY="http://localhost:3000"
GORACE=""
GOROOT="/home/rog/go"
GOTMPDIR=""
GOTOOLDIR="/home/rog/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/rog/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build723184334=/tmp/go-build -gno-record-gcc-switches"
% rm go.mod
% go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/rog/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/rog/src/go"
GOPROXY="http://localhost:3000"
GORACE=""
GOROOT="/home/rog/go"
GOTMPDIR=""
GOTOOLDIR="/home/rog/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build238480985=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I was building a module with a dependency that had been forked.
The go.mod file specified this dependency.

github.com/altoros/gosigma v0.0.0-20170523021020-a27b59fe2be9 // indirect

Note that commit a27b59fe2be9 does not appear in the github.com/altoros/gosigma repository, although it does appear in a PR on that repository. The dependency does not contain a go.mod file.

The commit is also in a fork of the repository at its HEAD (github.com/juju/gosigma).

My build succeeds locally, and I can see that somehow that commit has made its way into $GOPATH/pkg/mod:

% cd $GOPATH/pkg/mod
% find . -name '*sigma*'
./cache/download/github.com/juju/gosigma
./cache/download/github.com/altoros/gosigma
./github.com/juju/juju@v0.0.0-20181115160945-c44e31634a53/provider/cloudsigma
./github.com/juju/gosigma@v0.0.0-20170523021020-a27b59fe2be9
./github.com/juju/gosigma@v0.0.0-20170523021020-a27b59fe2be9/gosigma.go
./github.com/juju/gosigma@v0.0.0-20170523021020-a27b59fe2be9/gosigma_test.go
./github.com/juju/juju@v0.0.0-20181130004655-17e480906ca9/provider/cloudsigma
./github.com/juju/juju@v0.0.0-20181030015355-5e39de8318f2/provider/cloudsigma
./github.com/juju/juju@v0.0.0-20181030015355-5e39de8318f2/patches/gosigma-race-fix.diff
./github.com/juju/juju@v0.0.0-20180829075458-b95187af705e/provider/cloudsigma
./github.com/juju/juju@v0.0.0-20180829075458-b95187af705e/patches/gosigma-race-fix.diff
./github.com/juju/juju@v0.0.0-20181130132359-7f8f23fe15cd/provider/cloudsigma
./github.com/altoros/gosigma@v0.0.0-20150408145232-31228935eec6
./github.com/altoros/gosigma@v0.0.0-20150408145232-31228935eec6/gosigma.go
./github.com/altoros/gosigma@v0.0.0-20150408145232-31228935eec6/gosigma_test.go
./github.com/altoros/gosigma@v0.0.0-20170523021020-a27b59fe2be9
./github.com/altoros/gosigma@v0.0.0-20170523021020-a27b59fe2be9/gosigma.go
./github.com/altoros/gosigma@v0.0.0-20170523021020-a27b59fe2be9/gosigma_test.go

When I pushed my package to be tested with CI, the CI failed because it could not find github.com/altoros/gosigma@v0.0.0-20170523021020-a27b59fe2be9.

Unfortunately, I don't know how this happened, but I'm recording the issue here as a placeholder in case other people see the same thing. When I tried to reproduce the issue, I failed. I tried with a fresh GOPATH and go get github.com/juju/gosigma@v0.0.0-20170523021020-a27b59fe2be9, in case that would poison the cache, but it didn't.

What did you expect to see?

I expected to get a build failure locally.

What did you see instead?

The build succeeded locally but failed with a fresh $GOPATH.
@bcmills
Copy link
Contributor

bcmills commented Dec 5, 2018

Is it possible that your module cache was very old?

https://golang.org/cl/126956 (released in go1.11) is the last change to that logic I'm aware of, but if you happened to have a larger git history already fetched into your module cache it could include otherwise-unreachable commits.

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 5, 2018
@bcmills bcmills added this to the Go1.13 milestone Dec 5, 2018
@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Dec 5, 2018
@rogpeppe
Copy link
Contributor Author

rogpeppe commented Dec 6, 2018

I'm fairly sure that I'd removed the cache directory relatively recently. FWIW the modification date on the erroneous cache entry was in the last couple of days.

@bcmills bcmills removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jan 15, 2019
@bcmills
Copy link
Contributor

bcmills commented Jan 15, 2019

@rogpeppe, have you seen this again recently? Without steps to reproduce I'm not sure there's anything we can do to move forward on this. (My best guess is maybe a difference in the way git fetch works across git versions.)

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jan 15, 2019
@rogpeppe
Copy link
Contributor Author

No, I haven't seen this issue again since.

@bcmills
Copy link
Contributor

bcmills commented Jan 16, 2019

Ok. I'm going to close this issue for now, but please do reopen if you see it again.

@bcmills bcmills closed this as completed Jan 16, 2019
@golang golang locked and limited conversation to collaborators Jan 16, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Go1.13
Development

No branches or pull requests
3 participants
@rogpeppe @bcmills @gopherbot