Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/internal/moddeps: test TestAllDependenciesVendored/golang.org/x/tools fails #37929

Closed
dmitshur opened this issue Mar 18, 2020 · 8 comments
Closed

cmd/internal/moddeps: test TestAllDependenciesVendored/golang.org/x/tools fails #37929

dmitshur opened this issue Mar 18, 2020 · 8 comments
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure. Vendoring
Milestone
Go1.15

Comments

@dmitshur
Copy link
Contributor

dmitshur commented Mar 18, 2020 •
edited

What version of Go are you using (go version)?

$ go version                        
go version devel +6412750f32 Wed Mar 18 01:03:36 2020 +0000 darwin/amd64

Does this issue reproduce with the latest release?

No, the cmd/internal/moddeps package is new to master and doesn't exist in Go 1.14.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/dmitshur/Library/Caches/go-build"
GOENV="/Users/dmitshur/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/dmitshur/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/dmitshur/gotip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/dmitshur/gotip/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/dmitshur/gotip/src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/zb/5p8cwfhj29gf_m8vdy8ylmlr00jwcj/T/go-build665868502=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I ran all.bash or go test cmd/internal/moddeps on clean tip:

gotip $ git rev-parse HEAD
6412750f32224f6820f781d09b2092c5c358dddc
gotip $ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

What did you expect to see?

Test TestAllDependenciesVendored passing.

What did you see instead?

Test TestAllDependenciesVendored fails:

src $ ./all.bash
[...]
ok  	cmd/internal/dwarf	0.352s
ok  	cmd/internal/edit	0.711s
ok  	cmd/internal/goobj	2.582s
--- FAIL: TestAllDependenciesVendored (8.66s)
    --- FAIL: TestAllDependenciesVendored/golang.org/x/tools (0.17s)
        moddeps_test.go:126: /Users/dmitshur/gotip/bin/go list -mod=mod -m all: exit status 1
            go: updating go.mod: open /Users/dmitshur/gotip/pkg/obj/gopath/pkg/mod/golang.org/x/tools@v0.0.0-20190325161752-5a8dccf5b48a/go.mod: permission denied
FAIL
FAIL	cmd/internal/moddeps	8.928s
ok  	cmd/internal/obj	0.347s
ok  	cmd/internal/obj/arm64	0.903s
ok  	cmd/internal/obj/riscv	1.430s
[...]
ok  	cmd/pack	3.467s
ok  	cmd/trace	0.331s
ok  	cmd/vet	13.507s
FAIL
go tool dist: Failed: exit status 1

$ go test cmd/internal/moddeps 
--- FAIL: TestAllDependenciesVendored (1.22s)
    --- FAIL: TestAllDependenciesVendored/golang.org/x/tools (0.01s)
        moddeps_test.go:126: /Users/dmitshur/gotip/bin/go list -mod=mod -m all: exit status 1
            go: updating go.mod: open /Users/dmitshur/gotip/pkg/obj/gopath/pkg/mod/golang.org/x/tools@v0.0.0-20190325161752-5a8dccf5b48a/go.mod: permission denied
FAIL
FAIL	cmd/internal/moddeps	1.529s
FAIL

This may be related to #37749.

It fails 100% reproducibly for me. I don't know why it's not caught on builders. If it doesn't reproduce easily for others, let me know, and I'll provide more information about my environment.

/cc @bcmills @matloob @jayconrod
@dmitshur dmitshur added Vendoring NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. modules labels Mar 18, 2020
@dmitshur dmitshur added this to the Go1.15 milestone Mar 18, 2020
@bcmills
Copy link
Contributor

bcmills commented Mar 18, 2020

What does sudo ls -al /Users/dmitshur/gotip/pkg/obj/gopath/pkg/mod/golang.org/x/tools@v0.0.0-20190325161752-5a8dccf5b48a/go.mod report?

@bcmills bcmills added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Mar 18, 2020
@dmitshur
Copy link
Contributor Author 

$ sudo ls -al /Users/dmitshur/gotip/pkg/obj/gopath/pkg/mod/golang.org/x/tools@v0.0.0-20190325161752-5a8dccf5b48a/go.mod
-r--r--r--  1 dmitshur  primarygroup  87 Mar 25  2019 /Users/dmitshur/gotip/pkg/obj/gopath/pkg/mod/golang.org/x/tools@v0.0.0-20190325161752-5a8dccf5b48a/go.mod

@dmitshur
Copy link
Contributor Author

I'll try making a new temporary directory, clone https://go.googlesource.com/go, and see if it reproduces there.

@bcmills
Copy link
Contributor

bcmills commented Mar 18, 2020

Hang on, how did you end up with a module cache in GOROOT/pkg/obj/gopath?

pkg is .gitignored, which is why it isn't being caught on the builders: they're not doing whatever step you did that generated that directory.

go/.gitignore

Line 31 in 6197104

/pkg/

@dmitshur
Copy link
Contributor Author

dmitshur commented Mar 18, 2020 •
edited

It doesn't reproduce with a fresh checkout:

ok  	cmd/internal/moddeps	8.331s

$HOME/gotip is my normal checkout of the main Go repo for development. I constantly switch branches and do various things in it, and rely on clean git status and running all.bash to bring me to a clean state. I can't easily find out what it was that caused that file to exist there.

I think it's reasonable to close this for now, but if it continues to re-occur, we should change .gitignore or tests to avoid this problem from continuing to happen.

@dmitshur dmitshur self-assigned this Mar 18, 2020
@gopherbot
Copy link

Change https://golang.org/cl/223924 mentions this issue: cmd/internal/moddeps: skip GOROOT/pkg when locating modules

@bcmills
Copy link
Contributor

bcmills commented Mar 18, 2020

Given that we know the mechanism, let's just change the test. Skipping pkg should make it marginally more efficient anyway.

@dmitshur dmitshur removed their assignment Mar 18, 2020
@bcmills bcmills assigned dmitshur and bcmills and unassigned dmitshur Mar 18, 2020
@bcmills bcmills added NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure. labels Mar 18, 2020
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 18, 2020
@dmitshur
Copy link
Contributor Author

dmitshur commented Mar 18, 2020 •
edited

Manually removing the pkg subdirectory with sudo rm -rf ./pkg resolved the problem for me.

CL 223924 looks like a good change to make the test more robust. Thanks!

@golang golang locked and limited conversation to collaborators Mar 18, 2021
@rsc rsc unassigned bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure. Vendoring
Projects
None yet
Milestone
Go1.15
Development

No branches or pull requests
3 participants
@dmitshur @bcmills @gopherbot