cmd/go: unclear error when 'go mod vendor' is run with no packages to vendor #36580

jayconrod · 2020-01-15T19:16:32Z

What version of Go are you using (`go version`)?

$ go version
go version devel +3743d21270 Wed Jan 15 17:53:43 2020 +0000 darwin/amd64

Does this issue reproduce with the latest release?

no, only tip

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jayconrod/Library/Caches/go-build"
GOENV="/Users/jayconrod/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/jayconrod/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/go/installed"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/opt/go/installed/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/jayconrod/Code/test3/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/rq/x0692kqj6ml8cvrhcqh5bswc008xj1/T/go-build143764245=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

In an empty directory:

go mod init example.com/m
go get -d golang.org/x/tools
go mod vendor

What did you expect to see?

This is the error message from 1.13.6. This message is also printed at tip if there are no module requirements.

go: no dependencies to vendor

What did you see instead?

go mod vendor: open /Users/jayconrod/Code/test3/vendor/modules.txt: no such file or directory

This message is printed when go mod vendor tries to write modules.txt after copying packages into the vendor directory. There are no packages to vendor, so it hasn't actually created the vendor directory, and the write fails.

The text was updated successfully, but these errors were encountered:

jayconrod · 2020-01-15T19:17:47Z

Milestoning for Go1.14 because this is a regression, and it would be nice to fix it.

Not a release blocker because it's an error message on an edge case with little impact.

cc @bcmills @matloob

jcrowgey · 2020-01-25T06:50:17Z

I've been learning go for the last year or so. I'd love to learn more about the toolchain compiler and thought I might look around here for some beginner issues as an opportunity to learn. I've repro'd this bug locally so maybe I can offer a fix.

Looks like lines 122 -- 128 of src/cmd/go/internal/modcmd/vendor.go

Shall I take a stab at this one?

jcrowgey · 2020-01-26T04:35:19Z

Actually, I'm not 100% sure I understand the desired behavior. Do we want to create the vendor directory so that modules.txt can get the ## explicit line? Or do we simply not want to create the vendor dir yet (nor modules.txt) since there are no package to vendor?

If the latter, it seems like we can just check len(modpkgs) instead of the write buffer for modules.txt. If the former, since we can no longer rely on vendor dir being created by a call to vendorPkg, I just need to call MkdirAll on vdir before we try to write "modules.txt" into it.
@jayconrod

Now that ## explicit lines may appear in modules.txt, the presence of bytes in the modules.txt buffer doesn't indicate that we have created a vendor/ directory, nor that we need a modules.txt. Updates: golang#36580

bcmills · 2020-01-27T16:29:30Z

Probably we should create the vendor directory and vendor/modules.txt file: that way go build will default to -mod=vendor for future commands and not try to access the network to load the (unnecessary) dependency.

jayconrod · 2020-01-27T18:26:25Z

+1 to creating vendor/modules.txt. This is an important side effect of go mod vendor.

@jcrowgey You are very welcome to work on this. Make sure to test the new behavior. A small change in src/cmd/go/testdata/script/mod_vendor_nodeps.txt would probably be best.

I'm going to remove the Go1.14 milestone from this issue. The 1.14 branch will be cut very soon (maybe today?), so this should probably go into 1.15. (@bcmills let me know if you disagree though).

jcrowgey · 2020-01-28T04:35:18Z

Thanks to both of you!

Probably we should create the vendor directory and vendor/modules.txt file: that way go build will default to -mod=vendor for future commands and not try to access the network to load the (unnecessary) dependency.

Following that logic, that running go mod vendor should create the side effect where future builds (and presumably go-gets) would default to vendor mode, would we want to create vendor/ and just touch an empty modules.txt file when there's absolutely no dependencies in the pkg cache (ie, even when no go get command has been run)?

Even an an empty build list merits a vendor/modules.txt. Updates: golang#36580

bcmills · 2020-01-28T14:54:16Z

Maybe? It's not clear to me what should happen if the module has no dependencies whatsoever, since we won't hit the network even with -mod=mod. I'm inclined to leave that case as-is (i.e. don't create the vendor directory or vendor/modules.txt).

`go mod vendor` should create vendor/modules.txt even when the only deps in go.mod are unused. Updates: golang#36580

jcrowgey · 2020-01-29T20:58:07Z

Ok, perhaps this is on the right track:

It -relies on os.MkdirAll's idempotency (and, hopefully, efficiently doing nothing on a second call) to- ensures the creation of vdir whenever we process an explicit line from the go.mod file which isn't actually used as a dependency.

I added a regression test based on the use case described in the OP here. It's not clear to me if that regression test invokes unnecessary network bandwith if that particular tools version isn't in the local pkg/cache.

EDIT:

This second commit updates to use sync.Once instead of potentially calling os.MkdirAll many times. Anyway, if this is on the right track, I can go ahead and do the CLA thing and open a PR with a branch. Cheers.

`go mod vendor` should create vendor/modules.txt even when the only deps in go.mod are unused. Updates: golang#36580

`go mod vendor` should create vendor/modules.txt even when the only deps in go.mod are unused. Fixes: golang#36580

gopherbot · 2020-01-31T05:41:51Z

Change https://golang.org/cl/217135 mentions this issue: cmd/vendor: create modules.txt even for unused deps

`go mod vendor` now attempts to create vendor/modules.txt even when the only dependencies in go.mod are unused. This change ensures that vendor/ is created whenever there is any text to be written into modules txt. The previous approach relied on a side effect of a call to vendorPkg to create the directory. Since the change described in [1], this is no longer fully reliable, resolving [2]. [1]: https://golang.org/issue/33848#issuecomment-537222782 [2]: Fixes: golang#36580

'go mod vendor' now attempts to create vendor/modules.txt even when the only dependencies in go.mod are unused. This change ensures that vendor/ is created whenever there is any text to be written into modules txt. The previous approach relied on a side effect of a call to vendorPkg to create the directory. Since the change described in [1], this is no longer fully reliable, resolving [2]. [1]: https://golang.org/issue/33848#issuecomment-537222782 [2]: Fixes golang#36580

jayconrod added the NeedsFix label Jan 15, 2020

jayconrod added this to the Go1.14 milestone Jan 15, 2020

bcmills added the modules label Jan 22, 2020

jayconrod modified the milestones: Go1.14, Go1.15 Jan 27, 2020

jcrowgey added a commit to jcrowgey/go that referenced this issue Jan 28, 2020

cmd/vendor: always create a vendor/modules.txt

4e9acca

Even an an empty build list merits a vendor/modules.txt. Updates: golang#36580

jcrowgey pushed a commit to jcrowgey/go that referenced this issue Jan 29, 2020

cmd/vendor: Create modules.txt even for unused deps

07e9376

`go mod vendor` should create vendor/modules.txt even when the only deps in go.mod are unused. Updates: golang#36580

jcrowgey pushed a commit to jcrowgey/go that referenced this issue Jan 31, 2020

cmd/vendor: Create modules.txt even for unused deps

86837d1

`go mod vendor` should create vendor/modules.txt even when the only deps in go.mod are unused. Updates: golang#36580

jcrowgey added a commit to jcrowgey/go that referenced this issue Jan 31, 2020

cmd/vendor: Create modules.txt even for unused deps

c0325bc

`go mod vendor` should create vendor/modules.txt even when the only deps in go.mod are unused. Fixes: golang#36580

jcrowgey mentioned this issue Jan 31, 2020

cmd/vendor: create modules.txt even for unused deps #36920

Closed

bcmills mentioned this issue Feb 26, 2020

cmd/go: go mod vendor in empty new project produces confusing error message in 1.14 #37464

Closed

gopherbot closed this as completed in 26aadfa Feb 26, 2020

golang locked and limited conversation to collaborators Feb 25, 2021

gopherbot added the FrozenDueToAge label Feb 25, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: unclear error when 'go mod vendor' is run with no packages to vendor #36580

cmd/go: unclear error when 'go mod vendor' is run with no packages to vendor #36580

jayconrod commented Jan 15, 2020

jayconrod commented Jan 15, 2020

jcrowgey commented Jan 25, 2020 •

edited

Loading

jcrowgey commented Jan 26, 2020 •

edited

Loading

bcmills commented Jan 27, 2020

jayconrod commented Jan 27, 2020

jcrowgey commented Jan 28, 2020 •

edited

Loading

bcmills commented Jan 28, 2020

jcrowgey commented Jan 29, 2020 •

edited

Loading

gopherbot commented Jan 31, 2020

cmd/go: unclear error when 'go mod vendor' is run with no packages to vendor #36580

cmd/go: unclear error when 'go mod vendor' is run with no packages to vendor #36580

Comments

jayconrod commented Jan 15, 2020

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

jayconrod commented Jan 15, 2020

jcrowgey commented Jan 25, 2020 • edited Loading

jcrowgey commented Jan 26, 2020 • edited Loading

bcmills commented Jan 27, 2020

jayconrod commented Jan 27, 2020

jcrowgey commented Jan 28, 2020 • edited Loading

bcmills commented Jan 28, 2020

jcrowgey commented Jan 29, 2020 • edited Loading

gopherbot commented Jan 31, 2020

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

jcrowgey commented Jan 25, 2020 •

edited

Loading

jcrowgey commented Jan 26, 2020 •

edited

Loading

jcrowgey commented Jan 28, 2020 •

edited

Loading

jcrowgey commented Jan 29, 2020 •

edited

Loading