cmd/go: go mod tidy, vendor hide module load errors #27063

rsc · 2018-08-17T19:47:52Z

The module loader leaves some problems for the standard loader to find, like not being able to resolve imports. But 'go mod tidy' and 'go mod vendor' do not run the standard loader, so those problems go undiagnosed. Maybe we should look for them explicitly.

For example make a file x.go that says import "nonexist". go mod tidy succeeds quietly.

kevinburke · 2018-09-26T00:50:41Z

#27868 is possibly a duplicate of this issue; I'm not positive.

jsternberg · 2018-10-09T17:08:44Z

We have run into this issue where using go mod tidy on a machine that doesn't have bzr, but that has bzr dependencies, will just not mention that it didn't work correctly.

We caught it because our CI system is now running go mod tidy and it has bzr installed.

gopherbot · 2018-12-10T20:46:16Z

Change https://golang.org/cl/153459 mentions this issue: cmd/go: fix 'go test' and 'go fmt' with files outside a module

Use the actual loader result in findModule instead of making assumptions about nesting in the build list. As a side-effect, this produces much clearer error messages for packages that (for one reason or another) failed to load. Adjust the package and module path outside a module to "command-line-arguments". That string already appears in the output of a number of (module-mode and GOPATH-mode) commands for file arguments, and as far as I can tell operation outside a module is currently the only case in which the module path of a package is not actually a prefix of the import path. Fixes #28011 Fixes #27099 Fixes #28943 Updates #27102 Updates #28459 Updates #27063 Change-Id: I61d5556df7b1b7d1efdaffa892f0e3e95b612d87 Reviewed-on: https://go-review.googlesource.com/c/153459 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>

myitcv · 2019-01-18T20:16:59Z

This might be another instance of this, I'm not 100% sure:

The following:

cd $(mktemp -d)
go mod init mod
cat <<EOD > main.go
package x

import (
        _ "github.com/hashicorp/vault/api"
)

func main() {
}
EOD
go mod tidy
go mod graph | grep labix

gives no results.

But if you then do:

go list all
go mod tidy
go mod graph | grep labix

you get:

mod labix.org/v2/mgo@v0.0.0-20140701140051-000000000287

which I think is as expected because:

$ go mod why -m labix.org/v2/mgo
# labix.org/v2/mgo
mod
github.com/hashicorp/vault/api
github.com/hashicorp/vault/api.test
github.com/hashicorp/vault/helper/builtinplugins
github.com/hashicorp/vault/builtin/logical/nomad
github.com/hashicorp/nomad/api
github.com/hashicorp/nomad/nomad/structs
github.com/hashicorp/go-msgpack/codec
github.com/hashicorp/go-msgpack/codec.test
labix.org/v2/mgo/bson

leitzler · 2019-05-06T10:05:08Z

@bcmills this issue is tagged with NeedsInvestigation, are there any outstanding questions that need answers before solving this one?

As context I think this is a quite important issue to solve before go1.13 as it adds a lot of confusion for users.

I've seen several cases the thing that happens when bzr isn't installed (as described above by Paul and Jonathan), as well as when indirect dependencies have dependencies that gets removed/broken.

thepudds · 2019-05-06T15:44:41Z

Hi @leitzler or @jsternberg, I think launchpad.net/gocheck and labix.org/v2/mgo might be examples, but do have a simple set of "from scratch" steps that are an example this with bzr where go mod tidy does not complain? Could be helpful to make sure the ultimate fix here covers that case as well.

leitzler · 2019-05-06T20:21:18Z

Sure! So far I have three different cases that I think is directly caused by this issue.

A Bazaar dependency when the bazaar client bzr isn't installed.

$ docker run -it --rm golang sh
# go version
go version go1.12.4 linux/amd64
# cd $(mktemp -d)
# rm $(which bzr)
# go mod init x
go: creating new go.mod: module x
# cat > main.go
package x

import _ "launchpad.net/gocheck"
# go mod tidy
# cat go.mod
module x

go 1.12
# go build
main.go:3:8: unknown import path "launchpad.net/gocheck": bzr branch --use-existing-dir https://launchpad.net/~niemeyer/gocheck/trunk . in /go/pkg/mod/cache/vcs/f46ce2ae80d31f9b0a29099baa203e3b6d269dace4e5357a2cf74bd109e13339: exec: "bzr": executable file not found in $PATH

An direct (or indirect) import can't be found:

$ cd $(mktemp -d) && go mod init x
go: creating new go.mod: module x
$ cat > main.go
package x

import _ "github.com/modsdemo/brokenimport"
$ go mod tidy
go: finding github.com/modsdemo/brokenimport latest
$ cat go.mod
module x

go 1.12

require github.com/modsdemo/brokenimport v0.0.0-20190506193535-03bc7b9ad6c8
$ go build
/home/leitzler/go/pkg/mod/github.com/modsdemo/brokenimport@v0.0.0-20190506193535-03bc7b9ad6c8/brokenimport.go:3:8: unknown import path "github.com/modsdemo/havebeenrenamed": cannot find module providing package github.com/modsdemo/havebeenrenamed

Using invalid GOPROXY setting

$ go mod init x
go: creating new go.mod: module x
$ cat > main.go
package x

import _ "github.com/modsdemo/notags"
$ GOPROXY=proxy.golang.org go mod tidy           # Missing https://, gives no error msg
$ cat go.mod
module x

go 1.12
$ GOPROXY=proxy.golang.org go build              # Missing https://
main.go:3:8: unknown import path "github.com/modsdemo/notags": cannot find module providing package github.com/modsdemo/notags                                  
$ go mod tidy                                    # Works as expected 
go: finding github.com/modsdemo/notags latest
$ go build                                       # Works as expected
$

thepudds · 2019-05-06T20:26:35Z

@leitzler That's great! Thank you for taking the time to do that.

bcmills · 2019-05-28T15:38:12Z

@leitzler

are there any outstanding questions that need answers before solving this one?

We need to figure out which paths are missing the needed error checks, and how to add them without breaking things like go list -e.

rogpeppe · 2019-07-02T10:24:50Z

FWIW I just ran across what I think was this issue when using go mod vendor and it took me ages to work out what the problem was. There was an "ambiguous import" issue in one module; running go mod vendor appeared to succeed but actually didn't vendor the modules that there was a problem with, so we just saw an error about a missing module when CI ran.

gopherbot · 2019-08-02T20:25:11Z

Change https://golang.org/cl/188763 mentions this issue: cmd/go: report loading errors from 'go mod tidy' and 'go mod vendor'

bcmills · 2019-08-02T20:26:56Z

The fix for this turned out to be super simple, given the nice import-stack reporting that @jayconrod added earlier in the 1.13 cycle. I'm hoping we can get this into the next 1.13 build.

rsc added this to the Go1.12 milestone Aug 17, 2018

rsc added the modules label Aug 17, 2018

This was referenced Aug 17, 2018

cmd/go: vague error when VCS tool not available #26982

Closed

modules: go mod vendor does not fulfill all dependencies #27006

Closed

rsc changed the title ~~cmd/go: go mod tidy hides module load errors~~ cmd/go: go mod tidy, vendor hide module load errors Aug 18, 2018

bcmills mentioned this issue Nov 9, 2018

cmd/go: go mod tidy does not fail when ambiguous import path found #28685

Closed

bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 9, 2018

bcmills modified the milestones: Go1.12, Go1.13 Nov 9, 2018

bcmills self-assigned this Nov 9, 2018

This was referenced Nov 14, 2018

cmd/go: go mod fails to find direct dependencies #28629

Closed

cmd/go: mod vendor command skip failed dependencies #28447

Closed

bcmills mentioned this issue Dec 6, 2018

cmd/go: does not verify existence of replace directory #28032

Open

bcmills mentioned this issue Dec 12, 2018

cmd/go: go mod tidy always prints "finding" log messages #27123

Closed

bcmills modified the milestones: Go1.13, Go1.12 Dec 12, 2018

bcmills mentioned this issue Jan 9, 2019

cmd/go: malformed file path: invalid char ':' #28001

Closed

myitcv mentioned this issue Jan 16, 2019

cmd/go: 'go list -m' fails to follow dependencies through older versions of the main module #29773

Closed

mark-rushakoff mentioned this issue Jan 17, 2019

influxdata/platform dependencies break 3rd party code influxdata/influxdb#11222

Closed

andybons modified the milestones: Go1.12, Go1.13 Feb 12, 2019

bcmills mentioned this issue Mar 7, 2019

cmd/go: 'looping trying to add package' when running 'mod tidy' with a package not found in the containing module #30394

Closed

thepudds mentioned this issue May 31, 2019

cmd/go: go.mod should codify dependencies in multi-module mode before tagging new module #32344

Closed

bcmills mentioned this issue Jun 21, 2019

cmd/go: "finding" lines always printed for dependency with seemingly consistent import path usage #32720

Closed

andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019

zikaeroh mentioned this issue Jul 13, 2019

cmd/go: "go.mod has post-v0 module path ... at revision" with GOPROXY=direct, but not with proxy or Go 1.12 #33099

Closed

bcmills modified the milestones: Go1.14, Go1.13 Aug 2, 2019

bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Aug 2, 2019

gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 2, 2019

gopherbot closed this as completed in 07b3e62 Aug 2, 2019

mattysweeps mentioned this issue Aug 22, 2019

Fix/remove submodule google/kf#592

Merged

bcmills mentioned this issue Aug 28, 2019

cmd/go: tries to download v2.2.2 (no suffix) when checking a replaced v2.2.2+incompatible for a missing import #33795

Closed

taiyangc mentioned this issue Sep 10, 2019

Fix golang 1.13 compilation issues (on master) TRON-US/go-btfs#156

Merged

bcmills mentioned this issue Oct 18, 2019

cmd/go: "leading dot in path element" for previously-undiagnosed paths in go 1.13 #34992

Closed

golang locked and limited conversation to collaborators Aug 1, 2020

gopherbot added the FrozenDueToAge label Aug 1, 2020

rsc unassigned bcmills Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: go mod tidy, vendor hide module load errors #27063

cmd/go: go mod tidy, vendor hide module load errors #27063

rsc commented Aug 17, 2018

kevinburke commented Sep 26, 2018

jsternberg commented Oct 9, 2018

gopherbot commented Dec 10, 2018

myitcv commented Jan 18, 2019

leitzler commented May 6, 2019

thepudds commented May 6, 2019

leitzler commented May 6, 2019

thepudds commented May 6, 2019

bcmills commented May 28, 2019

rogpeppe commented Jul 2, 2019

gopherbot commented Aug 2, 2019

bcmills commented Aug 2, 2019

cmd/go: go mod tidy, vendor hide module load errors #27063

cmd/go: go mod tidy, vendor hide module load errors #27063

Comments

rsc commented Aug 17, 2018

kevinburke commented Sep 26, 2018

jsternberg commented Oct 9, 2018

gopherbot commented Dec 10, 2018

myitcv commented Jan 18, 2019

leitzler commented May 6, 2019

thepudds commented May 6, 2019

leitzler commented May 6, 2019

thepudds commented May 6, 2019

bcmills commented May 28, 2019

rogpeppe commented Jul 2, 2019

gopherbot commented Aug 2, 2019

bcmills commented Aug 2, 2019