cmd/go: go test in a directory without any test files uses network access to try to fetch the source for the code running locally #32917

davecheney · 2019-07-03T13:28:09Z

What version of Go are you using (`go version`)?

% go version
go version devel +c485e8b559 Sun Jun 30 05:48:31 2019 +0000 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (`go env`)?

go env Output

% go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/dfc/Library/Caches/go-build"
GOENV="/Users/dfc/Library/Application Support/go/env"
GOEXE=""
GOFLAGS="-ldflags=-w"
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/dfc"
GOPRIVATE=""
GOPROXY="https://gocenter.io"
GOROOT="/Users/dfc/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/dfc/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/dfc/devel/mod/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/by/3gf34_z95zg05cyj744_vhx40000gn/T/go-bui
ld247327765=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

(~/devel) % mkdir mod
(~/devel) % cd mod
/Users/dfc/devel/mod
(~/devel/mod) % go mod init github.com/foo/bar
go: creating new go.mod: module github.com/foo/bar
(~/devel/mod) % go test -x

What did you expect to see?

A message about no test files for the package.

What did you see instead?

The go tool is accessing a network service to try to fetch the source code for the package I am currently in.

# get https://gocenter.io/github.com/@v/list
# get https://gocenter.io/github.com/foo/@v/list
# get https://gocenter.io/github.com/@v/list: 404 Not Found (1.182s)
# get https://gocenter.io/github.com/foo/@v/list: 404 Not Found (1.184s)
can't load package: package github.com/foo/bar: unknown import path "github.com/foo/bar": cannot find module providing package github.com/foo/bar

The text was updated successfully, but these errors were encountered:

bcmills · 2019-07-03T15:32:19Z

The underlying cause is likely the same as #28459, but I'm going to leave this one open since the failure mode is somewhat different.

bcmills · 2019-07-08T19:11:23Z

This certainly needs to be fixed, but a proper fix is fairly invasive (and probably best left for 1.14).

I think it is important that we at least eliminate the network access for 1.13, but the error message will probably be abysmal.

(CC @jayconrod in case he disagrees or can spot a simpler robust fix.)

jayconrod · 2019-07-08T20:00:20Z

Can we actually avoid going out to the network without breaking existing behavior?

Suppose the main module is example.com/a, and there's another module, example.com/a/b in a different repository (not in the directory tree). There is a package example.com/a/b/c in example.com/a, not in example.com/a/b. That means b must be a directory without .go files in the example.com/a. If we change to that directory and run go build . or go test ., the Go command actually should fetch example.com/a/b if it's not present in the cache. We may need to add a new requirement on module example.com/a/b as well.

Sorry this is a contrived example, but as long as we allow packages to be provided by any module whose path is a prefix, I think we'll have this problem.

bcmills · 2019-07-08T20:07:45Z

If we change to that directory and run go build . or go test ., the Go command actually should fetch example.com/a/b if it's not present in the cache.

I disagree. In general, we only allow paths starting with . in module mode to refer to packages within the same module. (Consider, for example, the behavior of go test ./....) Since example.com/a/b is not in module example.com/a, running go build . in within the ./b subdirectory of example.com/a should be an error.

jayconrod · 2019-07-08T21:48:56Z

We could fix this by requiring relative paths to be packages in the main module. It looks like if a relative path contains a wildcard, we expand it to paths in the main module containing go files, and we won't go out to the network. If a path does not contain a wildcard, we join it with the module path without checking if anything is there, treating it as a full package path thereafter.

So currently, go build . is equivalent to go build example.com/a/b in my example. But we could fix go build . while keeping the current behavior of go build example.com/a/b.

bcmills · 2019-07-08T22:00:49Z

Yes. The tricky part is the vendor directory: we should probably resolve it to the corresponding module path if -mod=vendor is set and the directory contains .go source files, and refuse to resolve it otherwise.

bcmills · 2019-07-08T22:34:46Z

I think I've spotted a simple fix for 1.13. Will send a CL tomorrow (assuming it works out).

gopherbot · 2019-07-09T21:18:19Z

Change https://golang.org/cl/185344 mentions this issue: cmd/go/internal/search: record errors in the Match struct

gopherbot · 2019-07-09T21:18:20Z

Change https://golang.org/cl/185343 mentions this issue: cmd/go/internal/modload: make packageNotInModuleError reasonable for the Target module

gopherbot · 2019-07-09T21:18:33Z

Change https://golang.org/cl/185345 mentions this issue: cmd/go: rationalize errors in internal/load and internal/modload

gopherbot · 2019-07-09T22:26:33Z

Change https://golang.org/cl/185417 mentions this issue: cmd/go: check for source files in relative paths before attempting to determine the package path

…the Target module Updates #28459 Updates #32917 Change-Id: Iced562cb7c2e0ac075d8345f1e4ad3b073842dcf Reviewed-on: https://go-review.googlesource.com/c/go/+/185343 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>

gopherbot · 2020-02-27T22:10:05Z

Change https://golang.org/cl/221458 mentions this issue: cmd/go/internal/search: cache IsLocalImport and IsAbs predicates in a Match field

Previously, we would either invoke base.Fatalf (which is too aggressive), or log.Print (which is too passive). Updates #32917 Change-Id: I5475e873e76948de7df65dca08bc0ce67a7fc827 Reviewed-on: https://go-review.googlesource.com/c/go/+/185344 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>

…atch methods This change consolidates predicates currently scattered throughout various parts of the package and module loader into methods on the search.Match type. That not only makes them more concise, but also encourages consistency, both in the code and in reasoning about the kinds of patterns that need to be handled. (For example, the IsLocal predicate was previously two different calls, either of which could be easily forgotten at a given call site.) Factored out from CL 185344 and CL 185345. Updates #32917 Change-Id: Ifa450ffaf6101f673e0ed69ced001a487d6f9335 Reviewed-on: https://go-review.googlesource.com/c/go/+/221458 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>

This change is a non-minimal fix for #32917, but incidentally fixes several other bugs and makes the error messages much more ergonomic. Updates #32917 Updates #27122 Updates #28459 Updates #29280 Updates #30590 Updates #37214 Updates #36173 Updates #36587 Fixes #36008 Fixes #30992 Change-Id: Iedb26d2e0963697c130df5d0f72e7f83ec2dcf06 Reviewed-on: https://go-review.googlesource.com/c/go/+/185345 Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>

bcmills added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jul 3, 2019

bcmills added this to the Go1.13 milestone Jul 3, 2019

bcmills mentioned this issue Jul 3, 2019

cmd/go: reject vendor directories as module-mode build arguments unless -mod=vendor is set #32921

Closed

andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019

bcmills modified the milestones: Go1.14, Go1.13 Jul 8, 2019

bcmills added the release-blocker label Jul 8, 2019

bcmills self-assigned this Jul 8, 2019

gopherbot closed this as completed in b9edee3 Jul 15, 2019

golang locked and limited conversation to collaborators Feb 26, 2021

gopherbot added the FrozenDueToAge label Feb 26, 2021

rsc unassigned bcmills Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: go test in a directory without any test files uses network access to try to fetch the source for the code running locally #32917

cmd/go: go test in a directory without any test files uses network access to try to fetch the source for the code running locally #32917

davecheney commented Jul 3, 2019

bcmills commented Jul 3, 2019

bcmills commented Jul 8, 2019

jayconrod commented Jul 8, 2019

bcmills commented Jul 8, 2019

jayconrod commented Jul 8, 2019

bcmills commented Jul 8, 2019

bcmills commented Jul 8, 2019

gopherbot commented Jul 9, 2019

gopherbot commented Jul 9, 2019

gopherbot commented Jul 9, 2019

gopherbot commented Jul 9, 2019

gopherbot commented Feb 27, 2020

cmd/go: go test in a directory without any test files uses network access to try to fetch the source for the code running locally #32917

cmd/go: go test in a directory without any test files uses network access to try to fetch the source for the code running locally #32917

Comments

davecheney commented Jul 3, 2019

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

bcmills commented Jul 3, 2019

bcmills commented Jul 8, 2019

jayconrod commented Jul 8, 2019

bcmills commented Jul 8, 2019

jayconrod commented Jul 8, 2019

bcmills commented Jul 8, 2019

bcmills commented Jul 8, 2019

gopherbot commented Jul 9, 2019

gopherbot commented Jul 9, 2019

gopherbot commented Jul 9, 2019

gopherbot commented Jul 9, 2019

gopherbot commented Feb 27, 2020

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?