Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: 'go build ./...' in module mode outside of a module traverses the filesystem instead of failing fast #37521

Closed
perillo opened this issue Feb 27, 2020 · 10 comments
Closed

cmd/go: 'go build ./...' in module mode outside of a module traverses the filesystem instead of failing fast #37521

perillo opened this issue Feb 27, 2020 · 10 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. ToolSpeed
Milestone
Go1.15

Comments

@perillo
Copy link
Contributor

perillo commented Feb 27, 2020 •
edited

What version of Go are you using (go version)?

$ go version
go version go1.14 linux/amd64

Does this issue reproduce with the latest release?

Yes.
It also reproduces with go1.13 but not with go1.12.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN="/home/manlio/.local/bin"
GOCACHE="/home/manlio/.cache/go-build"
GOENV="/home/manlio/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY="github.com/perillo/go-init"
GONOSUMDB="github.com/perillo/go-init"
GOOS="linux"
GOPATH="/home/manlio/.local/lib/go:/home/manlio/src/go"
GOPRIVATE="github.com/perillo/go-init"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build500333414=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version go1.14 linux/amd64
GOROOT/bin/go tool compile -V: compile version go1.14
uname -sr: Linux 5.5.5-arch1-1
/usr/lib/libc.so.6: GNU C Library (GNU libc) stable release version 2.31.
gdb --version: GNU gdb (GDB) 9.1

What did you do?

By accident I typed go build ./... from inside my home directory, and I notice that it took a lot of time, spinning my hard disk.

$ sync
$ sudo sh -c 'echo 3 >/proc/sys/vm/drop_caches'
$ time go build ./...                                                                                                                                                       
go: cannot find main module, but found .git/config in /home/manlio
	to create a module there, run:
	go mod init
go build ./...  13.39s user 14.64s system 19% cpu 2:20.65 total

Called again, with warm cache:

$ time go build ./...                                                                                                                                                       
go: cannot find main module, but found .git/config in /home/manlio
	to create a module there, run:
	go mod init
go build ./...  5.18s user 3.44s system 112% cpu 7.647 total

This only happens from inside my home directory, where there is a .git/config file.
Also, go build traverse the filesystem the entire filesystem only when the ./... pattern is specified.

$ time go build                                                                                                                                                             
go: cannot find main module, but found .git/config in /home/manlio
	to create a module there, run:
	go mod init
go build  0.02s user 0.04s system 6% cpu 1.031 total

(with no cached data in the kernel).
@bcmills
Copy link
Contributor

bcmills commented Feb 28, 2020 •
edited

Heh, that's odd. I can reproduce it as well, both with GO111MODULE=on and without. Thanks for the report!

@bcmills
Copy link
Contributor

bcmills commented Feb 28, 2020

CC @jayconrod @matloob

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 28, 2020
@bcmills bcmills added this to the Go1.15 milestone Feb 28, 2020
@bcmills
Copy link
Contributor

bcmills commented Feb 28, 2020

It also reproduces with go1.13 but not with go1.12.

I'm also seeing a very slow go build ./... with go1.12 when in GOPATH mode. So I suspect that what's happening here is that somehow the module-mode go build is trying to resolve ./... before it realizes that it isn't even in a module.

@perillo
Copy link
Contributor Author

perillo commented Feb 28, 2020

You are right.
It has nothing to do with a potential config file to convert. The issue title should be changed.

Thanks.

@bcmills bcmills changed the title cmd/go: build ./... tries to traverse the entire filesystem when outside a module but with a potential config file to convert to go.mod cmd/go: 'go build ./...' in module mode outside of a module traverses the filesystem instead of failing fast Feb 28, 2020
@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Feb 28, 2020
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 28, 2020
@perillo
Copy link
Contributor Author

perillo commented Feb 28, 2020

Unfortunately go can not simply ignore the pattern, as an example with go list std.

@perillo
Copy link
Contributor Author

perillo commented Feb 28, 2020 •
edited

Here is an example where a using a ... pattern will cause go list to return a confusing warning, instead of reporting an error since the current working directory is not part of a module:

go list github.com/perillo/...
go: warning: "github.com/perillo/..." matched no packages

@bcmills
Copy link
Contributor

bcmills commented Feb 28, 2020

That go list pattern is arguably correct: you can go list packages in the standard library when outside of a module, and no package in the standard library matches the pattern github.com/perillo/....

@gopherbot
Copy link

Change https://golang.org/cl/221544 mentions this issue: cmd/go: avoid matching wildcards rooted outside of available modules

@bcmills bcmills self-assigned this Feb 28, 2020
@perillo
Copy link
Contributor Author

perillo commented Feb 28, 2020

But the first path element has a dot, so it can not be a standard library package.

@bcmills
Copy link
Contributor

bcmills commented Feb 28, 2020

That's true. At any rate, it is clearly not the same issue as the unnecessary filesystem walk for ./..., so to the extent that it is worth addressing at all, it should be a separate issue.

@golang golang locked and limited conversation to collaborators Feb 27, 2021
@rsc rsc unassigned bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. ToolSpeed
Projects
None yet
Milestone
Go1.15
Development

No branches or pull requests
3 participants
@bcmills @perillo @gopherbot