Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: avoid invoking git for standard library queries #30885

Closed
eliasnaur opened this issue Mar 16, 2019 · 6 comments
Closed

cmd/go: avoid invoking git for standard library queries #30885

eliasnaur opened this issue Mar 16, 2019 · 6 comments
Labels
FrozenDueToAge GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Unplanned

Comments

@eliasnaur
Copy link
Contributor

What version of Go are you using (go version)?

$ go version
go version devel +14c3692502 Sat Mar 16 14:16:39 2019 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What did you do?

Removed git, then

$ cd $HOME/go/golang.org/x/tools # previously checked out x/tools
$ go clean -modcache
$ go list -- unsafe
go: golang.org/x/net@v0.0.0-20190311183353-d8887717615a: git init --bare in /home/elias/go/pkg/mod/cache/vcs/4a22365141bc4eea5d5ac4a1395e653f2669485db75ef119e7bbec8e19b12a21: exec: "git": executable file not found in $PATH

What did you expect to see?

No error

What did you see instead?

An error about missing git.

Queries such as the above are issued by go/packages in module mode, which is breaking many tests on GOOS=android where git is not installed. See x/tools: https://build.golang.org/log/763e5ef04fc3679a903b8cd85a7e0ececcdd15b69.

It seems to me it should be possible to avoid downloading dependencies and invoking git when querying the standard library.
@eliasnaur
Copy link
Contributor Author

CC @bcmills @jayconrod

@eliasnaur
Copy link
Contributor Author

Another bunch of errors from x/exp:

https://build.golang.org/log/546c34306ce262ec806cd8ccda7462f0ce503ad6

(Look for "could not determine GOARCH and Go compiler")

@jayconrod jayconrod added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go modules labels Mar 18, 2019
@jayconrod jayconrod self-assigned this Mar 18, 2019
@jayconrod jayconrod added this to the Go1.13 milestone Mar 18, 2019
@ericlagergren
Copy link
Contributor

Related to #29452

@jayconrod
Copy link
Contributor

This is happening when we load the build list. The first thing the Go command does in module mode (whether running go build, go test, go list, ...) is load the go.mod of the main module and any other modules that are transitively required. If go.mod files are not present in the module cache, they are fetched via GOPROXY or VCS tools.

It might be possible for us to limit this behavior, based on the command line arguments, but it would add a lot of complexity, and I think it would only be feasible for the main module and the standard library. It doesn't seem worth it.

GOPROXY should probably be the long-term solution. Can that be used here? Setting GOPROXY to a file:// URL pointing to a tree prepared ahead of time seems like a good setup for tests.

@jayconrod jayconrod modified the milestones: Go1.13, Unplanned Mar 19, 2019
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/168118 mentions this issue: misc/android: pass on GOPROXY to device environment

@eliasnaur
Copy link
Contributor Author

GOPROXY is a good idea. The builders already have GOPROXY setup for them; CL 168118 passes that to the device environment. Thank you, I believe that solves the issue with running go list on android.

I still believe go list queries for std and the main module are common enough to do something here, but I'll leave that discussion for #29452.

gopherbot pushed a commit that referenced this issue Mar 21, 2019
@eliasnaur
misc/android: pass on GOPROXY to device environment
409c97c 
Android devices don't have git available, so many go tool commands
that fetch dependencies fail. Builders already have a GOPROXY
available, so pass that along to the device environment.

Updates #30885

Change-Id: Id0d2338932f0cd7de4d95d9e0ca9b79d29336ffe
Reviewed-on: https://go-review.googlesource.com/c/go/+/168118
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@golang golang locked and limited conversation to collaborators Mar 18, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
4 participants
@jayconrod @eliasnaur @ericlagergren @gopherbot