Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: unnecessary go-get queries during go get golang.org/x/vgo #25565

Closed
myitcv opened this issue May 25, 2018 · 6 comments
Closed

cmd/go: unnecessary go-get queries during go get golang.org/x/vgo #25565

myitcv opened this issue May 25, 2018 · 6 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. ToolSpeed
Milestone

Comments

@myitcv
Copy link
Member

myitcv commented May 25, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.10.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ubuntu/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/tmp/tmp.qqjdyzCca9"
GORACE=""
GOROOT="/home/ubuntu/usr/go1.10.2"
GOTMPDIR=""
GOTOOLDIR="/home/ubuntu/usr/go1.10.2/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build674860894=/tmp/go-build -gno-record-gcc-switches"

What did you do?

From a relatively large AWS EC2 instance in Dublin, Ireland:

$ time go get -u golang.org/x/vgo

real    1m6.853s
user    0m1.148s
sys     0m0.200s

What did you expect to see?

The go get not to take so long.

What did you see instead?

It took over 1 minute.

For comparison, the Github clone takes ~1.2 seconds:

$ time git clone https://github.com/golang/vgo src/golang.org/x/vgo
Cloning into 'src/golang.org/x/vgo'...
remote: Counting objects: 1246, done.
remote: Compressing objects: 100% (75/75), done.
remote: Total 1246 (delta 52), reused 95 (delta 39), pack-reused 1132
Receiving objects: 100% (1246/1246), 692.85 KiB | 0 bytes/s, done.
Resolving deltas: 100% (447/447), done.
Checking connectivity... done.

real    0m1.217s
user    0m0.108s
sys     0m0.052s

But then the problem is:

$ go get -u golang.org/x/vgo
package golang.org/x/vgo: golang.org/x/vgo is a custom import path for https://go.googlesource.com/vgo, but /tmp/tmp.iqRH4vIElL/src/golang.org/x/vgo is checked out from https://github.com/golang/vgo
@gopherbot gopherbot added this to the vgo milestone May 25, 2018
@myitcv
Copy link
Member Author

myitcv commented May 25, 2018

The output from go get -u -v ... is more instructive:

Fetching https://golang.org/x/vgo?go-get=1
Parsing meta tags from https://golang.org/x/vgo?go-get=1 (status code 200)
get "golang.org/x/vgo": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo?go-get=1
golang.org/x/vgo (download)
Fetching https://golang.org/x/vgo/vendor/cmd/go?go-get=1
Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go?go-get=1 (status code 200)
get "golang.org/x/vgo/vendor/cmd/go": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go?go-get=1
get "golang.org/x/vgo/vendor/cmd/go": verifying non-authoritative meta tag
...
<snip>

So now I can start to understand with these sequential requests why the operation takes so long.

@rsc is this something you want to try and solve for now? I'm thinking perhaps it's not worth the effort...

@rsc
Copy link
Contributor

rsc commented May 30, 2018

What about git clone https://go.googlesource.com/vgo, since that's the source-of-truth repo being used?

@myitcv
Copy link
Member Author

myitcv commented May 30, 2018

What about git clone https://go.googlesource.com/vgo, since that's the source-of-truth repo being used?

That's super quick:

$ time git clone -q https://go.googlesource.com/vgo

real    0m0.563s
user    0m0.116s
sys     0m0.048s

It's the verifying of meta tags via https://golang.org that takes the time:

 0.01s: Fetching https://golang.org/x/vgo?go-get=1
 0.52s: Parsing meta tags from https://golang.org/x/vgo?go-get=1 (status code 200)
 0.52s: get "golang.org/x/vgo": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo?go-get=1
 0.52s: golang.org/x/vgo (download)
 1.11s: Fetching https://golang.org/x/vgo/vendor/cmd/go?go-get=1
 5.21s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go?go-get=1 (status code 200)
 5.21s: get "golang.org/x/vgo/vendor/cmd/go": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go?go-get=1
 5.21s: get "golang.org/x/vgo/vendor/cmd/go": verifying non-authoritative meta tag
 5.21s: Fetching https://golang.org/x/vgo?go-get=1
 5.33s: Parsing meta tags from https://golang.org/x/vgo?go-get=1 (status code 200)
 5.33s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/base?go-get=1
 5.52s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/base?go-get=1 (status code 200)
 5.52s: get "golang.org/x/vgo/vendor/cmd/go/internal/base": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/base?go-get=1
 5.52s: get "golang.org/x/vgo/vendor/cmd/go/internal/base": verifying non-authoritative meta tag
 5.53s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/cfg?go-get=1
 5.65s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/cfg?go-get=1 (status code 200)
 5.65s: get "golang.org/x/vgo/vendor/cmd/go/internal/cfg": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/cfg?go-get=1
 5.65s: get "golang.org/x/vgo/vendor/cmd/go/internal/cfg": verifying non-authoritative meta tag
 5.65s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/str?go-get=1
 9.08s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/str?go-get=1 (status code 200)
 9.08s: get "golang.org/x/vgo/vendor/cmd/go/internal/str": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/str?go-get=1
 9.08s: get "golang.org/x/vgo/vendor/cmd/go/internal/str": verifying non-authoritative meta tag
 9.08s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/bug?go-get=1
13.74s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/bug?go-get=1 (status code 200)
13.74s: get "golang.org/x/vgo/vendor/cmd/go/internal/bug": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/bug?go-get=1
13.74s: get "golang.org/x/vgo/vendor/cmd/go/internal/bug": verifying non-authoritative meta tag
13.74s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/envcmd?go-get=1
13.88s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/envcmd?go-get=1 (status code 200)
13.88s: get "golang.org/x/vgo/vendor/cmd/go/internal/envcmd": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/envcmd?go-get=1
13.88s: get "golang.org/x/vgo/vendor/cmd/go/internal/envcmd": verifying non-authoritative meta tag
13.88s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/cache?go-get=1
17.64s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/cache?go-get=1 (status code 200)
17.64s: get "golang.org/x/vgo/vendor/cmd/go/internal/cache": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/cache?go-get=1
17.64s: get "golang.org/x/vgo/vendor/cmd/go/internal/cache": verifying non-authoritative meta tag
17.64s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/load?go-get=1
17.79s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/load?go-get=1 (status code 200)
17.79s: get "golang.org/x/vgo/vendor/cmd/go/internal/load": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/load?go-get=1
17.79s: get "golang.org/x/vgo/vendor/cmd/go/internal/load": verifying non-authoritative meta tag
17.79s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/modinfo?go-get=1
17.91s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/modinfo?go-get=1 (status code 200)
17.91s: get "golang.org/x/vgo/vendor/cmd/go/internal/modinfo": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/modinfo?go-get=1
17.91s: get "golang.org/x/vgo/vendor/cmd/go/internal/modinfo": verifying non-authoritative meta tag
17.91s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/search?go-get=1
23.28s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/search?go-get=1 (status code 200)
23.28s: get "golang.org/x/vgo/vendor/cmd/go/internal/search": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/search?go-get=1
23.28s: get "golang.org/x/vgo/vendor/cmd/go/internal/search": verifying non-authoritative meta tag
23.28s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/vgo?go-get=1
23.41s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/vgo?go-get=1 (status code 200)
23.41s: get "golang.org/x/vgo/vendor/cmd/go/internal/vgo": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/vgo?go-get=1
23.41s: get "golang.org/x/vgo/vendor/cmd/go/internal/vgo": verifying non-authoritative meta tag
23.41s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/dirhash?go-get=1
25.00s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/dirhash?go-get=1 (status code 200)
25.00s: get "golang.org/x/vgo/vendor/cmd/go/internal/dirhash": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/dirhash?go-get=1
25.00s: get "golang.org/x/vgo/vendor/cmd/go/internal/dirhash": verifying non-authoritative meta tag
25.00s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/imports?go-get=1
25.13s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/imports?go-get=1 (status code 200)
25.13s: get "golang.org/x/vgo/vendor/cmd/go/internal/imports": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/imports?go-get=1
25.13s: get "golang.org/x/vgo/vendor/cmd/go/internal/imports": verifying non-authoritative meta tag
25.13s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/modconv?go-get=1
25.25s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/modconv?go-get=1 (status code 200)
25.25s: get "golang.org/x/vgo/vendor/cmd/go/internal/modconv": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/modconv?go-get=1
25.25s: get "golang.org/x/vgo/vendor/cmd/go/internal/modconv": verifying non-authoritative meta tag
25.25s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/module?go-get=1
25.37s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/module?go-get=1 (status code 200)
25.37s: get "golang.org/x/vgo/vendor/cmd/go/internal/module": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/module?go-get=1
25.37s: get "golang.org/x/vgo/vendor/cmd/go/internal/module": verifying non-authoritative meta tag
25.37s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/semver?go-get=1
25.49s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/semver?go-get=1 (status code 200)
25.49s: get "golang.org/x/vgo/vendor/cmd/go/internal/semver": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/semver?go-get=1
25.49s: get "golang.org/x/vgo/vendor/cmd/go/internal/semver": verifying non-authoritative meta tag
25.49s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch?go-get=1
27.35s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch?go-get=1 (status code 200)
27.35s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfetch": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch?go-get=1
27.35s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfetch": verifying non-authoritative meta tag
27.35s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/bitbucket?go-get=1
27.54s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/bitbucket?go-get=1 (status code 200)
27.54s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfetch/bitbucket": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/bitbucket?go-get=1
27.54s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfetch/bitbucket": verifying non-authoritative meta tag
27.54s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/codehost?go-get=1
28.40s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/codehost?go-get=1 (status code 200)
28.40s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfetch/codehost": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/codehost?go-get=1
28.40s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfetch/codehost": verifying non-authoritative meta tag
28.41s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/gitrepo?go-get=1
28.65s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/gitrepo?go-get=1 (status code 200)
28.65s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfetch/gitrepo": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/gitrepo?go-get=1
28.65s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfetch/gitrepo": verifying non-authoritative meta tag
28.65s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/github?go-get=1
31.21s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/github?go-get=1 (status code 200)
31.21s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfetch/github": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/github?go-get=1
31.21s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfetch/github": verifying non-authoritative meta tag
31.21s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/googlesource?go-get=1
32.50s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/googlesource?go-get=1 (status code 200)
32.50s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfetch/googlesource": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/modfetch/googlesource?go-get=1
32.50s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfetch/googlesource": verifying non-authoritative meta tag
32.50s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/modfile?go-get=1
33.93s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/modfile?go-get=1 (status code 200)
33.93s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfile": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/modfile?go-get=1
33.93s: get "golang.org/x/vgo/vendor/cmd/go/internal/modfile": verifying non-authoritative meta tag
33.93s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/web2?go-get=1
34.74s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/web2?go-get=1 (status code 200)
34.74s: get "golang.org/x/vgo/vendor/cmd/go/internal/web2": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/web2?go-get=1
34.74s: get "golang.org/x/vgo/vendor/cmd/go/internal/web2": verifying non-authoritative meta tag
34.74s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/mvs?go-get=1
35.55s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/mvs?go-get=1 (status code 200)
35.55s: get "golang.org/x/vgo/vendor/cmd/go/internal/mvs": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/mvs?go-get=1
35.55s: get "golang.org/x/vgo/vendor/cmd/go/internal/mvs": verifying non-authoritative meta tag
35.55s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/work?go-get=1
38.77s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/work?go-get=1 (status code 200)
38.77s: get "golang.org/x/vgo/vendor/cmd/go/internal/work": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/work?go-get=1
38.77s: get "golang.org/x/vgo/vendor/cmd/go/internal/work": verifying non-authoritative meta tag
38.78s: Fetching https://golang.org/x/vgo/vendor/cmd/internal/buildid?go-get=1
38.89s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/internal/buildid?go-get=1 (status code 200)
38.89s: get "golang.org/x/vgo/vendor/cmd/internal/buildid": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/internal/buildid?go-get=1
38.89s: get "golang.org/x/vgo/vendor/cmd/internal/buildid": verifying non-authoritative meta tag
38.90s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/web?go-get=1
39.04s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/web?go-get=1 (status code 200)
39.04s: get "golang.org/x/vgo/vendor/cmd/go/internal/web": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/web?go-get=1
39.04s: get "golang.org/x/vgo/vendor/cmd/go/internal/web": verifying non-authoritative meta tag
39.04s: Fetching https://golang.org/x/vgo/vendor/cmd/internal/browser?go-get=1
42.79s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/internal/browser?go-get=1 (status code 200)
42.79s: get "golang.org/x/vgo/vendor/cmd/internal/browser": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/internal/browser?go-get=1
42.79s: get "golang.org/x/vgo/vendor/cmd/internal/browser": verifying non-authoritative meta tag
42.80s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/clean?go-get=1
44.68s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/clean?go-get=1 (status code 200)
44.68s: get "golang.org/x/vgo/vendor/cmd/go/internal/clean": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/clean?go-get=1
44.68s: get "golang.org/x/vgo/vendor/cmd/go/internal/clean": verifying non-authoritative meta tag
44.68s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/doc?go-get=1
44.80s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/doc?go-get=1 (status code 200)
44.80s: get "golang.org/x/vgo/vendor/cmd/go/internal/doc": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/doc?go-get=1
44.80s: get "golang.org/x/vgo/vendor/cmd/go/internal/doc": verifying non-authoritative meta tag
44.80s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/fix?go-get=1
45.67s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/fix?go-get=1 (status code 200)
45.67s: get "golang.org/x/vgo/vendor/cmd/go/internal/fix": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/fix?go-get=1
45.67s: get "golang.org/x/vgo/vendor/cmd/go/internal/fix": verifying non-authoritative meta tag
45.68s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/fmtcmd?go-get=1
45.80s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/fmtcmd?go-get=1 (status code 200)
45.80s: get "golang.org/x/vgo/vendor/cmd/go/internal/fmtcmd": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/fmtcmd?go-get=1
45.80s: get "golang.org/x/vgo/vendor/cmd/go/internal/fmtcmd": verifying non-authoritative meta tag
45.80s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/generate?go-get=1
46.05s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/generate?go-get=1 (status code 200)
46.05s: get "golang.org/x/vgo/vendor/cmd/go/internal/generate": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/generate?go-get=1
46.05s: get "golang.org/x/vgo/vendor/cmd/go/internal/generate": verifying non-authoritative meta tag
46.05s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/get?go-get=1
46.17s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/get?go-get=1 (status code 200)
46.17s: get "golang.org/x/vgo/vendor/cmd/go/internal/get": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/get?go-get=1
46.17s: get "golang.org/x/vgo/vendor/cmd/go/internal/get": verifying non-authoritative meta tag
46.17s: Fetching https://golang.org/x/vgo/vendor/internal/singleflight?go-get=1
46.29s: Parsing meta tags from https://golang.org/x/vgo/vendor/internal/singleflight?go-get=1 (status code 200)
46.29s: get "golang.org/x/vgo/vendor/internal/singleflight": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/internal/singleflight?go-get=1
46.29s: get "golang.org/x/vgo/vendor/internal/singleflight": verifying non-authoritative meta tag
46.29s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/help?go-get=1
46.44s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/help?go-get=1 (status code 200)
46.44s: get "golang.org/x/vgo/vendor/cmd/go/internal/help": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/help?go-get=1
46.44s: get "golang.org/x/vgo/vendor/cmd/go/internal/help": verifying non-authoritative meta tag
46.44s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/list?go-get=1
46.96s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/list?go-get=1 (status code 200)
46.96s: get "golang.org/x/vgo/vendor/cmd/go/internal/list": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/list?go-get=1
46.96s: get "golang.org/x/vgo/vendor/cmd/go/internal/list": verifying non-authoritative meta tag
46.97s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/run?go-get=1
50.36s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/run?go-get=1 (status code 200)
50.36s: get "golang.org/x/vgo/vendor/cmd/go/internal/run": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/run?go-get=1
50.36s: get "golang.org/x/vgo/vendor/cmd/go/internal/run": verifying non-authoritative meta tag
50.36s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/test?go-get=1
50.48s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/test?go-get=1 (status code 200)
50.48s: get "golang.org/x/vgo/vendor/cmd/go/internal/test": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/test?go-get=1
50.48s: get "golang.org/x/vgo/vendor/cmd/go/internal/test": verifying non-authoritative meta tag
50.48s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/cmdflag?go-get=1
50.61s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/cmdflag?go-get=1 (status code 200)
50.61s: get "golang.org/x/vgo/vendor/cmd/go/internal/cmdflag": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/cmdflag?go-get=1
50.61s: get "golang.org/x/vgo/vendor/cmd/go/internal/cmdflag": verifying non-authoritative meta tag
50.61s: Fetching https://golang.org/x/vgo/vendor/cmd/internal/test2json?go-get=1
52.56s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/internal/test2json?go-get=1 (status code 200)
52.56s: get "golang.org/x/vgo/vendor/cmd/internal/test2json": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/internal/test2json?go-get=1
52.56s: get "golang.org/x/vgo/vendor/cmd/internal/test2json": verifying non-authoritative meta tag
52.56s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/tool?go-get=1
52.68s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/tool?go-get=1 (status code 200)
52.68s: get "golang.org/x/vgo/vendor/cmd/go/internal/tool": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/tool?go-get=1
52.68s: get "golang.org/x/vgo/vendor/cmd/go/internal/tool": verifying non-authoritative meta tag
52.68s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/version?go-get=1
55.56s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/version?go-get=1 (status code 200)
55.56s: get "golang.org/x/vgo/vendor/cmd/go/internal/version": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/version?go-get=1
55.56s: get "golang.org/x/vgo/vendor/cmd/go/internal/version": verifying non-authoritative meta tag
55.56s: Fetching https://golang.org/x/vgo/vendor/cmd/go/internal/vet?go-get=1
57.67s: Parsing meta tags from https://golang.org/x/vgo/vendor/cmd/go/internal/vet?go-get=1 (status code 200)
57.67s: get "golang.org/x/vgo/vendor/cmd/go/internal/vet": found meta tag get.metaImport{Prefix:"golang.org/x/vgo", VCS:"git", RepoRoot:"https://go.googlesource.com/vgo"} at https://golang.org/x/vgo/vendor/cmd/go/internal/vet?go-get=1
57.67s: get "golang.org/x/vgo/vendor/cmd/go/internal/vet": verifying non-authoritative meta tag

That't the output from the make-shift:

package main

import (
   "bufio"
   "fmt"
   "io"
   "os"
   "os/exec"
   "time"
)

func main() {
   r, w := io.Pipe()

   cmd := exec.Command("go", "get", "-v", "-u", "golang.org/x/vgo")
   cmd.Stdout = w
   cmd.Stderr = w

   start := time.Now()
   scanner := bufio.NewScanner(r)

   go func() {
      if err := cmd.Run(); err != nil {
         panic(err)
      }
      w.Close()
   }()

   for scanner.Scan() {
      fmt.Printf("%5.2fs: %v\n", time.Now().Sub(start).Seconds(), scanner.Text())
   }
   if err := scanner.Err(); err != nil {
      fmt.Fprintln(os.Stderr, "reading standard input:", err)
   }
}

@rsc
Copy link
Contributor

rsc commented Jun 6, 2018

Boy that's stupid. Not vgo though.

@rsc rsc changed the title x/vgo: go get of vgo from Europe takes a long time cmd/go: unnecessary go-get queries during go get golang.org/x/vgo Jun 6, 2018
@rsc
Copy link
Contributor

rsc commented Jun 6, 2018

I don't know if go get is just completely broken and does this all the time, or if it has to do with the fact that the imports are going into the vendor directory. Either way, once it has downloaded golang.org/x/vgo it should not be asking questions about directories like golang.org/x/vgo/vendor/cmd/go/internal/vet.

@rsc rsc modified the milestones: vgo, Go1.11 Jun 6, 2018
@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 6, 2018
@rsc
Copy link
Contributor

rsc commented Aug 9, 2018

This has to do with the overuse of the vendor directory in vgo, and it doesn't really matter since the old go get code is going away, as is vgo eventually. Going to leave as is and focus on bugs that won't take care of themselves.

@rsc rsc closed this as completed Aug 9, 2018
@golang golang locked and limited conversation to collaborators Aug 9, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. ToolSpeed
Projects
None yet
Development

No branches or pull requests

5 participants