Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: cache meta lookups during go get #9249

Closed
bradfitz opened this issue Dec 10, 2014 · 4 comments
Closed

cmd/go: cache meta lookups during go get #9249

bradfitz opened this issue Dec 10, 2014 · 4 comments
Labels
FrozenDueToAge
Milestone
Go1.5

Comments

@bradfitz
Copy link
Contributor

Installing godoc, I see many meta tag lookups. Can we do better?

$ go get -u -v golang.org/x/tools/cmd/godoc
Fetching https://golang.org/x/tools/cmd/godoc?go-get=1
Parsing meta tags from https://golang.org/x/tools/cmd/godoc?go-get=1 (status code 200)
get "golang.org/x/tools/cmd/godoc": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/cmd/godoc?go-get=1
get "golang.org/x/tools/cmd/godoc": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
golang.org/x/tools (download)
Fetching https://golang.org/x/tools/astutil?go-get=1
Parsing meta tags from https://golang.org/x/tools/astutil?go-get=1 (status code 200)
get "golang.org/x/tools/astutil": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/astutil?go-get=1
get "golang.org/x/tools/astutil": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/blog?go-get=1
Parsing meta tags from https://golang.org/x/tools/blog?go-get=1 (status code 200)
get "golang.org/x/tools/blog": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/blog?go-get=1
get "golang.org/x/tools/blog": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/blog/atom?go-get=1
Parsing meta tags from https://golang.org/x/tools/blog/atom?go-get=1 (status code 200)
get "golang.org/x/tools/blog/atom": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/blog/atom?go-get=1
get "golang.org/x/tools/blog/atom": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/present?go-get=1
Parsing meta tags from https://golang.org/x/tools/present?go-get=1 (status code 200)
get "golang.org/x/tools/present": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/present?go-get=1
get "golang.org/x/tools/present": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/container/intsets?go-get=1
Parsing meta tags from https://golang.org/x/tools/container/intsets?go-get=1 (status code 200)
get "golang.org/x/tools/container/intsets": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/container/intsets?go-get=1
get "golang.org/x/tools/container/intsets": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/go/callgraph?go-get=1
Parsing meta tags from https://golang.org/x/tools/go/callgraph?go-get=1 (status code 200)
get "golang.org/x/tools/go/callgraph": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/go/callgraph?go-get=1
get "golang.org/x/tools/go/callgraph": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/go/exact?go-get=1
Parsing meta tags from https://golang.org/x/tools/go/exact?go-get=1 (status code 200)
get "golang.org/x/tools/go/exact": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/go/exact?go-get=1
get "golang.org/x/tools/go/exact": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/go/gcimporter?go-get=1
Parsing meta tags from https://golang.org/x/tools/go/gcimporter?go-get=1 (status code 200)
get "golang.org/x/tools/go/gcimporter": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/go/gcimporter?go-get=1
get "golang.org/x/tools/go/gcimporter": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/go/types?go-get=1
Parsing meta tags from https://golang.org/x/tools/go/types?go-get=1 (status code 200)
get "golang.org/x/tools/go/types": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/go/types?go-get=1
get "golang.org/x/tools/go/types": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/go/loader?go-get=1
Parsing meta tags from https://golang.org/x/tools/go/loader?go-get=1 (status code 200)
get "golang.org/x/tools/go/loader": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/go/loader?go-get=1
get "golang.org/x/tools/go/loader": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/go/ssa?go-get=1
Parsing meta tags from https://golang.org/x/tools/go/ssa?go-get=1 (status code 200)
get "golang.org/x/tools/go/ssa": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/go/ssa?go-get=1
get "golang.org/x/tools/go/ssa": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/go/types/typeutil?go-get=1
Parsing meta tags from https://golang.org/x/tools/go/types/typeutil?go-get=1 (status code 200)
get "golang.org/x/tools/go/types/typeutil": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/go/types/typeutil?go-get=1
get "golang.org/x/tools/go/types/typeutil": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/go/pointer?go-get=1
Parsing meta tags from https://golang.org/x/tools/go/pointer?go-get=1 (status code 200)
get "golang.org/x/tools/go/pointer": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/go/pointer?go-get=1
get "golang.org/x/tools/go/pointer": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/go/ssa/ssautil?go-get=1
Parsing meta tags from https://golang.org/x/tools/go/ssa/ssautil?go-get=1 (status code 200)
get "golang.org/x/tools/go/ssa/ssautil": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/go/ssa/ssautil?go-get=1
get "golang.org/x/tools/go/ssa/ssautil": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/godoc?go-get=1
Parsing meta tags from https://golang.org/x/tools/godoc?go-get=1 (status code 200)
get "golang.org/x/tools/godoc": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/godoc?go-get=1
get "golang.org/x/tools/godoc": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/godoc/analysis?go-get=1
Parsing meta tags from https://golang.org/x/tools/godoc/analysis?go-get=1 (status code 200)
get "golang.org/x/tools/godoc/analysis": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/godoc/analysis?go-get=1
get "golang.org/x/tools/godoc/analysis": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/godoc/util?go-get=1
Parsing meta tags from https://golang.org/x/tools/godoc/util?go-get=1 (status code 200)
get "golang.org/x/tools/godoc/util": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/godoc/util?go-get=1
get "golang.org/x/tools/godoc/util": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/godoc/vfs?go-get=1
Parsing meta tags from https://golang.org/x/tools/godoc/vfs?go-get=1 (status code 200)
get "golang.org/x/tools/godoc/vfs": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/godoc/vfs?go-get=1
get "golang.org/x/tools/godoc/vfs": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/godoc/vfs/httpfs?go-get=1
Parsing meta tags from https://golang.org/x/tools/godoc/vfs/httpfs?go-get=1 (status code 200)
get "golang.org/x/tools/godoc/vfs/httpfs": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/godoc/vfs/httpfs?go-get=1
get "golang.org/x/tools/godoc/vfs/httpfs": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/godoc/redirect?go-get=1
Parsing meta tags from https://golang.org/x/tools/godoc/redirect?go-get=1 (status code 200)
get "golang.org/x/tools/godoc/redirect": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/godoc/redirect?go-get=1
get "golang.org/x/tools/godoc/redirect": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/godoc/static?go-get=1
Parsing meta tags from https://golang.org/x/tools/godoc/static?go-get=1 (status code 200)
get "golang.org/x/tools/godoc/static": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/godoc/static?go-get=1
get "golang.org/x/tools/godoc/static": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/godoc/vfs/gatefs?go-get=1
Parsing meta tags from https://golang.org/x/tools/godoc/vfs/gatefs?go-get=1 (status code 200)
get "golang.org/x/tools/godoc/vfs/gatefs": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/godoc/vfs/gatefs?go-get=1
get "golang.org/x/tools/godoc/vfs/gatefs": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/godoc/vfs/mapfs?go-get=1
Parsing meta tags from https://golang.org/x/tools/godoc/vfs/mapfs?go-get=1 (status code 200)
get "golang.org/x/tools/godoc/vfs/mapfs": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/godoc/vfs/mapfs?go-get=1
get "golang.org/x/tools/godoc/vfs/mapfs": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/godoc/vfs/zipfs?go-get=1
Parsing meta tags from https://golang.org/x/tools/godoc/vfs/zipfs?go-get=1 (status code 200)
get "golang.org/x/tools/godoc/vfs/zipfs": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/godoc/vfs/zipfs?go-get=1
get "golang.org/x/tools/godoc/vfs/zipfs": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
Fetching https://golang.org/x/tools/playground?go-get=1
Parsing meta tags from https://golang.org/x/tools/playground?go-get=1 (status code 200)
get "golang.org/x/tools/playground": found meta tag main.metaImport{Prefix:"golang.org/x/tools", VCS:"git", RepoRoot:"https://go.googlesource.com/tools"} at https://golang.org/x/tools/playground?go-get=1
get "golang.org/x/tools/playground": verifying non-authoritative meta tag
Fetching https://golang.org/x/tools?go-get=1
Parsing meta tags from https://golang.org/x/tools?go-get=1 (status code 200)
golang.org/x/tools/blog/atom
golang.org/x/tools/present
golang.org/x/tools/go/exact
golang.org/x/tools/astutil
golang.org/x/tools/container/intsets
golang.org/x/tools/godoc/vfs
golang.org/x/tools/go/types
golang.org/x/tools/godoc/util
golang.org/x/tools/godoc/vfs/httpfs
golang.org/x/tools/godoc/redirect
golang.org/x/tools/blog
golang.org/x/tools/godoc/static
golang.org/x/tools/godoc/vfs/gatefs
golang.org/x/tools/godoc/vfs/mapfs
golang.org/x/tools/godoc/vfs/zipfs
golang.org/x/tools/playground
golang.org/x/tools/go/gcimporter
golang.org/x/tools/go/types/typeutil
golang.org/x/tools/go/loader
golang.org/x/tools/go/ssa
golang.org/x/tools/go/callgraph
golang.org/x/tools/go/ssa/ssautil
golang.org/x/tools/go/pointer
golang.org/x/tools/godoc/analysis
golang.org/x/tools/godoc
golang.org/x/tools/cmd/godoc
@bradfitz bradfitz self-assigned this Dec 10, 2014
@rsc
Copy link
Contributor

rsc commented Apr 10, 2015

It's clearly a mistake to keep checking the same URLs over and over again during a single 'go get'. I think you wrote the code Brad. :-)

@rsc rsc changed the title cmd/go: many remote meta lookups cmd/go: cache meta lookups during go get Apr 10, 2015
@rsc rsc added this to the Go1.5Maybe milestone Apr 10, 2015
@bradfitz
Copy link
Contributor Author

Yup, will fix. I keep hitting this.

@cespare
Copy link
Contributor

cespare commented Apr 10, 2015

Ah, I guess this is the other facet to the issue I described in https://groups.google.com/d/msg/golang-nuts/ixF0h1YYCoQ/9wfBu4deaM0J. (That is, combine the meta lookups with https timeouts -- see #10120 -- and go get can take several minutes.)

@bradfitz
Copy link
Contributor Author

Sent https://go-review.googlesource.com/8727

@mikioh mikioh modified the milestones: Go1.5, Go1.5Maybe Apr 14, 2015
@golang golang locked and limited conversation to collaborators Jun 25, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
Go1.5
Development

No branches or pull requests
5 participants
@bradfitz @mikioh @rsc @cespare @gopherbot