New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cmd/go: parallelize go get downloads #11821
Comments
The main obstacle is that you have to fetch A to find out that it imports B and C. |
There are also rate-limit issues. e.g. common setup is to use a shared ssh session to github (sort of a connection pool via So this feature would need a bunch of tuneables. A different approach to target the increasing complexity of |
Is this correct way (in case GOPATH contains exactly one element) to speedup find $GOPATH/src -type d -name .git -printf "%h\0" |
xargs -0 -P64 -I{} sh -c "cd {}; git pull; git submodule update"
go get -v -t all I suppose it may miss some corner cases like default branch change on GitHub, but I'm not sure is |
Module downloads are performed in parallel. I don't think that retrofitting |
Running
go get -u
for an app with a decent number of dependencies can be extremely slow from a location with high latency to GitHub (and other code hosts, too -- it's even worse when there are redirects involved). For example, from my machine in Sydney the following command takes almost 90 seconds to download 18 repos:This is in stark contrast to general use of the
go
command, which is otherwise pleasantly fast.I notice that downloads happen serially. It seems like
go get
could benefit greatly from doing them concurrently. Are there any major obstacles to doing so?The text was updated successfully, but these errors were encountered: