Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: limit simultaneous connections when downloading modules #40671

Closed
bantl23 opened this issue Aug 10, 2020 · 3 comments
Closed

cmd/go: limit simultaneous connections when downloading modules #40671

bantl23 opened this issue Aug 10, 2020 · 3 comments
Labels
GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Backlog

Comments

@bantl23
Copy link

bantl23 commented Aug 10, 2020 •
edited

What version of Go are you using (go version)?

$ go version

go version go1.14.4 linux/amd64

Does this issue reproduce with the latest release?

New feature

What operating system and processor architecture are you using (go env)?

go env Output
$ go env

GO111MODULE=""

GOARCH="amd64"

GOBIN=""

GOCACHE="/home/vagrant/.cache/go-build"

GOENV="/home/vagrant/.config/go/env"

GOEXE=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="linux"

GOINSECURE=""

GONOPROXY="aetd-git.gsfc.nasa.gov"

GONOSUMDB="aetd-git.gsfc.nasa.gov"

GOOS="linux"

GOPATH="/home/vagrant/go"

GOPRIVATE="aetd-git.gsfc.nasa.gov"

GOPROXY="https://proxy.golang.org,direct"

GOROOT="/usr/local/go"

GOSUMDB="sum.golang.org"

GOTMPDIR=""

GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"

GCCGO="gccgo"

AR="ar"

CC="gcc"

CXX="g++"

CGO_ENABLED="1"

GOMOD=""

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build974215287=/tmp/go-build -gno-record-gcc-switches"

What did you do?

go build -v

What did you expect to see?

Modules downloaded properly

What did you see instead?

Modules downloaded properly

Feature request

We currently use go as for all of our code. We have go libraries as dependencies which use a private repository. I have setup the go env GOPRIVATE correctly.

The server hosting our private repository currently only allows ssh connections to git. I have successfully setup git to pull modules over ssh using git config --global url."git@<private_repo>:".insteadOf "https://<private_repo>/"

However our IT policy stats that we can only have 10 simultaneous ssh connections to our git repository.

This policy is implemented using the 10:30:x where after 10 simultaneous connections the drop rate is 30%.

If any code has more than 10 libraries then our CI/CD pipeline fails because of that rule and because go build -v pulls dependencies simultaneously.

The feature I'm requesting is to somehow limit the number of simultaneously connections to a host when downloading modules.

What comes to mind is a command line option in go mod download to limit simultaneous connections or maybe an env variable.

Thanks for your consideration.
@bantl23 bantl23 changed the title cmd/go Limit simultaneous connections when downloading modules proposal: cmd/go Limit simultaneous connections when downloading modules Aug 10, 2020
@gopherbot gopherbot added this to the Proposal milestone Aug 10, 2020
@ianlancetaylor ianlancetaylor changed the title proposal: cmd/go Limit simultaneous connections when downloading modules cmd/go: limit simultaneous connections when downloading modules Oct 28, 2020
@ianlancetaylor
Copy link
Contributor

This doesn't affect API so I don't think this has to be a proposal. Changing to a normal issue.

CC @bcmills @jayconrod

@ianlancetaylor ianlancetaylor added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. and removed Proposal labels Oct 28, 2020
@ianlancetaylor ianlancetaylor modified the milestones: Proposal, Backlog Oct 28, 2020
@ianlancetaylor ianlancetaylor added the GoCommand cmd/go label Oct 28, 2020
@bcmills
Copy link
Contributor

bcmills commented Jan 9, 2024

@bantl23, as of https://go.dev/cl/473277 (Go 1.21) you should be able to set something like GODEBUG=cmdgonetlimit=10 in your environment to limit the number of simultaneous connections.

Please give that a try, and if it works for you perhaps we can consider adding a flag for it instead of just a GODEBUG.

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jan 9, 2024
@gopherbot
Copy link

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

@gopherbot gopherbot closed this as not planned Won't fix, can't repro, duplicate, stale Feb 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
4 participants
@bantl23 @ianlancetaylor @bcmills @gopherbot