Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go mod tidy not using .git/config settings #65041

Open
noel-yap opened this issue Jan 9, 2024 · 9 comments
Open

cmd/go: go mod tidy not using .git/config settings #65041

noel-yap opened this issue Jan 9, 2024 · 9 comments
Labels
GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@noel-yap
Copy link

noel-yap commented Jan 9, 2024

Go version

go version go1.21.5 darwin/arm64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/nyap/Library/Caches/go-build'
GOENV='/Users/nyap/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/nyap/go/pkg/mod'
GONOPROXY='gitlab.com/grailbio/teams/*'
GONOSUMDB='gitlab.com/grailbio/teams/*'
GOOS='darwin'
GOPATH='/Users/nyap/go'
GOPRIVATE='gitlab.com/grailbio/teams/*'
GOPROXY='https://artifactory.ti-apps.aws.grail.com/artifactory/api/go/go,https://proxy.golang.org'
GOROOT='/Users/nyap/.gimme/versions/go1.21.5.darwin.arm64'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/Users/nyap/.gimme/versions/go1.21.5.darwin.arm64/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.21.5'
GCCGO='gccgo'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='0'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/w7/l4dcw2ss1l98wqcxsx1pq03h0000gs/T/go-build3824820425=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

In workspace's .git/config:

[url "ssh://git@gitlab.com"]
	insteadOf = https://gitlab.com

Then go mod tidy.

What did you see happen?

go mod tidy emits lots of no secure protocol found for repository errors:

go: downloading gitlab.com/grailbio/teams/bxds/gsl/core.git v0.3.1
go: grail.com/cmd/bio-gsl imports
	gitlab.com/grailbio/teams/bxds/gsl/core.git/bio/comparator/count: no secure protocol found for repository
go: grail.com/cmd/bio-gsl imports
…

What did you expect to see?

No errors.

After moving the url setting from .git/config to ~/.gitconfig, go mod tidy works as expected.

@bcmills bcmills changed the title go mod tidy not using .git/config settings cmd/go: go mod tidy not using .git/config settings Jan 9, 2024
@bcmills
Copy link
Contributor

bcmills commented Jan 9, 2024

The fact that the local .git/config is ignored is as designed. The git repo in which the go command downloads your dependency is not the same repo as the one containing your own module.

However, the need for this insteadOf directive in the first place is unexpected. gitlab.com is not one of the providers affected by #26134, so the go command should instead probe the repo using git ls-remote to determine which protocol to use. There may be a real bug there that we can fix so that you won't need the insteadOf directive in the first place.

@seankhliao
Copy link
Member

insteadOf might be there for auth reasons?

@bcmills
Copy link
Contributor

bcmills commented Jan 9, 2024

Looking at the code, I think the go command is trying to ping the repo using something like git ls-remote ssh://gitlab.com/user/repo.git.

In contrast, by default GitLab requires the specific user git, which would be a URL like ssh://git@gitlab.com/user/repo.git. We could certainly hard-code a user to try for specific common hosts (at least GitHub and GitLab), but I wonder if there is a more general way to detect which SSH user(s) we should try.

@bcmills bcmills self-assigned this Jan 9, 2024
@bcmills bcmills added this to the Backlog milestone Jan 9, 2024
@Jorropo
Copy link
Member

Jorropo commented Jan 9, 2024

More a work around than anything, but I have this exact configuration in ~/.gitconfig for auth reasons:

[url "ssh://git@github.com/"]
       insteadOf = https://github.com/
[url "ssh://git@gitlab.com/"]
       insteadOf = https://gitlab.com/

And it works perfectly with GOPRIVATE and GOPROXY=direct (go module invokes git cli which then do ssh).

@bcmills
Copy link
Contributor

bcmills commented Jan 9, 2024

@Jorropo, I'm curious: does it still work if you use insteadOf = ssh://github.com/ in that .gitconfig?

(My hypothesis is that it does, and if so I think the go command should just know to do that itself!)

@Jorropo
Copy link
Member

Jorropo commented Jan 10, 2024

@bcmills it does not:

> GOPRIVATE=github.com go get github.com/ipfs/$REPO@latest
go: github.com/ipfs/$REPO@latest: module github.com/ipfs/$REPO: git ls-remote -q origin in /home/hugo/go/pkg/mod/cache/vcs/1572f4aa1ba63bc6815a9b2d64ca8e071b19e19f33e22b6a95cf4bf0efb40471: exit status 128:
	fatal: could not read Username for 'https://github.com': terminal prompts disabled
Confirm the import path was entered correctly.
If this is a private repository, see https://golang.org/doc/faq#git_https for additional information.
> cat ~/.gitconfig
# ...
[url "ssh://git@github.com/"]
       insteadOf = ssh://github.com/
# ...
> nano ~/.gitconfig # put it back to insteadOf = https://github.com/
> GOPRIVATE=github.com go get github.com/ipfs/$REPO@latest # works
go: downloading github.com/ipfs/$REPO v0.2.20

@bcmills
Copy link
Contributor

bcmills commented Jan 10, 2024

@Jorropo, thanks, I guess that's because of #26134. What about for the gitlab.com URL?

@bcmills bcmills removed their assignment Mar 11, 2024
@bcmills bcmills removed the Thinking label Mar 11, 2024
@cherrymui cherrymui added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 11, 2024
@gonnafaraway
Copy link

Hi! Problem is actual, i tried to solve problem like this, but it doesn't help
echo "machine $CI_SERVER_HOST login gitlab-ci-token password ${CI_JOB_TOKEN}" > ~/.netrc export GOPRIVATE=gitlab.com/* export GOPROXY=direct export GOSUMDB=off export PKG_CONFIG_PATH="/usr/lib/x86_64-linux-gnu/pkgconfig/" apt update && apt install -y libssl-dev openssl git gcc musl-dev pkg-config git config --global http.sslverify false
can you help?
i think, that the best way, is to make new env for compiler like GOSSL=off or like GOSECURE=off

@yangyile1990
Copy link

lint-test-job:   # This job also runs in the test stage.
  stage: test    # It can run at the same time as unit-test-job (in parallel).
  script:
    - echo "Linting code..."
    - git config --global url."http://192.168.88.88:30080/".insteadof "https://gitlab.yyle.com/"
    - git config --global url."http://192.168.88.88:31080/".insteadof "http://gitlab.yyle.com/"
    - git config --global --list
    - go env -w GOPRIVATE=gitlab.yyle.com
    - go env -w GOPROXY=https://proxy.golang.org,direct
    - go mod tidy

no secure protocol found for repository

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

7 participants