Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: lack of descriptive error when resolving a dependency #47944

Closed
ETetzlaff opened this issue Aug 24, 2021 · 6 comments
Closed

cmd/go: lack of descriptive error when resolving a dependency #47944

ETetzlaff opened this issue Aug 24, 2021 · 6 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.

Comments

@ETetzlaff
Copy link

When updating to go 1.17, we ran into issues running go mod tidy and resolving private internal packages. These packages were still running on the old go pkg management. In order to fix this we went in and converted said private internal package to use go modules and things worked. However, the error message was from git and not from go and took unnecessary amounts of time to guess what the actual underlying issue was.

@skunkworker

What version of Go are you using (go version)?

$ go version 1.17.0

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Docker 1.17.0-alpine

go env Output
$ go env
```
GO111MODULE=""
GOARCH="amd64"GOBIN=""GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"GOEXE=""
GOEXPERIMENT=""GOFLAGS=""
GOHOSTARCH="amd64"GOHOSTOS="linux"
GOINSECURE=""GOMODCACHE="/go/pkg/mod"
GONOPROXY=""GONOSUMDB=""
GOOS="linux"GOPATH="/go"GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"GOVCS=""
GOVERSION="go1.17"GCCGO="gccgo"
AR="ar"CC="gcc"CXX="g++"
CGO_ENABLED="1"GOMOD="/go/src/gitlab..com/some_project/go.mod"
CGO_CFLAGS="-g -O2"CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1416499769=/tmp/go-build -gno-record-gcc-switches"
```

What did you do?

git config includes a url insteadOf block for private dependencies

[url "https://gitlab-ci-token:<some_token>@gitlab.<company_name>.com/"]
        insteadOf = https://gitlab.<company_name>.com/

GOPRIVATE="*.<company_name>.gitlab.com" go mod tidy

What did you expect to see?

A useful error message describing that private dependency was not resolvable due to modules not being used on it.
ex. Go mod failed trying to resolve dependency: Missing go.mod file inside of `gitlab.<company_name>.com/<package_name>

What did you see instead?

gitlab.<company_name>.com/<project>@v5.78.0+incompatible: reading gitlab.<company_name>.com/<project_name>/go.mod at revision v5.78.0: git ls-remote -q origin in /go/pkg/mod/cache/vcs/ae6c397bd6d9f8e2fb7e35d9db0a45b7a3048e39a7deca3c9e75e7bea2fbca02: exit status 128:
        remote: HTTP Basic: Access denied        fatal: Authentication failed for 'https://gitlab.<company_name>.com/<project>.git/'
@seankhliao
Copy link
Member

Modules can make use of dependencies that haven't adopted modules yet, and that looks clearly to be an authentication error and not due to a missing go.mod

Now that you have authentication working properly, go get gitlab.<company_name>.com/<project>@some-revision-without-go.mod should succeed

@seankhliao seankhliao changed the title go mod 1.17.0 lack of descriptive error when resolving a go pkg dependency cmd/go: lack of descriptive error when resolving a dependency Aug 24, 2021
@seankhliao seankhliao added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Aug 24, 2021
@ETetzlaff
Copy link
Author

Hey, thanks for the response!

I certainly agree the error message is pretty obvious. I think the main thing that was jarring/unexpected about this was more that a go mod tidy on any 1.16.x was successful. With 0 changes to our CI pipelines, a go mod tidy on 1.17.0 produced the reported error. We also tried with the -compat=1.16 flag.

It seems though, that the gitlab individual job tokens don't carry enough scopes for however 1.17.0 is resolving the private dependency. Our solution was to provide a separate token through CI variables specifically for our git .InsteadOf directive.

@bcmills bcmills added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Aug 26, 2021
@bcmills
Copy link
Contributor

bcmills commented Aug 26, 2021

@ETetzlaff, what happens if you run go clean -modcache before go mod tidy with Go 1.16.*? I don't recall any significant changes to how modules are fetched in 1.17, although CL 337850 could perhaps be related if your proxy or module cache already contained the .zip file (but not the synthesized go.mod file) for this repo.

CC @jayconrod @matloob

@bcmills bcmills added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Aug 26, 2021
@ETetzlaff
Copy link
Author

@bcmills So if I change our CI pipeline to include the go clean -modcache just before the go mod tidy it errors out. When I run those commands locally I don't receive an error. If I change the auth token from the job token to our new token in the CI, no error. Below is the error, just with some obfuscated info.

go: <gitlab_url>@v5.78.5+incompatible: reading <gitlab_url>/go.mod at revision v5.78.5: unknown revision v5.78.5

@bcmills
Copy link
Contributor

bcmills commented Aug 30, 2021

That specific error looks like a duplicate of #47028.

@ETetzlaff
Copy link
Author

Ah, thanks for the info! I'll go ahead and close this issue at this point as I have my workaround and I agree, it does look to be similar to that issue.

@golang golang locked and limited conversation to collaborators Aug 30, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ETetzlaff @bcmills @gopherbot @seankhliao