Skip to content

cmd/go: go get fails with go modules and with private repositories without a go.mod in Gitea #34075

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
rutgerbrf opened this issue Sep 4, 2019 · 8 comments
Closed

cmd/go: go get fails with go modules and with private repositories without a go.mod in Gitea #34075

rutgerbrf opened this issue Sep 4, 2019 · 8 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.

Comments

@rutgerbrf
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.13 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/rutgerb/.cache/go-build"
GOENV="/home/rutgerb/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY="git.profects.com"
GONOSUMDB="git.profects.com"
GOOS="linux"
GOPATH="/home/rutgerb/go"
GOPRIVATE="git.profects.com"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/rutgerb/go/src/git.profects.com/profects/integration-foo-srv/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build707620397=/tmp/go-build -gno-record-gcc-switches"

What did you do?

go get

What did you expect to see?

A successful retrieval of a dependency.

What did you see instead?

build git.profects.com/profects/integration-foo-srv: cannot load git.profects.com/profects/bar-srv/sdk: git ls-remote -q https://git.profects.com/profects/.git in /home/rutgerb/go/pkg/mod/cache/vcs/004fd41ef60cf213426a7c48239770da27fb967a10e0561646dba467c0fe6104: exit status 128:
        fatal: repository 'https://git.profects.com/profects/.git/' not found
@rutgerbrf rutgerbrf changed the title Error running go get with private repositories without a go.mod go get fails with go modules and with private repositories without a go.mod Sep 4, 2019
@bcmills
Copy link
Contributor

bcmills commented Sep 4, 2019 •
edited
Loading

This repository appears to require authentication. Did you place appropriate credentials in your .netrc file or make them available via a credential store in your global .gitconfig?

$ curl https://git.profects.com/profects/?go-get=1
<!doctype html>
<html>
        <head>
                <meta name="go-import" content="git.profects.com/profects git https://git.profects.com/profects/.git">
                <meta name="go-source" content="git.profects.com/profects _ https://git.profects.com/profects/src/branch/master{/dir} https://git.profects.com/profects/src/branch/master{/dir}/{file}#L{line}">
        </head>
        <body>
                go get git.profects.com/profects
        </body>
</html>

$ git clone https://git.profects.com/profects/.git
Cloning into 'profects'...
Username for 'https://git.profects.com': ^C

@bcmills bcmills changed the title go get fails with go modules and with private repositories without a go.mod cmd/go: go get fails with go modules and with private repositories without a go.mod Sep 4, 2019
@bcmills bcmills added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Sep 4, 2019
@rutgerbrf
Copy link
Author

We run Gitea on a dedicated machine.
I do have my credentials configured in my .netrc, and cloning with HTTPS has always worked fine.

@bcmills
Copy link
Contributor

bcmills commented Sep 4, 2019

There isn't much we can do without steps to reproduce the problem.

I suggest using the -x build flag (to print intermediate commands) and then trying those commands in a clean temp directory with GIT_TERMINAL_PROMPT=0 in your environment.

The commands will probably be something like:

git init --bare
git remote add origin -- https://git.profects.com/profects/.git
git ls-remote -q origin

Please try that and report what you find.

@bcmills bcmills added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Sep 4, 2019
@rutgerbrf
Copy link
Author

I see that go get attempts to run git ls-remote -q https://git.profects.com/profects/.git, but this is not a repository that exists. Under our domain we have organizations (profects is one of them), so trying to clone the .git directory in the organization won't work.

In the error

build git.profects.com/profects/integration-foo-srv: cannot load git.profects.com/profects/bar-srv/sdk: git ls-remote -q https://git.profects.com/profects/.git in /home/rutgerb/go/pkg/mod/cache/vcs/004fd41ef60cf213426a7c48239770da27fb967a10e0561646dba467c0fe6104: exit status 128:
        fatal: repository 'https://git.profects.com/profects/.git/' not found

an attempt is done to clone into this directory, but the actual remote would be https://git.profects.com/profects/bar-srv/sdk/.git, which is what I get reported when running curl:

$ curl https://git.profects.com/profects/bar-srv\?go-get\=1  
<!doctype html>
<html>
        <head>
                <meta name="go-import" content="git.profects.com/profects/bar-srv git https://git.profects.com/profects/bar-srv.git">
                <meta name="go-source" content="git.profects.com/profects/bar-srv _ https://git.profects.com/profects/bar-srv/src/branch/master{/dir} https://git.profects.com/profects/bar-srv/src/branch/master{/dir}/{file}#L{line}">
        </head>
        <body>
                go get git.profects.com/profects/bar-srv
        </body>
</html>

@rutgerbrf
Copy link
Author

There are other packages which we import from our private server which are resolved and retrieved correctly.

@bcmills
Copy link
Contributor

bcmills commented Sep 4, 2019 •
edited
Loading

If git.profects.com/profects/.git is not a repository that exists, then you should update the server at https://git.profects.com/profects so that it does not return a go-import <meta> tag referring to such a repository.

(In module mode in Go 1.13, the go command checks all possible module paths in parallel for the requested package. Your server is telling the go command that a module exists at https://git.profects.com/profects/.git, and the go command is erroring out because it can't fetch that module to determine whether it contains the requested package. The solution is to not tell the go command to look there in the first place.)

@rutgerbrf
Copy link
Author

I indeed see this happening with go get -x:

go get -x
(...)
# get //git.profects.com/profects?go-get=1: 200 OK (0.312s)
# get //git.profects.com/?go-get=1: 200 OK (0.315s)
mkdir -p /home/rutgerb/go/pkg/mod/cache/vcs # git3 https://git.profects.com//.git
# lock /home/rutgerb/go/pkg/mod/cache/vcs/f2b15ea9a55d983be763a4c15193e816589f01cc189c95ab8a0b8a67a28faa0e.lockmkdir -p /home/rutgerb/go/pkg/mod/cache/vcs # git3 https://git.profects.com/profects/.git
# /home/rutgerb/go/pkg/mod/cache/vcs/f2b15ea9a55d983be763a4c15193e816589f01cc189c95ab8a0b8a67a28faa0e for git3 https://git.profects.com//.git
cd /home/rutgerb/go/pkg/mod/cache/vcs/f2b15ea9a55d983be763a4c15193e816589f01cc189c95ab8a0b8a67a28faa0e; git ls-remote -q https://git.profects.com//.git
# lock /home/rutgerb/go/pkg/mod/cache/vcs/004fd41ef60cf213426a7c48239770da27fb967a10e0561646dba467c0fe6104.lock# /home/rutgerb/go/pkg/mod/cache/vcs/004fd41ef60cf213426a7c48239770da27fb967a10e0561646dba467c0fe6104 for git3 https://git.profects.com/profects/.git
cd /home/rutgerb/go/pkg/mod/cache/vcs/004fd41ef60cf213426a7c48239770da27fb967a10e0561646dba467c0fe6104; git ls-remote -q https://git.profects.com/profects/.git

So I suppose it is time to open an issue in Gitea.

@thepudds thepudds changed the title cmd/go: go get fails with go modules and with private repositories without a go.mod cmd/go: go get fails with go modules and with private repositories without a go.mod in Gitea Sep 4, 2019
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/193262 mentions this issue: cmd/go/internal/web: log complete URLs

@rutgerbrf rutgerbrf mentioned this issue Sep 5, 2019
Closed
7 tasks
gopherbot pushed a commit that referenced this issue Sep 11, 2019
cmd/go/internal/web: log complete URLs
51be44b 
Incoming URLs may omit the scheme to indicate “either HTTP or HTTPS”.
For such URLs, log the scheme actually used instead of leaving it out.

(This issue was noticed while triaging #34075.)

Updates #34075

Change-Id: I39e5ca83543dd780258d41d5c2c4ba907cd20e5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/193262
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@golang golang locked and limited conversation to collaborators Sep 3, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@bcmills @gopherbot @rutgerbrf