cmd/go: go module not forwarding credentials from .netrc to gitlab subgroups #29888

tigrato · 2019-01-23T11:48:23Z

What version of Go are you using (`go version`)?

$ go 1.11.4

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/tiago/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/tiago/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.4/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/tiago/go/src/gitlab.adinsertp.net/dai-platform/backend-services/media-cache/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/mw/2d5q259n04s64_ylppwvmr9h0000gn/T/go-build740296391=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Last Gitlab release (11.7) enabled go modules downloads for nested groups. It requires authentication for private groups/projects using .netrc file when requesting ?go-get=1 allowing gitlab to understand if the user has the correct access perms for the requested repo.
In case of valid credentials and perms the user should get a go import tag with the project that owns the requested repo.

My $HOME/.netrc file:
machine gitlab.mydomain.com login myuser password mysecrettocken

So I was trying to use go get -v gitlab.mydomain.com/groupA/subGroupB/project/pkg

What did you expect to see?

I expected that go get was able to send the http credentials in .netrc and get the repo that owns the imported package and be able to download it.

What did you see instead?

go get -v gitlab.mydomain.com/groupA/subGroupB/project/pkg Output

Fetching https://gitlab.mydomain.com/groupA/subgroupB/project/pkg?go-get=1
Parsing meta tags from https://gitlab.mydomain.com/groupA/subgroupB/project/pkg?go-get=1 (status code 200)
get "gitlab.mydomain.com/groupA/subgroupB/project": found meta tag get.metaImport{Prefix:"gitlab.mydomain.com/groupA/subgroupB", VCS:"git", RepoRoot:"https://gitlab.mydomain.com/groupA/subgroupB.git"} at https://gitlab.mydomain.com/groupA/subgroupB/project/pkg?go-get=1
get "gitlab.mydomain.com/groupA/subgroupB/project": verifying non-authoritative meta tag
Fetching https://gitlab.mydomain.com/groupA/subgroupB?go-get=1
Parsing meta tags from https://gitlab.mydomain.com/groupA/subgroupB?go-get=1 (status code 200)
Fetching https://gitlab.mydomain.com/groupA/subgroupB?go-get=1
Parsing meta tags from https://gitlab.mydomain.com/groupA/subgroupB?go-get=1 (status code 200)
get "gitlab.mydomain.com/groupA/subgroupB": found meta tag get.metaImport{Prefix:"gitlab.mydomain.com/groupA/subgroupB", VCS:"git", RepoRoot:"https://gitlab.mydomain.com/groupA/subgroupB.git"} at https://gitlab.mydomain.com/groupA/subgroupB?go-get=1
Fetching https://gitlab.mydomain.com/groupA?go-get=1
Parsing meta tags from https://gitlab.mydomain.com/groupA?go-get=1 (status code 200)
Fetching https://gitlab.mydomain.com?go-get=1
Parsing meta tags from https://gitlab.mydomain.com?go-get=1 (status code 200)
go get gitlab.mydomain.com/groupA/subgroupB/project: git ls-remote -q https://gitlab.mydomain.com/groupA/subgroupB.git in /xxx/go/pkg/mod/cache/vcs/5624e7b98ccb36067492e57ba75205fe3d693c9affee381093003e5268dbcaaf: exit status 128:
	remote: The project you were looking for could not be found.
	fatal: repository 'https://gitlab.mydomain.com/groupA/subgroupB.git/' not found

Which fails because it didn't find the correct repository in gitlab. Although if I run:
curl -n https://gitlab.mydomain.com/groupA/subGroupB/project/pkg?go-get=1`:

<html><head><meta name="go-import" content="gitlab.mydomain.com/groupA/subGroupB/project git https://gitlab.mydomain.com/groupA/subGroupB/project.git" /></head></html>
Which returns the correct repository path.

As far I can see, go get is not forwarding .netrc credentials when using subgroups.

The text was updated successfully, but these errors were encountered:

bcmills · 2019-01-24T12:10:45Z

Duplicate of #26232?

vmatyusGitHub · 2019-02-06T11:51:39Z

I encounter the same problem: go version go1.11.5 windows/amd64
Is there a go command flag or environment setting that can enable the connection with .netrc?

In the cmd\go\internal\web2\web.go can be found implementation of sending a GET request with .netrc config. But I think they are not used during go get and go build commands.

bcmills · 2019-02-07T19:18:27Z

No flag should be necessary. Is it possible that your .netrc file is not located in $HOME/.netrc (%USERPROFILE%/_netrc on Windows)?

vmatyusGitHub · 2019-02-07T20:25:43Z

I tried out on Windows and on Linux. The netrc file was at the proper place in both cases.
With curl -n I could receive the correct git repoRoot.

nilium · 2019-02-07T20:27:25Z

From what I can see, go get does not currently use the cmd/go/internal/web2 package that handles netrc loading -- only go mod download would (which has different response expectations than go get, I think?). So, that may explain why go get -v ... doesn't show it -- it's because it doesn't use netrc. Based on #26232, it seems like whether this will even be supported is still in planning.

gopherbot · 2019-02-08T16:31:46Z

Change https://golang.org/cl/161698 mentions this issue: cmd/go/internal/web2: make netrc parsing more robust

tigrato · 2019-02-12T22:09:48Z

From what I can see, go get does not currently use the cmd/go/internal/web2 package that handles netrc loading -- only go mod download would (which has different response expectations than go get, I think?). So, that may explain why go get -v ... doesn't show it -- it's because it doesn't use netrc. Based on #26232, it seems like whether this will even be supported is still in planning.

In this case it is important to have .netrc support for the go get since we can clone the git repo using ssh/git credentials but we cannot discover the repo that owns the package without it.

No flag should be necessary. Is it possible that your .netrc file is not located in $HOME/.netrc (%USERPROFILE%/_netrc on Windows)?
My .netrc is correctly located in $HOME/.netrc as you can see from the curl command with -n

tigrato · 2019-02-16T15:23:10Z

Some more information

Meta import tag request (...?go-get=1) is done using function

go/src/cmd/go/internal/get/vcs.go

Lines 855 to 886 in dca707b

    
           func metaImportsForPrefix(importPrefix string, mod ModuleMode, security web.SecurityMode) (urlStr string, imports []metaImport, err error) { 
        
           	setCache := func(res fetchResult) (fetchResult, error) { 
        
           		fetchCacheMu.Lock() 
        
           		defer fetchCacheMu.Unlock() 
        
           		fetchCache[importPrefix] = res 
        
           		return res, nil 
        
           	} 
        
           	resi, _, _ := fetchGroup.Do(importPrefix, func() (resi interface{}, err error) { 
        
           		fetchCacheMu.Lock() 
        
           		if res, ok := fetchCache[importPrefix]; ok { 
        
           			fetchCacheMu.Unlock() 
        
           			return res, nil 
        
           		} 
        
           		fetchCacheMu.Unlock() 
        
           		urlStr, body, err := web.GetMaybeInsecure(importPrefix, security) 
        
           		if err != nil { 
        
           			return setCache(fetchResult{urlStr: urlStr, err: fmt.Errorf("fetch %s: %v", urlStr, err)}) 
        
           		} 
        
           		imports, err := parseMetaGoImports(body, mod) 
        
           		if err != nil { 
        
           			return setCache(fetchResult{urlStr: urlStr, err: fmt.Errorf("parsing %s: %v", urlStr, err)}) 
        
           		} 
        
           		if len(imports) == 0 { 
        
           			err = fmt.Errorf("fetch %s: no go-import meta tag", urlStr) 
        
           		} 
        
           		return setCache(fetchResult{urlStr: urlStr, imports: imports, err: err}) 
        
           	}) 
        
           	res := resi.(fetchResult) 
        
           	return res.urlStr, res.imports, res.err 
        
           }

Which relies on web.GetMayBeInsecure function

go/src/cmd/go/internal/web/http.go

Lines 76 to 119 in dca707b

    
           func GetMaybeInsecure(importPath string, security SecurityMode) (urlStr string, body io.ReadCloser, err error) { 
        
           	fetch := func(scheme string) (urlStr string, res *http.Response, err error) { 
        
           		u, err := url.Parse(scheme + "://" + importPath) 
        
           		if err != nil { 
        
           			return "", nil, err 
        
           		} 
        
           		u.RawQuery = "go-get=1" 
        
           		urlStr = u.String() 
        
           		if cfg.BuildV { 
        
           			log.Printf("Fetching %s", urlStr) 
        
           		} 
        
           		if security == Insecure && scheme == "https" { // fail earlier 
        
           			res, err = impatientInsecureHTTPClient.Get(urlStr) 
        
           		} else { 
        
           			res, err = httpClient.Get(urlStr) 
        
           		} 
        
           		return 
        
           	} 
        
           	closeBody := func(res *http.Response) { 
        
           		if res != nil { 
        
           			res.Body.Close() 
        
           		} 
        
           	} 
        
           	urlStr, res, err := fetch("https") 
        
           	if err != nil { 
        
           		if cfg.BuildV { 
        
           			log.Printf("https fetch failed: %v", err) 
        
           		} 
        
           		if security == Insecure { 
        
           			closeBody(res) 
        
           			urlStr, res, err = fetch("http") 
        
           		} 
        
           	} 
        
           	if err != nil { 
        
           		closeBody(res) 
        
           		return "", nil, err 
        
           	} 
        
           	// Note: accepting a non-200 OK here, so people can serve a 
        
           	// meta import in their http 404 page. 
        
           	if cfg.BuildV { 
        
           		log.Printf("Parsing meta tags from %s (status code %d)", urlStr, res.StatusCode) 
        
           	} 
        
           	return urlStr, res.Body, nil 
        
           }

This last internally uses the http.DefaultClient and does not have any type of authentication or .netrc parsing as happens in the web2 package.

gabrielruiu · 2019-02-17T21:56:43Z

Ran into same problem which led me to this github issue. At this point it's a blocker and can't use private gitlab repositories as libraries for other go projects. Any workaround or fix will be greatly appreciated. I see this is tagged to be in Go 1.13. If by chance it's possible, releasing it in Go 1.12 would greatly help.

MeirionHughes · 2019-02-21T17:22:26Z

yeah I'm really struggling too - just moved from nodejs to go and this really is a pain point for me aswell (gitlab private nested packages). I kinda just want to pay for npm-like private repos and just have something that works off the bat.

alethenorio · 2019-03-01T15:56:13Z

Same issue here with Go 1.12

Works with curl

$ curl -k -n https://gitlab.com/group/subgroup/myrepo?go-get=1
<html><head><meta name="go-import" content="gitlab.com/group/subgroup/myrepo git https://gitlab.com/group/subgroup/myrepo.git" /></head></html>

But when I try Go get it fails

$ go get -v gitlab.com/group/subgroup/myrepo
Fetching https://gitlab.com/group/subgroup/myrepo?go-get=1
Parsing meta tags from https://gitlab.com/group/subgroup/myrepo?go-get=1 (status code 200)
get "gitlab.com/group/subgroup/myrepo": found meta tag get.metaImport{Prefix:"gitlab.com/group/subgroup", VCS:"git", RepoRoot:"https://gitlab.com/group/subgroup.git"} at https://gitlab.com/group/subgroup/myrepo?go-get=1
get "gitlab.com/group/subgroup/myrepo": verifying non-authoritative meta tag
Fetching https://gitlab.com/group/subgroup?go-get=1
Parsing meta tags from https://gitlab.com/group/subgroup?go-get=1 (status code 200)
Fetching https://gitlab.com/group/subgroup?go-get=1
Parsing meta tags from https://gitlab.com/group/subgroup?go-get=1 (status code 200)
get "gitlab.com/group/subgroup": found meta tag get.metaImport{Prefix:"gitlab.com/group/subgroup", VCS:"git", RepoRoot:"https://gitlab.com/group/subgroup.git"} at https://gitlab.com/group/subgroup?go-get=1
Fetching https://gitlab.com/group?go-get=1
Parsing meta tags from https://gitlab.com/group?go-get=1 (status code 200)
Fetching https://gitlab.com?go-get=1
Parsing meta tags from https://gitlab.com?go-get=1 (status code 200)
go get gitlab.com/group/subgroup/myrepo: git ls-remote -q https://gitlab.com/group/subgroup.git in /home/alethenorio/go/pkg/mod/cache/vcs/3e751597139208a6e518d0bd38652d0e04d1c2a63f7d667f6fb54374ccef23c7: exit status 128:
	remote: The project you were looking for could not be found.
	fatal: repository 'https://gitlab.com/group/subgroup.git/' not found

This allows `go get` to use netrc files. - Reimplement web.Get as a forwarded call to web2.Get. - Replace secure half of web.GetMaybeInsecure with web2.Get and return the body through it instead of the HTTP response. Related to golang#29888.

thepudds · 2019-03-14T15:38:13Z

Would someone here who has experienced a problem on this issue be willing to open up a problem report on the GitLab side? Maybe it is a case of documentation being incorrect, or maybe there is a bug on the GitLab side, or maybe there is a bug on the go tool side... but one of the challenges is I would assume the core Go team are not expert GitLab users and hence it would likely be worthwhile to make sure someone from GitLab is also looking at this set of problem(s). Perhaps there is a workaround, for example, or other possible near-term solution...

MeirionHughes · 2019-03-15T09:49:42Z

Its already documented that the issue is go does not authenticate when it tries to access https://gitlab.com/my/private/repo this in-turn means gitlab returns the incorrect gitlab.com/my/repo.git path by design so as to not give away folder structure within a private organization.

The following code comment documents this:

# If a project is found and the user has access, we return the full project path
# If not, we return the first two components as if it were a simple `namespace/project` path,
# so that we don't reveal the existence of a nested project the user doesn't have access to.
# This means that for an unauthenticated request to `group/subgroup/project/subpackage`
# for a private `group/subgroup/project` with subpackage path `subpackage`, GitLab will respond
# as if the user is looking for project `group/subgroup`, with subpackage path `project/subpackage`.
# Since `go get` doesn't authenticate by default, this means that
# `go get gitlab.com/group/subgroup/project/subpackage` will not work for private projects.
# `go get gitlab.com/group/subgroup/project.git/subpackage` will work, since Go is smart enough
# to figure that out. `import 'gitlab.com/...'` behaves the same as `go get`.

https://gitlab.com/gitlab-org/gitlab-ce/issues/37832

Currently the only robust way I know of is to a) add a system ssh key pair that git will use and gitlab.com will accept and b) have go mod redirect the package to a full git path: i.e.

go.mod: 

module gitlab.com/my/private/group/foo

go 1.12

require (
	gitlab.com/my/private/group/bar v0.0.1
)

replace gitlab.com/my/private/group/bar => gitlab.com/my/private/group/bar.git v0.0.11

this allows the importing (within go) of gitlab.com/my/private/group/bar

certainly works, but requires manually updating the go.mod file if you want to change versions or to simply add a new dependency

This issue deals specifically with the access of https://gitlab.com/my/private/repo such that the http client (that go uses) adds credentials that gitlab.com will accept and (hopefully) the workaround will no longer be required.

thepudds · 2019-03-16T15:58:30Z

The original report here in #29888 (comment) says:

As far I can see, go get is not forwarding .netrc credentials when using subgroups.

The issue one liner here currently reads "cmd/go: go module not forwarding credentials from .netrc to gitlab subgroups".

This issue on the gitlab side was opened a year ago, and is currently closed:
https://gitlab.com/gitlab-org/gitlab-ce/issues/37832

The GitLab 11.7 release notes from roughly 2 months ago say:

Go packages hosted in GitLab can be installed using go get, however this was not supported for private projects in subgroups. Starting with GitLab 11.7, any project can be used as a Go package, including private projects in subgroups.

Private packages are supported by the go get command using a .netrc file, and using a personal access token in the password field.

Thank you MortyChoi for the contribution!

I was pointed to this particular #29888 issue here by someone who stated roughly that the support for private projects in subgroups released in GitLab 11.7 "doesn't work at all".

Part of what I am not following is:

Did GitLab release and announce a capability in GitLab 11.7 that states it supports private projects in subgroups with the go command using a .netrc file, but that new capability never worked end-to-end with the go command?
If using a .netrc for private projects in subgroups with GitLab 11.7+ does work in some cases, what are those cases? (For example, does a .netrc with private projects in subgroups with GitLab 11.7+ work as long as one does go get gitlab.com/group/subgroup/project.git/subpackage rather than go get gitlab.com/group/subgroup/project/subpackage?)

tigrato · 2019-03-16T16:18:31Z

@thepudds

yes, they released it without it ever worked for private repos. It is impossible because when go tries to discover the repo that owns the go package with ?go-get=1 request it does not forward the .netrc credentials. For this go uses a function that is part of web module and credentials are only supported in web2 module as I have previously mentioned.
it only works if all paths (groups, subgroups and repos) are public and open access otherwise it will fail.

thepudds · 2019-03-16T16:43:04Z

Thanks for the answers. It seems surprising given what their release notes say.

I guess I'll ask again if anyone here is willing or interested in opening up an issue on the GitLab side. It seems they should at least update their documentation or release notes. It might also trigger a GitLab employee to show up here and comment further, which might help advance the resolution for this issue here. (I am not a GitLab user, so I am probably not the best person to open an issue there).

- Respect the NETRC environment variable if set. - Ignore lines that contain macro definitions. - Associate the 'machine' token with only the tokens that follow (not precede) it. Updates #29888 Updates #26232 Change-Id: I3128b7d6da2d6492df7c864e165eea1a27384f0f Reviewed-on: https://go-review.googlesource.com/c/go/+/161698 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>

gopherbot · 2019-04-05T17:27:59Z

Change https://golang.org/cl/170879 mentions this issue: cmd/go/internal/web{,2}: consolidate web packages

bcmills added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jan 23, 2019

bcmills added this to the Go1.13 milestone Jan 23, 2019

bcmills mentioned this issue Jan 28, 2019

wiki: add page on using private repositories (e.g., on-prem GitLab, private github.com repos, etc.) #29953

Open

bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Feb 7, 2019

bcmills mentioned this issue Apr 16, 2019

cmd/go: go get fails for long paths #30623

Closed

gopherbot closed this as completed in 58de7c6 Apr 23, 2019

bcmills mentioned this issue May 13, 2019

cmd/go: project depending on project using modules not building: "error loading module requirements" #31929

Closed

bcmills mentioned this issue Jun 13, 2019

cmd/go: GO111MODULE=on go get -v -insecure gitlab.com/xxx/xxx failed #32569

Closed

bcmills mentioned this issue Aug 28, 2019

cmd/go: modules in GitLab virtual directories resolve to truncated repo paths #33905

Closed

bcmills mentioned this issue Sep 5, 2019

cmd/go: get fails on gitlab subgroups due to go-import meta tags referring to nonexistent repos #34094

Closed

golang locked and limited conversation to collaborators Apr 22, 2020

gopherbot added the FrozenDueToAge label Apr 22, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: go module not forwarding credentials from .netrc to gitlab subgroups #29888

cmd/go: go module not forwarding credentials from .netrc to gitlab subgroups #29888

tigrato commented Jan 23, 2019

bcmills commented Jan 24, 2019

vmatyusGitHub commented Feb 6, 2019 •

edited

bcmills commented Feb 7, 2019

vmatyusGitHub commented Feb 7, 2019

nilium commented Feb 7, 2019 •

edited

gopherbot commented Feb 8, 2019

tigrato commented Feb 12, 2019

tigrato commented Feb 16, 2019 •

edited

gabrielruiu commented Feb 17, 2019

MeirionHughes commented Feb 21, 2019 •

edited

alethenorio commented Mar 1, 2019

thepudds commented Mar 14, 2019

MeirionHughes commented Mar 15, 2019 •

edited

thepudds commented Mar 16, 2019

tigrato commented Mar 16, 2019

thepudds commented Mar 16, 2019

gopherbot commented Apr 5, 2019

cmd/go: go module not forwarding credentials from .netrc to gitlab subgroups #29888

cmd/go: go module not forwarding credentials from .netrc to gitlab subgroups #29888

Comments

tigrato commented Jan 23, 2019

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

bcmills commented Jan 24, 2019

vmatyusGitHub commented Feb 6, 2019 • edited

bcmills commented Feb 7, 2019

vmatyusGitHub commented Feb 7, 2019

nilium commented Feb 7, 2019 • edited

gopherbot commented Feb 8, 2019

tigrato commented Feb 12, 2019

tigrato commented Feb 16, 2019 • edited

gabrielruiu commented Feb 17, 2019

MeirionHughes commented Feb 21, 2019 • edited

alethenorio commented Mar 1, 2019

thepudds commented Mar 14, 2019

MeirionHughes commented Mar 15, 2019 • edited

thepudds commented Mar 16, 2019

tigrato commented Mar 16, 2019

thepudds commented Mar 16, 2019

gopherbot commented Apr 5, 2019

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

vmatyusGitHub commented Feb 6, 2019 •

edited

nilium commented Feb 7, 2019 •

edited

tigrato commented Feb 16, 2019 •

edited

MeirionHughes commented Feb 21, 2019 •

edited

MeirionHughes commented Mar 15, 2019 •

edited