cmd/go: modules do not work with Gitlab.com/Company/Team/Repo Path format #29094

Marvalero · 2018-12-04T10:47:47Z

What version of Go are you using (`go version`)?

root@670ea5e97a56:/app# go version
go version go1.11.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

root@670ea5e97a56:/app# go version
go version go1.11.2 linux/amd64
root@670ea5e97a56:/app# go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/app/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build057206232=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I was testing if go modules made sense for my company. We want to use it to create private libraries. So I wrote a simple test to check I can import submodules:

// File: main.go 
package main

import (
	"gitlab.com/Beamery/DevOps/gotalker"
	"gitlab.com/Beamery/gozzlightyear"
	"gitlab.com/maria.campana/sounds"
)

func main() {
	sounds.Run() // This just calls fmt.Println("Muuu")
	gozzlightyear.Run() // This just calls fmt.Println("to infinity and beyond")
	gotalker.Run()  // This just calls fmt.Println("there is a snake in my boot")
}

// File: go.mod 
module gitlab.com/Beamery/DevOps/maria-modules-test

Then I try to build the code. I am actually running my code on a container. This is the setup I do:

$ docker run -it --entrypoint bash -v /tmp/maria-modules-test:/app -v /home/maria/.ssh/:/root/.ssh/ golang:1.11.2-stretch
root@670ea5e97a56# cd /app
root@670ea5e97a56# git config --global url."ssh://git@gitlab.com/".insteadOf "https://gitlab.com/"
root@670ea5e97a56:/app# go build
build gitlab.com/Beamery/DevOps/maria-modules-test: cannot find module for path gitlab.com/Beamery/DevOps/gotalker

I think the problem is that the repo name is server.com/Company/Team/Repo and go expects to see server.com/Company/Repo. I tried removing the path "gitlab.com/Beamery/DevOps/gotalker" the app gets built correctly

What did you expect to see?

I expect Go Modules to support longer paths like server.com/Company/Team/Repo

What did you see instead?

I see build gitlab.com/Beamery/DevOps/maria-modules-test: cannot find module for path gitlab.com/Beamery/DevOps/gotalker

And If I manually add a require for it on go.mod

module gitlab.com/Beamery/DevOps/maria-modules-test

require (
	gitlab.com/Beamery/gozzlightyear v1.0.0
	gitlab.com/maria.campana/sounds v1.0.0
	gitlab.com/Beamery/DevOps/gotalker v1.0.0
)

I see I try to import server.com/Company/Team/Repo and it will try to clone server.com/Company/Team and look for the tag version Repo/v1.0.0

root@670ea5e97a56:/app# go build
go: finding gitlab.com/Beamery/DevOps/gotalker v1.0.0
go: gitlab.com/Beamery/DevOps/gotalker@v1.0.0: unknown revision gotalker/v1.0.0
go: error loading module requirements

The text was updated successfully, but these errors were encountered:

agnivade · 2018-12-05T06:21:20Z

I think you have to add a go.mod to the gotalker repo. Without that, the module path does not match.

Roping in @myitcv for a deeper look.

Marvalero · 2018-12-05T10:04:21Z

I tried adding a go.mod to my "gitlab.com/Beamery/DevOps/gotalker" library but I cannot fetch it from my other repo. Thanks for looking into it tho 😄

JAicewizard · 2018-12-06T09:40:12Z

https://gitlab.com/antipy/antibuild/api works. maybe see if there is any diferences (or check if you can import that?)

myitcv · 2018-12-06T13:47:17Z

@Marvalero - please include the output from go build -x -v. Although from first glance it appears to suggest that gitlab.com/Beamery/DevOps/gotalker has not been tagged with v1.0.0.

Marvalero · 2018-12-10T15:41:13Z

I created gotalker, gozzlightyear and sounds and they all have a file with a Run() function and tags. If there was something missing in gotalker, it would be missing in the other 2 repos.

This is the output of go build -x -v (Also: thanks! nice command)

root@fd4caec26a46:/app# go build -x -v 
WORK=/tmp/go-build291933567
Fetching https://gitlab.com/Beamery/DevOps/gotalker?go-get=1
Parsing meta tags from https://gitlab.com/Beamery/DevOps/gotalker?go-get=1 (status code 200)
get "gitlab.com/Beamery/DevOps/gotalker": found meta tag get.metaImport{Prefix:"gitlab.com/Beamery/DevOps", VCS:"git", RepoRoot:"https://gitlab.com/Beamery/DevOps.git"} at https://gitlab.com/Beamery/DevOps/gotalker?go-get=1
get "gitlab.com/Beamery/DevOps/gotalker": verifying non-authoritative meta tag
Fetching https://gitlab.com/Beamery/DevOps?go-get=1
Parsing meta tags from https://gitlab.com/Beamery/DevOps?go-get=1 (status code 200)
# /go/pkg/mod/cache/vcs/0001a1afefa10f4157c4164eafd56dab52e8d34658da16e33602533f22c91849 for git2 https://gitlab.com/Beamery/DevOps.git
cd /go/pkg/mod/cache/vcs/0001a1afefa10f4157c4164eafd56dab52e8d34658da16e33602533f22c91849; git ls-remote -q https://gitlab.com/Beamery/DevOps.git
2.255s # cd /go/pkg/mod/cache/vcs/0001a1afefa10f4157c4164eafd56dab52e8d34658da16e33602533f22c91849; git ls-remote -q https://gitlab.com/Beamery/DevOps.git
Fetching https://gitlab.com/Beamery/DevOps?go-get=1
Parsing meta tags from https://gitlab.com/Beamery/DevOps?go-get=1 (status code 200)
get "gitlab.com/Beamery/DevOps": found meta tag get.metaImport{Prefix:"gitlab.com/Beamery/DevOps", VCS:"git", RepoRoot:"https://gitlab.com/Beamery/DevOps.git"} at https://gitlab.com/Beamery/DevOps?go-get=1
Fetching https://gitlab.com/Beamery?go-get=1
Parsing meta tags from https://gitlab.com/Beamery?go-get=1 (status code 200)
Fetching https://gitlab.com?go-get=1
Parsing meta tags from https://gitlab.com?go-get=1 (status code 200)
build gitlab.com/Beamery/DevOps/maria-modules-test: cannot find module for path gitlab.com/Beamery/DevOps/gotalker

Marvalero · 2018-12-10T15:48:21Z

Also, just in case you had doubts:

Beamery/Devops repo does not exists

root@fd4caec26a46:/app# git clone https://gitlab.com/Beamery/DevOps.git
Cloning into 'DevOps'...
GitLab: The project you were looking for could not be found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Gotalker exists (but it is private):

root@fd4caec26a46:/# git clone https://gitlab.com/Beamery/DevOps/gotalker.git
Cloning into 'gotalker'...
remote: Enumerating objects: 11, done.
remote: Counting objects: 100% (11/11), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 11 (delta 1), reused 0 (delta 0)
Receiving objects: 100% (11/11), done.
Resolving deltas: 100% (1/1), done.

And it has many tags (I created them to see if I could fix this import problem):

root@fd4caec26a46:/gotalker# git tag --list
gotalker/v1.0.0
gotalker/v1.0.1
v0.4.0
v1.0.0
v1.0.1

bcmills · 2018-12-12T22:47:46Z

In general it is up to the server to tell the go tool where to find a given repository. We have a few hosting services hard-coded, but gitlab.com is not one of them.

The protocol for other services uses an HTTPS request to resolve the repo root, and is documented here. So in this case the go command will issue a request for https://gitlab.com/Beamery/DevOps/gotalker?go-get=1.

When I do that (using curl https://gitlab.com/Beamery/DevOps/gotalker?go-get=1), I get the following response:

<html><head><meta name="go-import" content="gitlab.com/Beamery/DevOps git https://gitlab.com/Beamery/DevOps.git" /></head></html>

If I understand your description correctly, the actual repo location should instead be https://gitlab.com/Beamery/DevOps/gotalker.git.

So it seems that the issue here is in @gitlabhq's server, and you'll need to escalate to them.

JAicewizard · 2019-01-29T20:55:12Z

gitlab is not willing to propperly support the protocol for repo privacy reasons, would it be an option to add a hardcoded solution for gitlab with a special indicator indicating the repo(and the rest the path from the root) or something like that?
the fact that gitlab doesnt work propperly with go is frustating a lot of people and pretty much marked as no-fix on gitlabs end(privacy reasons)

bcmills · 2019-01-29T21:12:55Z

Sorry, but “privacy reasons” — without any sort of detail as to what those reasons are — is not a sufficient justification for hard-coding provider details into the go command.

We do need some sort of general authentication support for HTTPS fetches, but that's already tracked in #26232. Is there reason to believe that that won't work for GitLab?

JAicewizard · 2019-02-01T17:06:29Z

the issue with gitlab is documented here(https://gitlab.com/gitlab-org/gitlab-ce/issues/1337)
this is indeed discussed in #26232 and if that would come to live it could difinitly fix the issue.

Marvalero changed the title ~~Go modules do not work with Gitlab Path format~~ Go modules do not work with Gitlab.com/Company/Team/Repo Path format Dec 4, 2018

agnivade added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 5, 2018

agnivade changed the title ~~Go modules do not work with Gitlab.com/Company/Team/Repo Path format~~ cmd/go: modules do not work with Gitlab.com/Company/Team/Repo Path format Dec 5, 2018

bcmills added GoCommand cmd/go modules labels Dec 5, 2018

bcmills added this to the Go1.12 milestone Dec 5, 2018

agnivade added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Dec 7, 2018

agnivade removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Dec 10, 2018

Marvalero closed this as completed Dec 12, 2018

Marvalero reopened this Dec 12, 2018

bcmills closed this as completed Dec 12, 2018

bcmills mentioned this issue Jan 8, 2019

cmd/go: get fail with private and subdirectory repo in module mode #29470

Closed

palsivertsen mentioned this issue Apr 4, 2019

cmd/go: define HTTP authentication extension mechanism #26232

Open

bcmills mentioned this issue Jun 13, 2019

cmd/go: GO111MODULE=on go get -v -insecure gitlab.com/xxx/xxx failed #32569

Closed

bcmills mentioned this issue Aug 28, 2019

cmd/go: modules in GitLab virtual directories resolve to truncated repo paths #33905

Closed

golang locked and limited conversation to collaborators Feb 1, 2020

gopherbot added the FrozenDueToAge label Feb 1, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: modules do not work with Gitlab.com/Company/Team/Repo Path format #29094

cmd/go: modules do not work with Gitlab.com/Company/Team/Repo Path format #29094

Marvalero commented Dec 4, 2018

agnivade commented Dec 5, 2018

Marvalero commented Dec 5, 2018

JAicewizard commented Dec 6, 2018

myitcv commented Dec 6, 2018

Marvalero commented Dec 10, 2018

Marvalero commented Dec 10, 2018

bcmills commented Dec 12, 2018 •

edited

JAicewizard commented Jan 29, 2019

bcmills commented Jan 29, 2019

JAicewizard commented Feb 1, 2019

cmd/go: modules do not work with Gitlab.com/Company/Team/Repo Path format #29094

cmd/go: modules do not work with Gitlab.com/Company/Team/Repo Path format #29094

Comments

Marvalero commented Dec 4, 2018

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

agnivade commented Dec 5, 2018

Marvalero commented Dec 5, 2018

JAicewizard commented Dec 6, 2018

myitcv commented Dec 6, 2018

Marvalero commented Dec 10, 2018

Marvalero commented Dec 10, 2018

bcmills commented Dec 12, 2018 • edited

JAicewizard commented Jan 29, 2019

bcmills commented Jan 29, 2019

JAicewizard commented Feb 1, 2019

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

bcmills commented Dec 12, 2018 •

edited