Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: non-root modules not fetched from a repository hosted on GitHub Enterprise 2.16.12 #36991

Closed
augustoroman opened this issue Feb 3, 2020 · 11 comments
Closed

cmd/go: non-root modules not fetched from a repository hosted on GitHub Enterprise 2.16.12 #36991

augustoroman opened this issue Feb 3, 2020 · 11 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Unplanned

Comments

@augustoroman
Copy link
Contributor

augustoroman commented Feb 3, 2020 •
edited

What version of Go are you using (go version)?

$ go version
go version go1.13.7 darwin/amd64

Does this issue reproduce with the latest release?

Yes, and with go1.14beta1.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/augusto.roman/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.7/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOMOD=""

The GHE server version is 2.16.12

What did you do?

I created a test repository on the company's internal Github Enterprise (GHE) that had a module that was not at the root of the repo. I was unable to import the module from elsewhere using go get:

ghe.company.com/user/repo/packages/lib
  with:  go.mod  --> "module ghe.company.com/user/repo/packages/lib"
         lib.go

+ git tag: packages/lib/v1.0.0

Then elsewhere I tried:

GOPRIVATE=ghe.company.com go get ghe.company.com/user/repo/packages/lib@master

(or with tags of the format: packages/lib/v1.0.0)

go get Output
$ GOPRIVATE=ghe.company.com go get -x -v ghe.company.com/user/repo/packages/lib@v1.0.0
# get https://ghe.company.com/?go-get=1
# get https://ghe.company.com/user/repo/packages/lib?go-get=1
# get https://ghe.company.com/user/repo/packages?go-get=1
# get https://ghe.company.com/user/repo?go-get=1
# get https://ghe.company.com/user?go-get=1
# get //ghe.company.com/user/repo?go-get=1: 200 OK (0.364s)
get "ghe.company.com/user/repo": found meta tag get.metaImport{Prefix:"ghe.company.com/user/repo", VCS:"git", RepoRoot:"https://ghe.company.com/user/repo.git"} at //ghe.company.com/user/repo?go-get=1
mkdir -p /Users/augusto.roman/go/pkg/mod/cache/vcs # git3 https://ghe.company.com/user/repo.git
# lock /Users/augusto.roman/go/pkg/mod/cache/vcs/6c97662eddd96ff8454c4d00d298aa628735acaad52e65b5a3eb02a6df0b1447.lock# /Users/augusto.roman/go/pkg/mod/cache/vcs/6c97662eddd96ff8454c4d00d298aa628735acaad52e65b5a3eb02a6df0b1447 for git3 https://ghe.company.com/user/repo.git
go: finding ghe.company.com/user/repo v1.0.0
cd /Users/augusto.roman/go/pkg/mod/cache/vcs/6c97662eddd96ff8454c4d00d298aa628735acaad52e65b5a3eb02a6df0b1447; git tag -l
0.011s # cd /Users/augusto.roman/go/pkg/mod/cache/vcs/6c97662eddd96ff8454c4d00d298aa628735acaad52e65b5a3eb02a6df0b1447; git tag -l
cd /Users/augusto.roman/go/pkg/mod/cache/vcs/6c97662eddd96ff8454c4d00d298aa628735acaad52e65b5a3eb02a6df0b1447; git ls-remote -q https://ghe.company.com/user/repo.git
# get //ghe.company.com/?go-get=1: 200 OK (0.427s)
# get //ghe.company.com/user?go-get=1: 200 OK (0.435s)
# get //ghe.company.com/user/repo/packages?go-get=1: 200 OK (0.466s)
# get //ghe.company.com/user/repo/packages/lib?go-get=1: 200 OK (0.469s)
0.538s # cd /Users/augusto.roman/go/pkg/mod/cache/vcs/6c97662eddd96ff8454c4d00d298aa628735acaad52e65b5a3eb02a6df0b1447; git ls-remote -q https://ghe.company.com/user/repo.git
# get https://ghe.company.com/user/repo?go-get=1
# get https://ghe.company.com/?go-get=1
# get https://ghe.company.com/user/repo/packages?go-get=1
# get https://ghe.company.com/user?go-get=1
# get https://ghe.company.com/user/repo/packages/lib?go-get=1
# get //ghe.company.com/user/repo?go-get=1: 200 OK (0.076s)
get "ghe.company.com/user/repo": found meta tag get.metaImport{Prefix:"ghe.company.com/user/repo", VCS:"git", RepoRoot:"https://ghe.company.com/user/repo.git"} at //ghe.company.com/user/repo?go-get=1
go: finding ghe.company.com/user/repo v1.0.0
# get //ghe.company.com/?go-get=1: 200 OK (0.180s)
# get //ghe.company.com/user/repo/packages?go-get=1: 200 OK (0.198s)
# get //ghe.company.com/user?go-get=1: 200 OK (0.209s)
# get //ghe.company.com/user/repo/packages/lib?go-get=1: 200 OK (0.251s)
go get ghe.company.com/user/repo/packages/lib@v1.0.0: unrecognized import path "ghe.company.com/user/repo/packages/lib" (parse https://ghe.company.com/user/repo/packages/lib?go-get=1: no go-import meta tags ())

Workaround

It works if I name the module with repo.git instead of repo as:

ghe.company.com/user/repo/packages/lib
  with:  go.mod  --> "module ghe.company.com/user/repo.git/packages/lib"
         lib.go

And elsewhere I can do:

GOPRIVATE=ghe.company.com go get ghe.company.com/user/repo.git/packages/lib@master

or I can use tags of the format: packages/lib/v1.2.3

However, this only works if the go.mod file itself names the module with the .git suffix.

This is in contrast to importing a module at the root which works fine with or without the .git suffix.

(related #17898)
@bcmills
Copy link
Contributor

bcmills commented Feb 3, 2020 •
edited

The protocol that the go command uses to resolve import paths (other than from a handful of unfortunately hard-coded services) is described at https://golang.org/cmd/go/#hdr-Remote_import_paths. Most likely the server you are using does not fully implement that protocol.

So, what is the output of curl -sL https://ghe.company.com/user/repo/packages/lib?go-get=1 | grep go-import?

If the server implements the protocol correctly, it should be something like:

$ curl -sL https://ghe.company.com/user/repo/packages/lib?go-get=1 | grep go-import
	<meta name="go-import" content="ghe.company.com/user/repo git https://ghe.company.com/user/repo.git">

@bcmills bcmills added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Feb 3, 2020
@bcmills bcmills added this to the Unplanned milestone Feb 3, 2020
@bcmills bcmills changed the title cmd/go: add support for non-root modules on GitHub Enterprise import path cmd/go: non-root modules not fetched from a repository hosted on GitHub Enterprise Feb 3, 2020
@bcmills bcmills changed the title cmd/go: non-root modules not fetched from a repository hosted on GitHub Enterprise cmd/go: non-root modules not fetched from a repository hosted on GitHub Enterprise 2.16.12 Feb 3, 2020
@bcmills
Copy link
Contributor

bcmills commented Feb 3, 2020 •
edited

According to https://enterprise.github.com/releases, the current GHE release is 2.19.6. Does this issue reproduce with that version of the server?

@bcmills
Copy link
Contributor

bcmills commented Feb 3, 2020 •
edited

(Note that per https://enterprise.github.com/releases/series/2.16, “GitHub Enterprise Server 2.16 [is] deprecated as of January 22, 2020[.]”)

@augustoroman
Copy link
Contributor Author

Thanks @bcmills, the output for checking the subdirectory is nothing, only the root repo responds with a go-import meta tag:

$ curl -sL https://ghe.company.com/user/repo.git/packages/lib?go-get=1 | grep go-import
$ curl -sL https://ghe.company.com/user/repo.git/?go-get=1 | grep go-import
  <meta name="go-import" content="ghe.company.com/user/repo git https://ghe.company.com/user/repo.git">
$

I don't know what the current release response is -- I'll look into upgrading our server.

@bcmills
Copy link
Contributor

bcmills commented Feb 3, 2020 •
edited

Argh, I typo'd the command. Should be curl -sL https://ghe.company.com/user/repo/packages/lib?go-get=1 | grep go-import, omitting the internal .git suffix.

@augustoroman
Copy link
Contributor Author

Same output:

$ curl -sL https://ghe.company.com/user/repo/packages/lib?go-get=1 | grep go-import
$ curl -sL https://ghe.company.com/user/repo/?go-get=1 | grep go-import
  <meta name="go-import" content="ghe.company.com/user/repo git https://ghe.company.com/user/repo.git">
$

@augustoroman
Copy link
Contributor Author

Still seems buggy to me. If only the exact import path is allowed, why does it bother checking all the parent paths? Public github also doesn't respond to subpath imports

$ curl -sL https://github.com/augustoroman/test-go-modules/packages/lib?go-get=1 | grep go-import
$ curl -sL https://github.com/augustoroman/test-go-modules/?go-get=1 | grep go-import
  <meta name="go-import" content="github.com/augustoroman/test-go-modules git https://github.com/augustoroman/test-go-modules.git">

...but the import works fine. I realize there may be hardcoded exceptions, but it seems to me that if a git repo responds at the root then go get should check the git repo for appropriate sub-packages in the same manner that adding the .git suffix to the repo works.

@bcmills
Copy link
Contributor

bcmills commented Feb 3, 2020

go get accepts any package import path, not (only) an exact module path. It is checking the parent paths in order to attempt to locate the modules corresponding to those paths. (The package with import path ghe.company.com/user/repo/packages/lib could be provided by a module with any path that is a prefix of that import path.)

In contrast, other subcommands (such as go mod download) do know the exact module path, so if the server tells one of those subcommands that the module does not exist, that's it — no other paths are attempted.

So it is important that go get only resolve a module using the same path that any other subcommand would use.

@bcmills
Copy link
Contributor

bcmills commented Feb 3, 2020

Public GitHub is (unfortunately) hard-coded, so any bugs in its implementation of the go-import protocol are masked. (It is not a reliable example to emulate.)

@augustoroman
Copy link
Contributor Author

I've tested this on GHE 2.19.3 and it works as expected, with the subdirectories returning a proper go-import meta tag.

@bcmills
Copy link
Contributor

bcmills commented Feb 5, 2020

Great! Closing this issue, since the go command seems to be working as documented and we don't appear to need to follow up with the GHE maintainers.

@bcmills bcmills closed this as completed Feb 5, 2020
@golang golang locked and limited conversation to collaborators Feb 4, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
3 participants
@augustoroman @bcmills @gopherbot