Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: add support for GitHub Enterprise import path #17898

Closed
matope opened this issue Nov 12, 2016 · 22 comments
Closed

cmd/go: add support for GitHub Enterprise import path #17898

matope opened this issue Nov 12, 2016 · 22 comments
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Milestone

Comments

@matope
Copy link
Contributor

matope commented Nov 12, 2016

For now, go get does not fetch packages hosted on GitHub Enterprise correctly without any workarounds.

$ go get -v ghe.in.my.company/user/repo/sub
Fetching ghe.in.my.company/user/repo/sub?go-get=1
Parsing meta tags from https://ghe.in.my.company/user/repo/sub?go-get=1 (status code 404)
package ghe.in.my.company/user/repo/sub: unrecognized import path "ghe.in.my.company/user/repo/sub" (parse https://ghe.in.my.company/user/repo/sub?go-get=1: no go-import meta tags)

When I execute go get, it issues an HTTP request to find go-import meta-tags.
But GHE does not return any meta-tag because ghe.in.my.company/user/repo/sub is 404 for GHE ( ghe.in.my.company/user/repo/tree/master/sub would work fine).

So, in order to use golang with GHE, we have to

  • a) Use import path with .vcs. ex) ghe.in.my.company/user/repo.git/sub
  • b) or use a redirect server that attach go-import meta-tags.

But, a) would be a error prone especially for newbies. Many people at first download repo into ghe.in.my.company/user/repo but, after that, go get would download it's sub packages into ghe.in.my.company/user.git/repo. I think this would make people confused.
b) This would work fine, but I think managing a server only for dependency resolution is not very efficient.

So, I think it would be great if cmd/go natively support GitHub Enterprise.

@matope
Copy link
Contributor Author

matope commented Nov 12, 2016

My patch is here. Could anybody please review this?
https://go-review.googlesource.com/#/c/33171/

@gopherbot
Copy link

CL https://golang.org/cl/33171 mentions this issue.

@bradfitz bradfitz added this to the Go1.9 milestone Nov 12, 2016
@bradfitz bradfitz added NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. NeedsFix The path to resolution is known, but the work has not been done. labels Nov 12, 2016
@gopherbot
Copy link

CL https://golang.org/cl/37176 mentions this issue.

@matope
Copy link
Contributor Author

matope commented Feb 19, 2017

Hi, @bradfitz. As go cmd had a major refactoring, I've abandoned the previous CL and created new one which is rebased on Go1.8 with some cleanup.

I'd appreciate you if you could review my new one.

@rsc rsc modified the milestones: Go1.10, Go1.9 Jun 12, 2017
@rsc rsc changed the title cmd/go: add support for GithHub Enterprise import path cmd/go: add support for GitHub Enterprise import path Jun 12, 2017
@rsc
Copy link
Contributor

rsc commented Jun 12, 2017

It sounds like maybe the solution should be to support ?go-get=1 in the GHE 404 page, but we'd have to write down exactly what that would mean before approaching GitHub (or suggesting that their paying customers do that).

@bradfitz bradfitz removed the NeedsFix The path to resolution is known, but the work has not been done. label Jun 27, 2017
@blaskovicz
Copy link

Here is the Gitlab implementation, for reference. It seems that there was a similar issue with Golang subpackages for it as well.

@jijojv
Copy link

jijojv commented Aug 25, 2017

ok 1.9 is out and disappointed that it still doesn't work with GitHub Enterprise over ssh as-is.

git clone git@ghe.mycompany.com:org/repo.git does not create a .git directory but go decided not to allow any git url schemes and not to use the standard git cloning convention and create a directory with .git suffix which provides a confusing experience for everyone involved...

@ianlancetaylor
Copy link
Contributor

@jijojv I encourage you to send a patch, so that this can be fixed for 1.10.

@jijojv
Copy link

jijojv commented Aug 25, 2017

@ianlancetaylor sorry just a lowly github admin, but i can in python if it'll get accepted. Looks like someone attempted above, so shouldn't be too hard for a real go user/fan.

@matope
Copy link
Contributor Author

matope commented Sep 10, 2017

@rsc Hi, my colleague asked Github Enterprise team about this issue.

They said that they respond <meta> tag from repository page but do not respond from 404 page intentionally due to preventing to leak private repos.Also they suggest a) adding .git into import path, b) using git clone instead of go get or c) using some proxy.

@matope
Copy link
Contributor Author

matope commented Sep 10, 2017

I think their intention is reasonable. And as I mentioned in the description of this issue, all of the suggested way are painful a little. So I'd like to propose adding support for Github Enterprise in go get.

@jameschensmith
Copy link

@matope, our team is getting 200 response (via git config --global url."https://${GITHUB_TOKEN}:x-oauth-basic@my.ghe.domain/".insteadOf "https://my.ghe.domain/"), but since our repo is private, GitHub Enterprise doesn't return the meta tag unless you're authenticated. Is there a separate issue that needs to be created for this, or is there a workaround other than the options (a - c) that you suggested above? Thanks

@matope
Copy link
Contributor Author

matope commented Sep 12, 2017

@poeia As far as I've tested briefly, my patch works for private repositories too.

The CL I sent regards any domains as github instance if the servers return "Server: GitHub.com" header. It doesn't depend on meta-tag. And because Github Eenterprise (probably) returns the header from every pages, my CL could treat private repos well too.

# go1.8
$ go get -v my.ghe.domain/matope/private-repo
Fetching https://my.ghe.domain/matope/private-repo?go-get=1
Parsing meta tags from https://my.ghe.domain/matope/private-repo?go-get=1 (status code 404)
package my.ghe.domain/matope/private-repo: unrecognized import path "my.ghe.domain/matope/private-repo" (parse https://my.ghe.domain/matope/private-repo?go-get=1: no go-import meta tags ())

# go1.8 with CL37176
$ ./git/go/bin/go get -v my.ghe.domain/matope/private-repo
Fetching https://my.ghe.domain/matope/private-repo?go-get=1
Parsing meta tags from https://my.ghe.domain/matope/private-repo?go-get=1 (status code 404)
my.ghe.domain/matope/private-repo (download)
my.ghe.domain/matope/private-repo/sub2
my.ghe.domain/matope/private-repo/sub
my.ghe.domain/matope/private-repo

Note that I'm using not tokens but certificates to authenticate.

@jameschensmith
Copy link

@matope, my concern was specifically for a 200 response, and whether or not it is linked with the same issue the 404 response gives. Do they both send you to the login page for private repos? I'm wondering if there is a way to bypass the login without the options (a-c) that you provided, or if we will have to wait for this patch. Just curious. Hope this patch gets implemented into 1.10!

@rsc
Copy link
Contributor

rsc commented Dec 1, 2017

Moving to Go 1.11. I hope we'll have a more comprehensive story around company repos then.

@rsc rsc modified the milestones: Go1.10, Go1.11 Dec 1, 2017
@lpar
Copy link

lpar commented Jan 29, 2018

I just tried

env GOPATH=/home/meta/tmp go get -v github.mycompany.com/meta/mypackage

and it worked, fetched my package and its dependencies into ~/tmp. Is it possible that this has been fixed on the server side?

@jijojv
Copy link

jijojv commented Jan 29, 2018

@lpar is private mode not enabled on your GHE ? - i.e. can you clone WITHOUT any auth - that's why it might work.

@lpar
Copy link

lpar commented Jan 29, 2018

Aha, further investigation reveals that a long time ago I put

[url "git@github.mycompany.com:"]
insteadOf = https://github.mycompany.com/

in my ~/.gitconfig and forgot about it, and that's why it worked.

@dmitris
Copy link
Contributor

dmitris commented Mar 7, 2018

we had to use b) or use a redirect server that attach go-import meta-tags. for years. I'd love to see a way for go get to work with GHE "directly" but not sure how the thorny issue of GHE authentication can be worked out: go or vgo would need to be able to pick the "ambient" credentials such as from the ssh-agent, would be very interested in any suggestions or experiences. (I'll subscribe to the issue, would be happy to test if this comes to any changes or PoC).

@cjs
Copy link

cjs commented Mar 27, 2018

With the GitHub Enterprise 2.13 release, on-premises GitHub Enterprise instances will now send the <meta> tags expected by go get for private instances and all repositories.

For instances that are in private mode, you will still need to manage authentication for the git pull phase of go get, but there are easy workarounds such as putting an authentication token in your .netrc or the insteadOf technique to force SSH as mentioned above.

@rsc
Copy link
Contributor

rsc commented Mar 27, 2018

Calling this fixed. Thanks so much @cjs.

@rsc rsc closed this as completed Mar 27, 2018
@matope
Copy link
Contributor Author

matope commented Apr 14, 2018

@cjs Our team confirmed that it works. Thank you very much!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Projects
None yet
Development

No branches or pull requests