Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go get doesn't work with private gitbucket repo #19766

Closed
tundratim opened this issue Mar 29, 2017 · 11 comments
Closed

cmd/go: go get doesn't work with private gitbucket repo #19766

tundratim opened this issue Mar 29, 2017 · 11 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@tundratim
Copy link

tundratim commented Mar 29, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.8

What operating system and processor architecture are you using (go env)?

x64 RHEL7

What did you do?

Attempt to do a deep import (go get) on private gitbucket repo

What did you expect to see?

Correct import

What did you see instead?

Fails because clone URL is not getting correctly parsed from gitbucket

I am doing devops support for a client doing a bunch of new go stuff. They are using an internal instance of gitbucket for their corporate code. I am not a go expert by any means, but as I understand it ....

  • Ordinary "go get" works fine, but
  • A "deep" get - somewhere down the repo tree fails because the resultant URL go tries to use is wrong

One of the devs showed me the list of rexeges baked into go for the various vcs products out there, but gitbucket seems not to be on there as best as I could tell.

So ... I guess the ask is either:

  • Fix go to properly interact with gitbucket
  • Provide some clean way for devops folk to add new vcs regex definitions without having to recompile the language.

Any thoughts here would be welcome.

@bradfitz
Copy link
Contributor

See go help importpath (or https://golang.org/cmd/go/#hdr-Remote_import_paths). While there are some specially-recognized cases, any URL works fine if it has the right <meta ...> tags.

gitbucket is not a hosted service, right? So we couldn't regexp it anyway, since users can run it on any hostname?

So I think the best option is for you to modify gitbucket to add the right <meta> tags when cmd/go makes a request to it. The cmd/go command always sends a request with ?go-get=1, which gitbucket can check for and go into a special code path.

Let me know if I misunderstand, but I think there's nothing for us to do here so I'm going to close this. Holler if I should reopen.

@tundratim
Copy link
Author

I think the essence of the problem occurs when we use a git repo not in your list of public servers, we default to the regex found at line 902 in:

https://golang.org/src/cmd/go/vcs.go

This regex creates the following problem as I understand it: For any git repository, if you go get domain.of.repo/repo/name.git, go tries to clone domain.of.repo/repo/name instead of name.git.

IOW, the belief by our devs is that this is incorrect behavior for the default case.

As an aside - I see more and more customers spinning up their own repos on their own hosts wherein this default behavior might come back to bite them.

P.S. We also believe there is an issue with GitBucket itself here as well:

If you try to import domain.of.repo/repo/name/deep/import, go tries to http get domain.of.repo/repo/name/deep/import, which 404s, because gitbucket serves that path at domain.of.repo/repo/name/tree/master/deep/import.

Thanks for the quick turnaround here!

@bradfitz
Copy link
Contributor

What is your concrete problem?

And you didn't reply to my suggestion of using the Go <meta> tags and adding support for that to gitbucket.

@bradfitz
Copy link
Contributor

(And by concrete problem, I mean to give me an actual command go get command that fails.)

@tundratim
Copy link
Author

I have contacted the devs involved and asked for more specific details.

@bentucker
Copy link

Hey Brad,

Here's the issue that I'm seeing:

$  go get -v git.example.local/repo/repo-name.git
# cd .; git ls-remote https://git.example.local/repo/repo-name
fatal: repository 'https://git.example.local/repo/repo-name/' not found
# cd .; git ls-remote git+ssh://git.example.local/repo/repo-name
Unknown command: git-upload-pack '/repo/repo-name'
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
# cd .; git ls-remote ssh://git.example.local/repo/repo-name
Unknown command: git-upload-pack '/repo/repo-name'
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Please note that git ls-remote ssh://git.example.local/repo/repo-name.git works as expected.

From what I can tell, the default vcsPath is being matched, and the regexp is stripping the '.git' from the vcs repo. (At line 902 in vcs.go)

@bradfitz
Copy link
Contributor

bradfitz commented Apr 2, 2017

What does:

git ls-remote https://git.example.local/repo/repo-name.git

... say?

@bentucker
Copy link

That returns the same references that are returned by git ls-remote ssh://...

@bradfitz bradfitz reopened this Apr 5, 2017
@bradfitz bradfitz changed the title Possible Bug And/Or Feature Request cmd/go: go get doesn't work with private gitbucket repo Apr 5, 2017
@rsc
Copy link
Contributor

rsc commented Apr 5, 2017

If your Git server is not one of the recognized ones (say, Github) and not pointed at by a custom domain using tags, then the final fallback is that we'll recognize x.git/y as meaning checkout x with git and then look at y inside the repo. Note that some Git servers (like Github) make the .git optional so that x and x.git mean the same thing, which can be a source of confusion here, but the .git in this syntax is so go get knows to use git at all, not sent to the server. Otherwise it would not be possible to use a repo that didn't end in .git.

If on your server x needs to end in .git, you need to say it twice, once for your server and once for go get:

go get -v git.example.local/repo/repo-name.git.git

Of course that looks awful but it's implied by the use of this fallback. In general it's far preferable to set up an HTTPS redirector to let you separate the import paths from the exact location of the git servers (for example curl https://rsc.io/pdf). I'd suggest doing that if .git.git is deemed too ugly.

The fallback here predates the custom URLs. If I had it to do again I'd probably just remove the fallback entirely and require HTTPS tags.

@hex108
Copy link

hex108 commented Mar 23, 2018

Hi @bradfitz @rsc , could we add a configure to add our own git servers like the public serves in code(https://golang.org/src/cmd/go/vcs.go)? Then go get could skip sending a request with ?go-get=1. It is very useful for those private repos that does not response ?go-get=1 request correctly.

@andybons andybons added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 23, 2018
@andybons andybons added this to the Unplanned milestone Mar 23, 2018
@bcmills
Copy link
Contributor

bcmills commented Jan 18, 2019

#27344 appears to be a duplicate of this issue. The steps to reproduce the error there are easier for me to follow, so I'm going to consolidate the issue there.

@bcmills bcmills closed this as completed Jan 18, 2019
@golang golang locked and limited conversation to collaborators Jan 18, 2020
@rsc rsc removed their assignment Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

8 participants