Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: document lookup process for qualified module paths #41267

Closed
dwschulze opened this issue Sep 7, 2020 · 7 comments
Closed

cmd/go: document lookup process for qualified module paths #41267

dwschulze opened this issue Sep 7, 2020 · 7 comments
Labels
Documentation FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.

Comments

@dwschulze
Copy link

dwschulze commented Sep 7, 2020 •
edited by ALTree

What version of Go are you using (go version)?

go version go1.15.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/dean/.cache/go-build"
GOENV="/home/dean/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/dean/src/golang/go3p/pkg/mod"
GONOPROXY="ubuntu-18-extssd"
GONOSUMDB="ubuntu-18-extssd"
GOOS="linux"
GOPATH="/home/dean/src/golang/go3p:/home/dean/src/golang/examples/go-module-package-test/package/package-driver"
GOPRIVATE="ubuntu-18-extssd"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/dean/bin/go1.15.1.linux-amd64/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/dean/bin/go1.15.1.linux-amd64/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build013931004=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I have a git repo on a computer on my network (hostname ubuntu-18-extssd, IP 192.168.0.12):

$ git remote --v
origin	dean@ubuntu-18-extssd:gitrepo/go-package-test-stringutil (fetch)

Inside of the repo is a package stringpackage.

The go get command fails:

$ go get -v ubuntu-18-extssd/gitrepo/go-package-test-stringutil/stringpackage
package ubuntu-18-extssd/gitrepo/go-package-test-stringutil/stringpackage: unrecognized import path "ubuntu-18-extssd/gitrepo/go-package-test-stringutil/stringpackage": import path does not begin with hostname

So I try with the IP address:

$ go get -v 192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage
package 192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage: unrecognized import path "192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage": https fetch: Get "https://192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage?go-get=1": dial tcp 192.168.0.12:443: connect: connection refused

I add the following to ~/.gitconfig:

[url "ssh://dean@192.168.0.12:"]
	insteadOf = https://192.168.0.12/

I get the same error.

I've tried multiple ways of setting GOPRIVATE

export GOPRIVATE=192.168.0.12/gitrepo/*
export GOPRIVATE=ubuntu-18-extssd

but get the same error.

You need to actually try this on a local computer with a non-routeable IP address. I've tried the same thing on github.com and it works. The go get command only wants to work on public, routeable IP addresses.

Many companies keep their source code internally on hosts with non-routable IP addresses, and this prevents sharing code and makes using modules impossible.
@ALTree ALTree changed the title go get can't work with a non-routeable IP address cmd/go: get can't work with a non-routeable IP address Sep 8, 2020
@ALTree ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 8, 2020
@seankhliao
Copy link
Member

This looks like 2 separate errors

import path does not begin with hostname

go currently does not consider names without dots as hosts, as of #37641 these are considered reserved names for the stdlib

https fetch: Get "https://192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage?go-get=1": dial tcp 192.168.0.12:443: connect: connection refused

remote imports need to follow https://golang.org/cmd/go/#hdr-Remote_import_paths
specifically either end with .git for a git repo or answer to a http/https request with the meta tag

@dwschulze
Copy link
Author

dwschulze commented Sep 8, 2020 •
edited

I added .git to the url like this

go get 192.168.0.12/gitrepo/go-package-test-stringutil.git/stringpackage

and it gives a different message:

package 192.168.0.12/gitrepo/go-package-test-stringutil.git/stringpackage: cannot download, git://192.168.0.12/gitrepo/go-package-test-stringutil uses insecure protocol

My ~/.gitconfig has this

[url "ssh://dean@192.168.0.12:"] insteadOf = https://192.168.0.12/

When go get calls git it seems to be picking up what I have in ~/.gitconfig but converts the ssh: prefix to git:. I believe that is the same thing, though. I have no idea why go get considers that an insecure protocol.

@jayconrod
Copy link
Contributor

@dwschulze Could you run with -x and post the log? That should show the actual commands run.

The git protocol is not secure. The Protocols has more information. git+ssh may be what you want instead. Or if you trust your local network, set GOINSECURE=192.168.0.12.

@jayconrod jayconrod added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Sep 8, 2020
@seankhliao
Copy link
Member

your git config also needs to rewrite the git protocol to ssh

@dwschulze
Copy link
Author

I removed the ssh:// prefix from my ~/.gitconfig and now it is working:

[url "dean@192.168.0.12:"] insteadOf = https://192.168.0.12/

The docs really need to cover this and explain why having the ssh:// prefix gives an insecure protocol message. That is very confusing.

@jayconrod
Copy link
Contributor

I'd like to guide this issue to some resolution. I don't think we really know what went wrong. Having the -x log would help.

This doesn't seem related to whether the host contains a non-routable address or not. We don't allow module paths that don't have a dot in the first path element (see Module paths and versions, but local names and IP addresses are fine.

Finding a repository for a module path describes the process. That section doesn't cover qualified module paths, so it should definitely be updated.

When the module path ends with a VCS qualifier like .git, the go command will derive a repository URL directly from that path instead of sending a ?go-get=1 request. It doesn't know the scheme, so it tries git, https, http, git+ssh, and ssh (git and http are only allowed if GOINSECURE matches).

@jayconrod jayconrod changed the title cmd/go: get can't work with a non-routeable IP address cmd/go: document lookup process for qualified module paths Sep 21, 2020
@jayconrod jayconrod added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Sep 21, 2020
@gopherbot
Copy link

Change https://golang.org/cl/276354 mentions this issue: content/static/doc: document vcs qualifiers in module paths

@golang golang locked and limited conversation to collaborators Dec 10, 2021
passionSeven added a commit to passionSeven/website that referenced this issue Oct 18, 2022
@passionSeven
content/static/doc: document vcs qualifiers in module paths
326f75f 
For golang/go#33637
Fixes golang/go#41267

Change-Id: Ic7928c05ef200b574afc15acdbabdc6ab2d5e30d
Reviewed-on: https://go-review.googlesource.com/c/website/+/276354
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Documentation FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@jayconrod @dwschulze @ALTree @gopherbot @seankhliao