Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: unclear error when server does not provide 'go-import' metadata for a module in a subdirectory #35013

Open
graywolf opened this issue Oct 19, 2019 · 13 comments
Labels
Documentation modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@graywolf
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.13.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/wolf/.cache/go-build"
GOENV="/home/wolf/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/wolf/go"
GOPRIVATE=""
GOPROXY="direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/wolf/tmp/golang_test/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build652827916=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Created go module:

+$ tree .
.
└── bar
    ├── bar.go
    └── go.mod

1 directory, 2 files
+$ find . -name .git -prune -o -type f -print -exec cat {} \;
./bar/bar.go
package bar

func Bar() string {
        return "bar"
}
./bar/go.mod
module git.sr.ht/~graywolf/foo/bar

go 1.13

pushed it to the repository (https://git.sr.ht/~graywolf/foo) and tried to use it in following program:

package main

import (
        "fmt"
        "git.sr.ht/~graywolf/foo/bar"
)

func main() {
        fmt.Println(bar.Bar())
}

What did you expect to see?

+$ go run sr.go
bar

What did you see instead?

+$ go run sr.go
go: finding git.sr.ht/~graywolf/foo latest
build command-line-arguments: cannot load git.sr.ht/~graywolf/foo/bar: module git.sr.ht/~graywolf/foo@latest (v0.0.0-20191019153505-33a4721605aa) found, but does not contain package git.sr.ht/~graywolf/foo/bar
@bcmills
Copy link
Contributor

bcmills commented Oct 21, 2019

Thanks for the clear steps to reproduce the problem.

The root cause seems to be that your HTTPS server is not serving a <meta name="go-import" […]> tag for the path git.sr.ht/~graywolf/foo/bar — it is only serving a tag for the /foo prefix:

example.com$ curl -sL https://git.sr.ht/~graywolf/foo | grep -A1 go-import
<meta name="go-import"
  content="git.sr.ht/~graywolf/foo git https://git.sr.ht/~graywolf/foo">

example.com$ curl -sL https://git.sr.ht/~graywolf/foo/bar | grep -A1 go-import

example.com$

That causes the go command to believe that there is no such module for that path.
Probably the best we could do in the go command is emit a better diagnostic for this case, but it's not obvious to me how we could detect it in general.

Or, perhaps we can improve the documentation for the remote import path protocol.

What documentation were you using when setting up the server?

CC @jayconrod @thepudds

@bcmills bcmills added Documentation modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Oct 21, 2019
@bcmills bcmills added this to the Unplanned milestone Oct 21, 2019
@bcmills bcmills changed the title Error while using go module located in a subdirectory cmd/go: Error while using go module located in a subdirectory Oct 21, 2019
@bcmills bcmills changed the title cmd/go: Error while using go module located in a subdirectory cmd/go: unclear error when server does not provide 'go-import' metadata for a module in a subdirectory Oct 21, 2019
@graywolf
Copy link
Author

But github does not provide that either

+$ curl -sSf https://github.com/graywolf/foo | grep -A1 go-import
  <meta name="go-import" content="github.com/graywolf/foo git https://github.com/graywolf/foo.git">

+$ curl -sSf https://github.com/graywolf/foo/bar | grep -A1 go-import
curl: (22) The requested URL returned error: 404 Not Found

and over there it works just fine. As far as I can tell only difference is the .git in the repository path on github

+$ curl -sSf https://github.com/graywolf/foo | grep -A1 go-import
  <meta name="go-import" content="github.com/graywolf/foo git https://github.com/graywolf/foo.git">

+$ curl -sSf https://git.sr.ht/~graywolf/foo | grep -A1 go-import
<meta name="go-import"
  content="git.sr.ht/~graywolf/foo git https://git.sr.ht/~graywolf/foo">

Actually, when I try to import it as foo.git in "kinda" works even from the source hut

+$ cat sr2.go
package main

import (
        "fmt"
        "git.sr.ht/~graywolf/foo.git/bar"
)

func main() {
        fmt.Println(bar.Bar())
}
+$ go run sr2.go
go: finding git.sr.ht/~graywolf/foo.git latest
go: finding git.sr.ht/~graywolf/foo.git/bar latest
go: git.sr.ht/~graywolf/foo.git/bar: git.sr.ht/~graywolf/foo.git/bar@v0.0.0-20191019153505-33a4721605aa: parsing go.mod:
        module declares its path as: git.sr.ht/~graywolf/foo/bar
                but was required as: git.sr.ht/~graywolf/foo.git/bar

What documentation were you using when setting up the server?

I do not know, it's a hosted service not setup by me.

/cc @ddevault

@ddevault
Copy link

See this issue:

#32260

The fix has landed for the next Go release.

@ddevault
Copy link

Err, no, that was cgo-specific. This may be a git.sr.ht bug.

@bcmills
Copy link
Contributor

bcmills commented Oct 21, 2019

But github does not provide that either

GitHub is (unfortunately) a hard-coded special case at the moment, so that's probably why it works there. (But that means that GitHub is not a good example to follow when constructing your own go-import tags.)

See also #26134.

// Github
{
prefix: "github.com/",
regexp: lazyregexp.New(`^(?P<root>github\.com/[A-Za-z0-9_.\-]+/[A-Za-z0-9_.\-]+)(/[\p{L}0-9_.\-]+)*$`),
vcs: "git",
repo: "https://{root}",
check: noVCSSuffix,
},

@graywolf
Copy link
Author

It's really shame that biggest open source hosting is special cased so it cannot be used for comparing with other implementations. Are there any plans to de-hard-code them?

@ddevault
Copy link

It's also a shame that Golang has to be special cased for every repository hosting platform. I'd really rather have a more general solution than adding Go-specific meta tags to each repo.

@bcmills
Copy link
Contributor

bcmills commented Oct 21, 2019

@ddevault, per https://golang.org/cmd/go/#hdr-Remote_import_paths, you can always use a .git path suffix in lieu of the go-import tag. (The go-import tag is only needed to tell the go command where the repo root is within the path and which VCS to use, and the suffix communicates both of those.)

@bcmills
Copy link
Contributor

bcmills commented Oct 21, 2019

@graywolf we have no plan to remove the hard-coded paths at this time. Presumably as a first step we would have to get upstream fixes for them to implement the full go-import protocol. 😅

@leafrhythms
Copy link

leafrhythms commented Dec 21, 2019

Not about the error message or go get mechanisms, but sr.ht fixed its implementation of remote import protocol on Dec 19.
https://git.sr.ht/~sircmpwn/git.sr.ht/commit/2bdb70d928f1e41e1d4fa1e15e24246f0c37873a

@ddevault
Copy link

"Fixed". I still think it's really silly that a git repo has to have language-specific meta tags. There really ought to be a more universal standard.

@dimpase
Copy link

dimpase commented Nov 26, 2020

@ddevault complains he got blocked on this repo. Is it some kind of ban done by a bot?

@101arrowz
Copy link

It's quite strange to me that Go has given specific exceptions to the go-import meta tag requirement for popular package repositories. I found this issue when I discovered that GitHub still implements go-import wrong for subdirectories.

At the very least this behavior should be documented - at the moment the docs seem to say that GitHub's go-import tags are actually used by go get:

GitHub and other popular hosting services respond to ?go-get=1 queries for all repositories, so usually no server configuration is necessary for modules hosted at those sites.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Documentation modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

6 participants