Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: get $VANITYDOMAIN is flaky #9357

Closed
speter opened this issue Dec 16, 2014 · 10 comments
Closed

cmd/go: get $VANITYDOMAIN is flaky #9357

speter opened this issue Dec 16, 2014 · 10 comments
Milestone

Comments

@speter
Copy link

speter commented Dec 16, 2014

go get $VANITYDOMAIN doesn't work (doesn't even do any network request):

user@host:~$ GOPATH=`mktemp -d` go get -x gogetplayground1.github.io
package gogetplayground1.github.io: unrecognized import path "gogetplayground1.github.io"
user@host:~$ 

On the other hand, go get $VANITYDOMAIN/$PKG works and the sub-package can import the package at the root of $VANITYDOMAIN that was got together with it as well:

user@host:~$ GOPATH=`mktemp -d` go get -x gogetplayground1.github.io/sub
cd .
git clone https://github.com/gogetplayground1/gogetplayground1.github.io /tmp/tmp.gVMaZncb1g/src/gogetplayground1.github.io
cd /tmp/tmp.gVMaZncb1g/src/gogetplayground1.github.io
git show-ref
cd /tmp/tmp.gVMaZncb1g/src/gogetplayground1.github.io
git checkout master
WORK=/tmp/go-build238190946
mkdir -p $WORK/gogetplayground1.github.io/_obj/
mkdir -p $WORK/
cd /tmp/tmp.gVMaZncb1g/src/gogetplayground1.github.io
/opt/pkg/go/pkg/tool/linux_amd64/6g -o $WORK/gogetplayground1.github.io.a -trimpath $WORK -p gogetplayground1.github.io -complete -D _/tmp/tmp.gVMaZncb1g/src/gogetplayground1.github.io -I $WORK -pack ./root.go
mkdir -p /tmp/tmp.gVMaZncb1g/pkg/linux_amd64/
mv $WORK/gogetplayground1.github.io.a /tmp/tmp.gVMaZncb1g/pkg/linux_amd64/gogetplayground1.github.io.a
mkdir -p $WORK/gogetplayground1.github.io/sub/_obj/
mkdir -p $WORK/gogetplayground1.github.io/
cd /tmp/tmp.gVMaZncb1g/src/gogetplayground1.github.io/sub
/opt/pkg/go/pkg/tool/linux_amd64/6g -o $WORK/gogetplayground1.github.io/sub.a -trimpath $WORK -p gogetplayground1.github.io/sub -complete -D _/tmp/tmp.gVMaZncb1g/src/gogetplayground1.github.io/sub -I $WORK -I /tmp/tmp.gVMaZncb1g/pkg/linux_amd64 -pack ./sub.go
mkdir -p /tmp/tmp.gVMaZncb1g/pkg/linux_amd64/gogetplayground1.github.io/
mv $WORK/gogetplayground1.github.io/sub.a /tmp/tmp.gVMaZncb1g/pkg/linux_amd64/gogetplayground1.github.io/sub.a
user@host:~$ 

It seems that the implementation doesn't make a request unless the cleaned import path contains a '/', but the current behavior is clearly inconsistent, so I believe it should also get the repo when using the first command (some people may be relying on the behavior of the second command already -- even unknowingly).

@speter
Copy link
Author

speter commented Dec 16, 2014

On a side note, I intend to move (copy) the repro to a permanent location that includes the issue ID, but I'm not sure if it's better to update the original comment with the final URLs, or just post them in a new comment (or it's better to just leave the repro where it is, or it's not even necessary to keep it after the issue is resolved).

@davecheney
Copy link
Contributor

I don't think this is a bug, excluding vanity domains for a second, would
you expect

go get github.com

To work ? I don't think it should, and for the same reason that

go get dfc.io

Would also not work.
On 17 Dec 2014 09:51, "speter" notifications@github.com wrote:

On a side note, I intend to move (copy) the repro to a permanent location
that includes the issue ID, but I'm not sure if it's better to update the
original comment with the final URLs, or just post them in a new comment
(or it's better to just leave the repro where it is, or it's not even
necessary to keep it after the issue is resolved).


Reply to this email directly or view it on GitHub
#9357 (comment).

@adg
Copy link
Contributor

adg commented Dec 16, 2014

go get is for fetching packages, bare domains. I believe this is working as intended.

@speter
Copy link
Author

speter commented Dec 16, 2014

@davecheney
I never said I expeced it to work. I said it's flaky / inconsistent: it fails in one example and works in another. I think it should be consistent. And if one is to be changed, the more conservative way is to make it work in both cases.
BTW I don't think it's unreasonable to expect it to work: go get generally allows repos to be prefixed at any level including root (not on github of course), and repos can contain packages at any level including root, so disallowing having a package in a repo root, only in the case when it happens to be prefixed at a domain root, can be seen an odd exclusion. But I don't mind either way as long as it's consistent.

@adg
By working as intended, do you mean I can rely on it to continue to work in the future in the cases when it works with Go1.4? (The second example above, "$DOMAIN" imported via "$DOMAIN/$SUBPKG"?)

@davecheney
Copy link
Contributor

@speter. I don't think it is inconsistent, go get has never allowed you to go get a bare domain, there has always been a requirement to to have at least one path element. So

dfc.io

Is not a package that go get knows how to get, while

dfc.io/quux

is a package that go get knows how to get. Given the way the go get infers URLs from import statements, this seems like a reasonable trade off.

@adg
Copy link
Contributor

adg commented Dec 17, 2014

@speter I misread your original report. I agree this inconsistency is weird. It's also problematic because if package foo imports the package with the import path "example.com", go get of that package foo will fail to fetch "example.com", even though go get example.com/... might work.

@adg adg reopened this Dec 17, 2014
@speter
Copy link
Author

speter commented Dec 21, 2014

Thank you for taking the time to look at it (again). Sorry, my initial report could have been organized/worded better, and it included too much low-level technical details with little background explanation. I've thought about it a bit more and hopefully managed to summarize the issue(s) better.

TL;DR version:

  1. The documentation about acceptable import paths should be made clearer.
  2. go get $PRJDOMAIN/... should work with a "flat" package structure (import paths like "$PRJDOMAIN/$PKG").
  3. Disallowing import "$PRJDOMAIN" (or perhaps import "$PRJDOMAIN/") seems odd, unnecessary, and hard to implement consistently.

Detailed version:

I was thinking in the context of a domain specifically for a single project (as opposed to github or user domains that host multiple projects/repos, thus implying a hierarchy in the package path). I wanted to enumerate the potential layouts I should/could consider, and started playing with this because it wasn't obvious from the docs (go help importpath) whether hosting a "primary" package at the domain top was (intended to be) supported. So an important part of the issue probably is that documentation regarding acceptable import paths should be made clearer. (Actually even if import "$PRJDOMAIN" was supported, for the current project I'm inclined to use a "flat" collection of packages like "$PRJDOMAIN/$PKG".)

@adg 's example go get $PRJDOMAIN/... is great because it covers both the top level and subdirectories in a single command. So if the top level is prohibited as an import path, it seems somewhat ambiguous what the "correct" semantics should be for this command. But as a pragmatic consideration, at least if a project only uses packages in subdirectories like "$PRJDOMAIN/$PKG", it seems useful and reasonable to expect that users can run go get $PRJDOMAIN/... to get and build all of them -- which it currently doesn't:

user@host:~$ GOPATH=`mktemp -d` go get -x goget9357topellipsisflat.github.io/...
package goget9357topellipsisflat.github.io/...: unrecognized import path "goget9357topellipsisflat.github.io/..."
user@host:~$ 

The same works when the prefix is a subdirectory:

user@host:~$ GOPATH=`mktemp -d` go get -x goget9357direllipsisflat.github.io/dir/...                  
cd .
git clone https://github.com/goget9357direllipsisflat/goget9357direllipsisflat.github.io /tmp/tmp.YFSP4w7RLp/src/goget9357direllipsisflat.github.io/dir
cd /tmp/tmp.YFSP4w7RLp/src/goget9357direllipsisflat.github.io/dir
git show-ref
cd /tmp/tmp.YFSP4w7RLp/src/goget9357direllipsisflat.github.io/dir
git checkout master
WORK=/tmp/go-build057321063
mkdir -p $WORK/goget9357direllipsisflat.github.io/dir/a/_obj/
mkdir -p $WORK/goget9357direllipsisflat.github.io/dir/
cd /tmp/tmp.YFSP4w7RLp/src/goget9357direllipsisflat.github.io/dir/a
/opt/pkg/go/pkg/tool/linux_amd64/6g -o $WORK/goget9357direllipsisflat.github.io/dir/a.a -trimpath $WORK -p goget9357direllipsisflat.github.io/dir/a -complete -D _/tmp/tmp.YFSP4w7RLp/src/goget9357direllipsisflat.github.io/dir/a -I $WORK -pack ./a.go
mkdir -p $WORK/goget9357direllipsisflat.github.io/dir/b/_obj/
cd /tmp/tmp.YFSP4w7RLp/src/goget9357direllipsisflat.github.io/dir/b
/opt/pkg/go/pkg/tool/linux_amd64/6g -o $WORK/goget9357direllipsisflat.github.io/dir/b.a -trimpath $WORK -p goget9357direllipsisflat.github.io/dir/b -complete -D _/tmp/tmp.YFSP4w7RLp/src/goget9357direllipsisflat.github.io/dir/b -I $WORK -pack ./b.go
mkdir -p /tmp/tmp.YFSP4w7RLp/pkg/linux_amd64/goget9357direllipsisflat.github.io/dir/
mv $WORK/goget9357direllipsisflat.github.io/dir/a.a /tmp/tmp.YFSP4w7RLp/pkg/linux_amd64/goget9357direllipsisflat.github.io/dir/a.a
mv $WORK/goget9357direllipsisflat.github.io/dir/b.a /tmp/tmp.YFSP4w7RLp/pkg/linux_amd64/goget9357direllipsisflat.github.io/dir/b.a
user@host:~$ 

Then, for both the transitive import example in the initial report, and in case we want to allow go get $PRJDOMAIN/..., it seems quite hard to me to consistently disallow using "$PRJDOMAIN" as an import path. I also don't see any compelling technical or conceptual reason for disallowing it. (If not having a slash in remote import paths is deemed technologically problematic or confusing, "$PRJDOMAIN/" can be another option for indicating the top level.) To be honest, I was quite surprised by the initial negative response just based on having mentioned the idea of using the top level of a domain as an import path. Internet infrastructure has evolved greatly so that instead of URLs like http://www2.companyname.com.cctld/companyname/pcweb/products/$THETHINGYOUARELOOKINGFOR/$THETHINGYOUARELOOKINGFOR.htm we can often just have http://$THETHINGYOUARELOOKINGFOR/ or at least something much-much closer. Given this trend in URLs, it could even be argued that disallowing the top level is contrary to the spirit of Go's choice to embrace URLs as import paths.

Consider for example if a package like fsnotify was called "fileevents", and hosted at, say, https://file.events/. Which of import "file.events" and import "file.events/fileevents" would you consider more idiomatic? To me the former seems better, both based on the evolution of URL schemes, as well as the preference to avoid stutter with naming things in Go. (Ultimately I believe both aim at the same thing; enhancing signal-to-noise ratio.)

Even just thinking about how such a limitation could be documented (in a way that is useful for package publishers) makes it sound very odd to me: "You can put packages in any directory of your repo, except if you want to host it at the top level of your domain, in which case you cannot have a package in the repo root." Or, "You can host your repository at any level of your domain, except if it has a package in the root directory, in which case you must use a subdirectory of your domain."

@speter
Copy link
Author

speter commented Dec 26, 2014

For reference, Google's Search Engine Optimization Starter Guide ("Use words in URLs", page 9) directs to

Avoid

  • using lengthy URLs with unnecessary parameters and session IDs
  • choosing generic page names like "page1.html"
  • using excessive keywords like "baseball-cards-baseball-cards-baseballcards.htm"

This seems to support my observation that requiring an extra directory level when the domain name already conveys the package's name/purpose works against present day URL design best practices. (I believe that in the above example comparing "file.events" vs "file.events/fileevents", the repeated "fileevents" would qualify as an excessive keyword, and using a different directory name such as "file.events/pkg" would also be undesirable.)

@rsc rsc added this to the Go1.5Maybe milestone Apr 10, 2015
@rsc rsc removed the repo-main label Apr 14, 2015
@rsc rsc modified the milestones: Go1.5, Go1.5Maybe Jun 29, 2015
@gopherbot
Copy link

CL https://golang.org/cl/12193 mentions this issue.

@gopherbot
Copy link

CL https://golang.org/cl/18152 mentions this issue.

rsc pushed a commit to golang/tools that referenced this issue Dec 29, 2015
This is the same change as in https://golang.org/cl/12193.

Fixes golang/go#13506.

Related to golang/go#9357.

Change-Id: I9c7d956008641b1907e14bcb08198235f5f9552f
Reviewed-on: https://go-review.googlesource.com/18152
Reviewed-by: Russ Cox <rsc@golang.org>
@golang golang locked and limited conversation to collaborators Dec 29, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants