Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: allow "go get" when outside a module in module mode #24250

Closed
smyrman opened this issue Mar 5, 2018 · 45 comments
Closed

cmd/go: allow "go get" when outside a module in module mode #24250

smyrman opened this issue Mar 5, 2018 · 45 comments
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Milestone

Comments

@smyrman
Copy link

smyrman commented Mar 5, 2018

UPDATE: This is a feature request to be able to install a program/main package to a user's home directory (basically $GOBIN) outside of a module context.

Background

While dep, glide and other tools provide a way of fetching project dependencies, part of what they are missing which I think belongs in any official go get with versioning, is a way to install programs (tools, CLI, or a binary compiled from any main package) into a user's PATH (e.g. GOBIN, which may be relevant, even if GOPATH get's deprecated).


Please answer these questions before submitting your issue. Thanks!

What did you do?

$ vgo get github.com/golang/dep/cmd/dep@v0.4.1

Example command only; but this would be the closest to how things currently work with go get.

What did you expect to see?

dep binary (in this case) installed to either $GOBIN, $GOPATH[0]/bin or $HOME/go/bin, in that order.

What did you see instead?

$ vgo get github.com/golang/dep/cmd/dep@v0.4.1
cannot determine module root; please create a go.mod file there

System details

go version go1.10 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/smyrman/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/smyrman/.local"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.10/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.10/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/sj/d5g1dfkd3g75ccz83k505v3r0000gn/T/go-build848907380=/tmp/go-build -gno-record-gcc-switches -fno-common"
VGOMODROOT=""
GOROOT/bin/go version: go version go1.10 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.10
uname -v: Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.13.3
BuildVersion:	17D102
lldb --version: lldb-900.0.64
  Swift-4.0
@gopherbot gopherbot added this to the vgo milestone Mar 5, 2018
@smyrman smyrman changed the title x/vgo: feature: Allow installing _programs_ at specific versions x/vgo: feature: Allow installing programs at specific versions Mar 5, 2018
@smyrman smyrman changed the title x/vgo: feature: Allow installing programs at specific versions x/vgo: feature: allow installing programs at specific versions Mar 5, 2018
@kardianos
Copy link
Contributor

@smyrman I think this is a genuine concern, but it isn't just about pulling down a specific version of a program; it is using vgo outside the context of a module. You'll get the same error if you omit the versuib @v0.0.0 specifier as well. vgo thinks you are trying to pull in a dependency; you are trying to install a program into GOBIN.

Maybe:
x/vgo: no longer supports installing an executable from outside of a module
?

@smyrman smyrman changed the title x/vgo: feature: allow installing programs at specific versions x/vgo: feature: allow installing programs (at specific versions) Mar 5, 2018
@smyrman
Copy link
Author

smyrman commented Mar 5, 2018

You'll get the same error if you omit the versuib @v0.0.0 specifier as well. vgo thinks you are trying to pull in a dependency; you are trying to install a program into GOBIN.

@kardianos, thanks for the response. I am actually not concerned about the error message being unclear. This is meant as a feature request to (somehow) still be able to install binaries to a user's PATH, no matter what your current working directory is, similar to what we can with go get today. Even better, the possibility to install a program at as specific version. I believe a language package manger should not be just about dependencies.

I am not to particular about the syntax, but maybe there could be a flag like -home, -user or even -gopath, that allowed installing stuff to either $GOBIN or $GOPATH/bin. This would be similar to how e.g. pip works, for Python, to mention one, where if you use e.g. venev or virtualenv for a per-project experience, you can still add an option to pip to install tools to a user's home:

While OS package managers are good at doing more or less the same thing (not to a user's home of course, but system wide), there is often a delay for programs to be packaged, and language power users may prefer to use a language package manger for installing his/hers software, while also being able to pull down unreleased software, as we do today with go get. Users operating on a shared system may have a similar need to install things in their home directories.

Further, even though vgo deprecates GOPATH/GOBIN for project development, it's not necessarily right to deprecate GOPATH for other potentially valid use-cases, but that's probably another discussion, where the only thing I am interested in through this issue, is the GOBIN part.

@smyrman smyrman changed the title x/vgo: feature: allow installing programs (at specific versions) x/vgo: feature: allow installing programs (at specific versions) to GOBIN Mar 5, 2018
@smyrman
Copy link
Author

smyrman commented Mar 5, 2018

PS! I am also not concerned about this issue being solved right away, but it would be really cool to solve it before vgo goes main-line, if that's what going to happen.

@smyrman
Copy link
Author

smyrman commented Mar 5, 2018

Updated the description.

@rsc
Copy link
Contributor

rsc commented Mar 30, 2018

$ vgo get github.com/golang/dep/cmd/dep@v0.4.1
cannot determine module root; please create a go.mod file there

This clearly must work eventually. The thing I'm not sure about is exactly what this does as far as the version is concerned: does it create a temporary module root and go.mod, do the install, and then throw it away? Probably. But I'm not completely sure, and for now I didn't want to confuse people by making vgo do things outside go.mod trees. Certainly the eventual go command integration has to support this.

Note that installs already happen to $GOBIN or else $GOPATH/bin (so default $HOME/go/bin), not the modue tree.

Will mark NeedsDecision because we still need to work out how to do this, but yes absolutely the command must work.

@rsc rsc added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Mar 30, 2018
@smyrman
Copy link
Author

smyrman commented Jun 7, 2018

does it create a temporary module root and go.mod, do the install, and then throw it away? Probably.

This is a reasonable approach, and have some obvious benefits, such as the result of go get github.com/golang/dep/cmd/dep@v0.4.1 not being affected by other tools installed on the system. To make a decision easier, I will supply an alternative approach and highlight some problems with it:

I suppose an alternative would be to have a "default" module that is used when a flag is provided to the go get command.

E.g. one could picture a default module in $GOPATH[0]/src/u (first entry of GOPATH) that is automatically created and used when the -user flag is given to the go get command line. This would let go get -user github.com/golang/dep/cmd/dep@v0.4.1 be semantically equivalent to:

#!/bin/sh
set -e
_module_path=$(echo $GOPATH | awk -F':' '{ print $1}')/src/u
mkdir -p ${_module_path}
cd  ${_module_path}
test -f go.mod || echo 'module "u"' > go.mod
go get  github.com/golang/dep/cmd/dep@v0.4.1

Some benefits

  • One easy flag to access a user-specific module for user-installed cross-project binaries.
  • The module path is preserved on disk, so power users could sync the folder via tools like Git or Dropbox to keep their tools with them at specific versions across machines.

Major drawback

  • Tool installation (e.g. installation of dep at a specific version as used in the example above), may be affected by (minimal) version selection of overlapping dependencies with other tools, leading to potentially weird bugs.

Conclusion

For me, I belive the drawback outweighs any benefit, and a temporary module root / mod.go root is the semantically correct solution; perhaps implementation wise the temporary module root and go.mod file only exists in memory.

@rsc rsc modified the milestones: vgo, Go1.11 Jul 12, 2018
@rsc rsc added the modules label Jul 12, 2018
@rsc rsc changed the title x/vgo: feature: allow installing programs (at specific versions) to GOBIN cmd/go: feature: allow installing programs (at specific versions) to GOBIN Jul 12, 2018
@rsc rsc changed the title cmd/go: feature: allow installing programs (at specific versions) to GOBIN cmd/go: allow installing programs (at specific versions) to GOBIN Jul 17, 2018
@rsc
Copy link
Contributor

rsc commented Aug 9, 2018

This isn't blocking Go 1.11. The main reason being that if GO111MODULE=auto (the default) then we can't reasonably do anything with modules when outside a go.mod tree. That's the opt-in. And inside a go.mod tree this already works.

@rsc rsc modified the milestones: Go1.11, Go1.12 Aug 9, 2018
@earthboundkid
Copy link
Contributor

I agree that this doesn't block Go 1.11, but I do think it should be a blocker for 1.12. There are a lot of repos that have READMEs saying "just run go get github.com/me/myrepo to install". That should continue to work once GO111MODULE changes to "on".

@bcmills
Copy link
Contributor

bcmills commented Aug 10, 2018

Agreed. I've marked it as a release-blocker for 1.12.

@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Nov 28, 2018
@gopherbot gopherbot removed the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Nov 28, 2018
Arista-Jenkins pushed a commit to aristanetworks/goarista that referenced this issue Jan 11, 2019
The fork is pinned at v1.6.5 which is the last release before
influxdata/influxdb@f2898d1.
The commit is part of the larger effort in
influxdata/influxdb#10618 which makes a
breaking change.

We can't handle this with Go modules because of
golang/protobuf#751, which in turn depends on
golang/go#24250 which will be fixed with
go1.12.

Change-Id: I43b643fcab202d94bd529dfce135fb4e3f5add52
@nhooyr
Copy link
Contributor

nhooyr commented Feb 26, 2019

@bcmills

Regarding @cespare's comment

I'm not sure I totally follow the entire lengthy discussion here, but it sounds like even with this change we would still need to do cd /somewhere/outside/of/a/module && go get blah.

This seems to be a major missing feature. Its nice things work concurrently now but sometimes you're in your project directory and want to download a tool. What should I do in this case? Just cd to a tmp dir?

@nhooyr
Copy link
Contributor

nhooyr commented Feb 26, 2019

Nvm.

To anyone wondering about my question in the future: https://github.com/go-modules-by-example/index/blob/master/010_tools/README.md

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Projects
None yet
Development

No branches or pull requests