Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: allow top-level package outside $GOPATH #3009

Closed
evmar opened this issue Feb 13, 2012 · 9 comments
Closed

cmd/go: allow top-level package outside $GOPATH #3009

evmar opened this issue Feb 13, 2012 · 9 comments
Milestone

Comments

@evmar
Copy link

evmar commented Feb 13, 2012

Suppose I wanted to check some Go code into an existing source tree.  (To help make the
example concrete, agl wrote a tool in Go to extract some data out of some table in some
other file in the Chrome tree.)

It appears that "go build" is designed only for projects that are written
solely in Go, as there's not a obvious place to point GOROOT etc.

We can instead write out build files using "go tool 6g" etc. to produce
binaries, but it appears that "go test" also relies on being in a Go-only
source tree.  Is there an alternative approach for testing we should take?   (For
example, it seems the "testing" standard library also is intertwined with
"go test" behavior.)
@evmar
Copy link
Author

evmar commented Feb 13, 2012

Comment 1:

(PS: lest I scare off Adam, I don't intend to check in his particular program.  The
example just helps describe the general problem I've run into.)

@rsc
Copy link
Contributor

rsc commented Feb 14, 2012

Comment 2:

I would like to have an answer for you.  Are you producing standalone
binaries from a single directory, or do you have import paths referring
to other directories?  The main reason for $GOPATH is the latter case,
so that when you see an import you know where to look for the 
imported code.  For standalone single-directory programs we can
relax the rules and let the go build / go test work outside $GOPATH,
and I plan to do that.
That is, code you import must be in a $GOPATH (otherwise how can
we find it) but the top-level package (usually package main) need not be.
I plan to remove the restriction about the top-level package being
in $GOPATH.
Does this address the case you have in mind?  If not, can you give
more details about how the go tool would resolve import paths in
the case you have in mind?

Labels changed: added priority-go1, go1-must, removed priority-triage.

Owner changed to builder@golang.org.

Status changed to Accepted.

@evmar
Copy link
Author

evmar commented Feb 14, 2012

Comment 3:

In the particular case I was worrying about, I think just allowing the top-level package
be outside of GOPATH would be fine.  (This is all kinda hypothetical as I've just been
trying to evaluate whether Go will work for me for various projects.)
However, continuing the hypothetical, it does seem plausible to me that we'd eventually
write some bits of shared Go code.  To make the example concrete, say we have tools that
scrape our buildbot outputs and generate various reports; maybe we'd have some sort of
"buildbot log parser" module that would be shared by various programs that would all
live in the Chrome tree under tools/ somewhere.
If refactoring out shared code meant that that we'd need to switch to a tree layout
matching the expected Go layout, then there's little point in special-casing just the
top-level package.
(I'm not sure how "project comprised of multiple modules" is intended to be addressed in
Go, even without the problem this bug was created about: like if my project internally
has a "scraper" module but that isn't useful for top-level module namespace, so it
doesn't make any sense to "install" such a module.  Sorry for all the confusion in my
report, I just really don't get how it's all intended to work.)

@evmar
Copy link
Author

evmar commented Feb 14, 2012

Comment 4:

Oh, and to answer your last question, as to how it could work: I dunno.  My
understanding so far is that, given that our tree has code other than Go code in it,
we're supposed to be using "go tool 6g" etc., that means we'll need our own build rules
for building everything which means we'll be managing our own include paths etc.
manually.  Which is fine enough, except for that I don't know how to make the Go
"testing" package work in such a world.

@gopherbot
Copy link

Comment 5 by yiyu.jgl:

About how import paths could be resolved, I think all the binary files could be stored
inside GOPATH, in a known location related to the local path.
A new option would be needed:
    -l localpath   act like if localpath were $GOPATH/src/localpath
So, for example, I have the directories /home/user/project1/{app1,app2,pkg1,pkg1/pkg2}.
Then, running:
    $ cd /home/user/project1
    $ go install -p . ./...
would install the files:
    $GOPATH/
      bin/
        app1
        app2
      pkg/
        linux_amd64/
          local/
            home/user/project1/
              pkg1.a
              pkg1/
                pkg2.a
When the go tool runs the compiler, the directory
$GOPATH/pkg/linux_amd64/local/home/user/project1 is added to the import path, so
packages imported with base in localpath are found without problems.
Certain details could be different, for example packages could go to
$GOPATH/pkg/local/$GOOS_$GOARCH, so that they don't get mixed with goinstalled packages,
but I think is cool knowing that I can import local packages with "local/path/to/pkg".
An alternative to storing binaries in GOPATH would be to leave them in place, or in
localpath/pkg. What is the problem with these solutions?

@gopherbot
Copy link

Comment 6 by yiyu.jgl:

By the way, an alternative to using go tool 6g or moving your files to GOPATH is to have
a GOPATH for your project. So, for example, set your GOPATH to
$GOPATH:$CRHOMIUMPATH/tools/go and then keep all your packages in tools/go/src.

@evmar
Copy link
Author

evmar commented Feb 16, 2012

Comment 7:

When I first brought this up (on Google+ inside Google), I listed a number of options
(including the one where I make a separate tree for go tools).  The downside is that it
results in parallel trees for go code and all the other code; for example, we put
networking code under net/ and build tools under tools/ and if we went this route we'd
need go/src/net/ and go/src/tools/.  (And even then, the package names would maybe
conflict with the real go package names when you "install" them?)
This all seems so wrong I figured I must be missing something important, which is why I
posted on G+ instead of filing a bug.  I came here once Ian suggested that the "go tool
6g" route was my best bet and that I should start a bug on the remaining issues.

@gopherbot
Copy link

Comment 8 by yiyu.jgl:

You are right, in that situation a dedicated GOPATH does not seem the best solution. (If
you took that route, your packages import paths should be of the form "chromium/net",
etc. to avoid conflicts, but I understand you may want to keep all the net code and all
the tools together).
Another option would be to use symlinks, once the relevant issue is solved. Then, if
your tree structure is something like:
  chromium/
    net/
      gonetpkg
    tools/
      gotool1
and you run:
  ln -s chromium $GOPATH/chromium
you could import gonetpkg from gotool1 as "chromium/net/gonetpkg".
The general solution I proposed in comment 5 would work too. You'd just need to use -l
chromium/..

@rsc
Copy link
Contributor

rsc commented Mar 1, 2012

Comment 9:

This issue was closed by revision b03a5f6.

Status changed to Fixed.

@rsc rsc added this to the Go1 milestone Apr 10, 2015
@golang golang locked and limited conversation to collaborators Jun 24, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants