cmd/go: main_test must depend on main.a, not main binary #3417

rogpeppe · 2012-03-28T13:31:21Z

I don't know how to reproduce this from scratch, but here are the symptoms I'm seeing
currently. I've seen the issue a couple of times now.

The problem goes away, as you can see, when I touch all the go files. It doesn't make a
difference if I import as the full path name or ".".

% go clean -r -i
% go test -i
% go test
# launchpad.net/juju/go/cmd/juju_test
./cmd_test.go:7: can't find import: "launchpad.net/juju/go/cmd/juju"
FAIL    launchpad.net/juju/go/cmd/juju [build failed]
% echo $status
2
% go list .../cmd/juju
launchpad.net/juju/go/cmd/juju
% # change source to use package that's genuinely not there.
% go test
# launchpad.net/juju/go/cmd/juju
cmd_test.go:7:2: import "launchpad.net/juju/go/cmd/jujux": cannot find package
% echo $status
1
% touch *.go
% go test
# launchpad.net/juju/go/cmd/juju_test
./cmd_test.go:76: undefined: allowsExtraArgs
FAIL    launchpad.net/juju/go/cmd/juju [build failed]
% echo $status
2
%

rsc · 2012-04-02T13:54:16Z

Comment 1:

If you can still make this happen, please try -v -x to see if you get any useful extra
output.
If all the code is inside one $GOPATH tree then I don't understand what's going on.
Also interesting might be plain 'go list -e -json' (no import path args).

Status changed to WaitingForReply.

dsymonds · 2012-04-04T06:36:21Z

Comment 2:

Labels changed: added priority-later, removed priority-triage.

rogpeppe · 2012-04-25T17:06:41Z

Comment 3:

I just encountered this problem again. Here's the output of go test -v -x:
% go test -v -x
WORK=/tmp/go-build386097667
mkdir -p $WORK/launchpad.net/juju/go/cmd/juju/_test/launchpad.net/juju/go/cmd/
mkdir -p $WORK/launchpad.net/juju/go/cmd/juju/_test/
cd /home/rog/src/go/src/launchpad.net/juju/go/cmd/juju
/home/rog/go/pkg/tool/linux_amd64/6g -o
$WORK/launchpad.net/juju/go/cmd/juju/_test/_go_.6 -p launchpad.net/juju/go/cmd/juju_test
-D _/home/rog/src/go/src/launchpad.net/juju/go/cmd/juju -I $WORK -I
/home/rog/src/go/pkg/linux_amd64 ./cmd_test.go ./main_test.go
# launchpad.net/juju/go/cmd/juju_test
./cmd_test.go:7: can't find import: "launchpad.net/juju/go/cmd/juju"
$WORK/launchpad.net/juju/go/cmd/juju/_test/juju.test -test.v=true
FAIL    launchpad.net/juju/go/cmd/juju [build failed]
% 
i've archived my $GOPATH directory in the hope that i might be able
to reproduce the problem later.

rsc · 2012-05-02T01:51:13Z

Comment 4:

The presence of the -D_... option suggests that the go command
believes you are outside the go root and any GOPATH directories.
Instead of go test -v -x what does 'go list -json' say?  Specifically,
what does the Root field say?  If it is "", that confirms my theory.
And yet the output line beginning with # makes clear that at least
part of the go command believes that you are in a directory with a
root.
Is it possible that there is a stale juju directory tree earlier in
your GOPATH or in the GOROOT itself?  In that situation the go command
does try to avoid believing that the current directory might be the
canonical juju package (since it is shadowed by that other one).

rogpeppe · 2012-05-02T08:02:52Z

Comment 5:

unfortunately, and possibly significantly, unarchiving $GOPATH (inc. restoring mtimes)
did not allow me to reproduce the issue. i should probably have archived $GOROOT too.
i'll have to wait until i see the issue again. let's hope it's magically gone away
forever.
it's highly unlikely that the was a stale juju directory tree earlier in GOPATH or
GOROOT. (i use a single directory in GOPATH, and there's nothing in GOROOT now and i
haven't removed anything since then). if there *was*, then it seems to me that the issue
would probably not be resolved by touching the files in the package directory, which it
was; another indication that that's probably not the case.

rogpeppe · 2012-05-16T13:30:25Z

Comment 6:

the issue just came up again. same directory, exactly the same symptoms as before.
theory not validated, i'm afraid - Root is not empty.
here's the output of go list -json.
{
    "Dir": "/home/rog/src/go/src/launchpad.net/juju/go/cmd/juju",
    "ImportPath": "launchpad.net/juju/go/cmd/juju",
    "Name": "main",
    "Target": "/home/rog/src/go/bin/juju",
    "Root": "/home/rog/src/go",
    "GoFiles": [
        "bootstrap.go",
        "destroy.go",
        "main.go"
    ],
    "Imports": [
        "launchpad.net/gnuflag",
        "launchpad.net/juju/go/cmd",
        "launchpad.net/juju/go/environs/ec2",
        "launchpad.net/juju/go/juju",
        "os"
    ],
    "Deps": [
        "archive/tar",
        "archive/zip",
        "bufio",
        "bytes",
        "compress/flate",
        "compress/gzip",
        "crypto",
        "crypto/aes",
        "crypto/cipher",
        "crypto/des",
        "crypto/dsa",
        "crypto/elliptic",
        "crypto/hmac",
        "crypto/md5",
        "crypto/rand",
        "crypto/rc4",
        "crypto/rsa",
        "crypto/sha1",
        "crypto/sha256",
        "crypto/subtle",
        "crypto/tls",
        "crypto/x509",
        "crypto/x509/pkix",
        "encoding/asn1",
        "encoding/base64",
        "encoding/binary",
        "encoding/hex",
        "encoding/pem",
        "encoding/xml",
        "errors",
        "fmt",
        "hash",
        "hash/crc32",
        "io",
        "io/ioutil",
        "launchpad.net/gnuflag",
        "launchpad.net/goamz/aws",
        "launchpad.net/goamz/ec2",
        "launchpad.net/goamz/s3",
        "launchpad.net/goyaml",
        "launchpad.net/gozk/zookeeper",
        "launchpad.net/juju/go/charm",
        "launchpad.net/juju/go/cloudinit",
        "launchpad.net/juju/go/cmd",
        "launchpad.net/juju/go/environs",
        "launchpad.net/juju/go/environs/ec2",
        "launchpad.net/juju/go/juju",
        "launchpad.net/juju/go/log",
        "launchpad.net/juju/go/schema",
        "launchpad.net/juju/go/state",
        "launchpad.net/juju/go/state/presence",
        "launchpad.net/juju/go/state/watcher",
        "launchpad.net/juju/go/version",
        "launchpad.net/mgo/bson",
        "launchpad.net/tomb",
        "log",
        "math",
        "math/big",
        "math/rand",
        "mime",
        "mime/multipart",
        "net",
        "net/http",
        "net/http/httputil",
        "net/textproto",
        "net/url",
        "os",
        "os/exec",
        "path",
        "path/filepath",
        "reflect",
        "regexp",
        "regexp/syntax",
        "runtime",
        "runtime/cgo",
        "runtime/debug",
        "sort",
        "strconv",
        "strings",
        "sync",
        "sync/atomic",
        "syscall",
        "time",
        "unicode",
        "unicode/utf8",
        "unsafe"
    ],
    "XTestGoFiles": [
        "cmd_test.go",
        "main_test.go"
    ],
    "XTestImports": [
        "flag",
        "fmt",
        "io/ioutil",
        "launchpad.net/gnuflag",
        "launchpad.net/gocheck",
        "launchpad.net/juju/go/cmd",
        "launchpad.net/juju/go/cmd/juju",
        "launchpad.net/juju/go/environs",
        "launchpad.net/juju/go/environs/dummy",
        "launchpad.net/juju/go/version",
        "os",
        "os/exec",
        "path/filepath",
        "reflect",
        "sort",
        "strings",
        "testing"
    ]
}
other info:
% whatis GOPATH
GOPATH=/home/rog/src/go
% whatis GOROOT
GOROOT=/home/rog/go
% whatis GOBIN
GOBIN: not found
%

rogpeppe · 2012-05-16T13:37:10Z

Comment 7:

for the record, i did rm -r $GOPATH/pkg/* and it started working again.
next time, i'll ls -l the contents first.

davecheney · 2012-05-26T07:54:51Z

Comment 8:

Roger, this happens to me if I do a hg update and ./all.bash. I think the underlying
problem is a version mismatch between the $GOROOT/pkg pkgs and $GOPATH/pkg packages
which are not being rebuilt because the source has not changed.

davecheney · 2012-05-26T07:55:33Z

Comment 9:

I should say, this happens only when testing cmds, not packages. So it might be a hole
in the logic in gotest for main packages.

davecheney · 2012-05-30T02:29:31Z

Comment 10:

I think this may be the way we're testing our main packages. Our package main tests
declare themselves package main_test. Looking at the standard library, this practice
appears to be unique. 
Adding a shim like this
lucky(~/src/launchpad.net/juju/go/cmd/jujud) % more export_test.go 
package main
// this file exists only to trick go test into creating a .a for this package.
// works around https://golang.org/issue/3417
works around the issue for me.

davecheney · 2012-06-29T11:59:25Z

Comment 11:

I think this issue should be closed. I think that we (juju) were using go test
incorrectly. Thoughts ?

rsc · 2012-09-12T21:41:15Z

Comment 12:

Labels changed: added go1.1.

rsc · 2012-12-10T04:01:50Z

Comment 13:

Labels changed: added size-m.

rsc · 2013-01-31T04:36:01Z

Comment 14:

Can anyone explain how to reproduce this? If not, let's close it.

davecheney · 2013-01-31T05:08:15Z

Comment 15:

I believe this was caused by using package main_test to test commands. Since juju moved
away from this practice, we have not seen the issue reoccur. 
I think this issue can be closed. For extra credit stopping go test from accepting
main_test as a package name would probably prevent it reoccurring.

rsc · 2013-01-31T06:57:12Z

Comment 16:

Closing the issue. I wrote a trivial main_test and it worked.

Status changed to Retracted.

rogpeppe · 2013-01-31T09:55:42Z

Comment 17:

yes, main_test seems to work... most of the time.
we avoided the issue by avoiding use of main_test, but i
think it's quite likely there's still a bug there.
*should* main_test work correctly?

rsc · 2013-01-31T15:35:16Z

Comment 18:

I'll try again. I do want to fix the bug if there's a bug.

Status changed to Accepted.

rsc · 2013-03-12T20:30:01Z

Comment 19:

Labels changed: added go1.1maybe, removed go1.1.

masiulaniec · 2013-04-05T11:29:39Z

Comment 20:

See also issue #5198.

robpike · 2013-05-18T15:04:12Z

Comment 21:

Labels changed: added go1.2maybe, removed go1.1maybe.

rsc · 2013-09-10T17:05:27Z

Comment 22:

The problem is that cmd/go does not distinguish the main.a from the generated binary.
If the binary is up to date, it does not rebuild main.a, so main_test cannot import it.
Here is a simple test case.
g% cat foo.go
package main
func F() {}
func main() {}
g% cat foo_test.go
package main_test
import "testing"
import . "foo"
func Test1(t *testing.T) {
    F()
}
g% touch foo.go
g% go test
PASS
ok      foo 0.016s
g% go install
g% go test
# foo_test
./foo_test.go:4: can't find import: "foo"
FAIL    foo [build failed]
g%

rsc · 2013-09-10T18:43:50Z

Comment 23:

This issue was closed by revision 627d17c.

Status changed to Fixed.

rogpeppe added fixed labels Sep 10, 2013

rsc added this to the Go1.2 milestone Apr 14, 2015

rsc removed the go1.2maybe label Apr 14, 2015

golang locked and limited conversation to collaborators Jun 24, 2016

gopherbot added the FrozenDueToAge label Jun 24, 2016

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: main_test must depend on main.a, not main binary #3417

cmd/go: main_test must depend on main.a, not main binary #3417

rogpeppe commented Mar 28, 2012

rsc commented Apr 2, 2012

dsymonds commented Apr 4, 2012

rogpeppe commented Apr 25, 2012

rsc commented May 2, 2012

rogpeppe commented May 2, 2012

rogpeppe commented May 16, 2012

rogpeppe commented May 16, 2012

davecheney commented May 26, 2012

davecheney commented May 26, 2012

davecheney commented May 30, 2012

davecheney commented Jun 29, 2012

rsc commented Sep 12, 2012

rsc commented Dec 10, 2012

rsc commented Jan 31, 2013

davecheney commented Jan 31, 2013

rsc commented Jan 31, 2013

rogpeppe commented Jan 31, 2013

rsc commented Jan 31, 2013

rsc commented Mar 12, 2013

masiulaniec commented Apr 5, 2013

robpike commented May 18, 2013

rsc commented Sep 10, 2013

rsc commented Sep 10, 2013

cmd/go: main_test must depend on main.a, not main binary #3417

cmd/go: main_test must depend on main.a, not main binary #3417

Comments

rogpeppe commented Mar 28, 2012

rsc commented Apr 2, 2012

dsymonds commented Apr 4, 2012

rogpeppe commented Apr 25, 2012

rsc commented May 2, 2012

rogpeppe commented May 2, 2012

rogpeppe commented May 16, 2012

rogpeppe commented May 16, 2012

davecheney commented May 26, 2012

davecheney commented May 26, 2012

davecheney commented May 30, 2012

davecheney commented Jun 29, 2012

rsc commented Sep 12, 2012

rsc commented Dec 10, 2012

rsc commented Jan 31, 2013

davecheney commented Jan 31, 2013

rsc commented Jan 31, 2013

rogpeppe commented Jan 31, 2013

rsc commented Jan 31, 2013

rsc commented Mar 12, 2013

masiulaniec commented Apr 5, 2013

robpike commented May 18, 2013

rsc commented Sep 10, 2013

rsc commented Sep 10, 2013