Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: 'go list -test' prints main package twice #34321

Closed
dmitshur opened this issue Sep 16, 2019 · 8 comments
Closed

cmd/go: 'go list -test' prints main package twice #34321

dmitshur opened this issue Sep 16, 2019 · 8 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone

Comments

@dmitshur
Copy link
Contributor

dmitshur commented Sep 16, 2019

image

--- FAIL: TestCallgraph (1.60s)
    main_test.go:85: internal error: go list gives conflicting information for package pkg [pkg.test]
    main_test.go:85: internal error: go list gives conflicting information for package pkg [pkg.test]
FAIL
FAIL	golang.org/x/tools/cmd/callgraph	1.622s

From https://build.golang.org/?repo=golang.org%2fx%2ftools, it looks like the failure started to happen roughly close to 24781a1, but that might not be the exact commit.

Unfortunately, I can't reproduce it locally yet. For now, I'll send a CL to get more debug info out of the builders where it reproduces easily.

/cc @jayconrod @matloob

@dmitshur dmitshur added Testing An issue that has been verified to require only test changes, not just a test failure. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Sep 16, 2019
@gopherbot gopherbot added this to the Unreleased milestone Sep 16, 2019
@gopherbot gopherbot added the Tools This label describes issues relating to any tools in the x/tools repository. label Sep 16, 2019
@gopherbot
Copy link

Change https://golang.org/cl/195617 mentions this issue: go/packages: include more detail in internal error

@dmitshur
Copy link
Contributor Author

CL 195617 produced some more debug information:

--- FAIL: TestCallgraph (0.61s)
    main_test.go:85: internal error: go list gives conflicting information for package pkg [pkg.test]:
        old = &packages.jsonPackage{ImportPath:"pkg [pkg.test]", Dir:"/workdir/gopath/src/golang.org/x/tools/cmd/callgraph/testdata/src/pkg", Name:"main", Export:"", GoFiles:[]string{"pkg.go", "pkg_test.go"}, CompiledGoFiles:[]string{"pkg.go", "pkg_test.go"}, CFiles:[]string(nil), CgoFiles:[]string(nil), CXXFiles:[]string(nil), MFiles:[]string(nil), HFiles:[]string(nil), FFiles:[]string(nil), SFiles:[]string(nil), SwigFiles:[]string(nil), SwigCXXFiles:[]string(nil), SysoFiles:[]string(nil), Imports:[]string(nil), ImportMap:map[string]string(nil), Deps:[]string{"internal/bytealg", "internal/cpu", "runtime", "runtime/internal/atomic", "runtime/internal/math", "runtime/internal/sys", "unsafe"}, TestGoFiles:[]string{"pkg_test.go"}, TestImports:[]string(nil), XTestGoFiles:[]string(nil), XTestImports:[]string(nil), ForTest:"pkg", DepOnly:true, Error:(*packages.jsonPackageError)(nil)}
        new = &packages.jsonPackage{ImportPath:"pkg [pkg.test]", Dir:"/workdir/gopath/src/golang.org/x/tools/cmd/callgraph/testdata/src/pkg", Name:"main", Export:"", GoFiles:[]string{"pkg.go", "pkg_test.go"}, CompiledGoFiles:[]string{"pkg.go", "pkg_test.go"}, CFiles:[]string(nil), CgoFiles:[]string(nil), CXXFiles:[]string(nil), MFiles:[]string(nil), HFiles:[]string(nil), FFiles:[]string(nil), SFiles:[]string(nil), SwigFiles:[]string(nil), SwigCXXFiles:[]string(nil), SysoFiles:[]string(nil), Imports:[]string(nil), ImportMap:map[string]string(nil), Deps:[]string{"internal/bytealg", "internal/cpu", "runtime", "runtime/internal/atomic", "runtime/internal/math", "runtime/internal/sys", "unsafe"}, TestGoFiles:[]string{"pkg_test.go"}, TestImports:[]string(nil), XTestGoFiles:[]string(nil), XTestImports:[]string(nil), ForTest:"pkg", DepOnly:false, Error:(*packages.jsonPackageError)(nil)}
    main_test.go:85: internal error: go list gives conflicting information for package pkg [pkg.test]:
        old = &packages.jsonPackage{ImportPath:"pkg [pkg.test]", Dir:"/workdir/gopath/src/golang.org/x/tools/cmd/callgraph/testdata/src/pkg", Name:"main", Export:"", GoFiles:[]string{"pkg.go", "pkg_test.go"}, CompiledGoFiles:[]string{"pkg.go", "pkg_test.go"}, CFiles:[]string(nil), CgoFiles:[]string(nil), CXXFiles:[]string(nil), MFiles:[]string(nil), HFiles:[]string(nil), FFiles:[]string(nil), SFiles:[]string(nil), SwigFiles:[]string(nil), SwigCXXFiles:[]string(nil), SysoFiles:[]string(nil), Imports:[]string(nil), ImportMap:map[string]string(nil), Deps:[]string{"internal/bytealg", "internal/cpu", "runtime", "runtime/internal/atomic", "runtime/internal/math", "runtime/internal/sys", "unsafe"}, TestGoFiles:[]string{"pkg_test.go"}, TestImports:[]string(nil), XTestGoFiles:[]string(nil), XTestImports:[]string(nil), ForTest:"pkg", DepOnly:true, Error:(*packages.jsonPackageError)(nil)}
        new = &packages.jsonPackage{ImportPath:"pkg [pkg.test]", Dir:"/workdir/gopath/src/golang.org/x/tools/cmd/callgraph/testdata/src/pkg", Name:"main", Export:"", GoFiles:[]string{"pkg.go", "pkg_test.go"}, CompiledGoFiles:[]string{"pkg.go", "pkg_test.go"}, CFiles:[]string(nil), CgoFiles:[]string(nil), CXXFiles:[]string(nil), MFiles:[]string(nil), HFiles:[]string(nil), FFiles:[]string(nil), SFiles:[]string(nil), SwigFiles:[]string(nil), SwigCXXFiles:[]string(nil), SysoFiles:[]string(nil), Imports:[]string(nil), ImportMap:map[string]string(nil), Deps:[]string{"internal/bytealg", "internal/cpu", "runtime", "runtime/internal/atomic", "runtime/internal/math", "runtime/internal/sys", "unsafe"}, TestGoFiles:[]string{"pkg_test.go"}, TestImports:[]string(nil), XTestGoFiles:[]string(nil), XTestImports:[]string(nil), ForTest:"pkg", DepOnly:false, Error:(*packages.jsonPackageError)(nil)}
FAIL

The diff:

 &packages.jsonPackage{
 	[...]
-	DepOnly:true,
+	DepOnly:false,
 	[...]
 }

@matloob
Copy link
Contributor

matloob commented Sep 16, 2019

I'm able to reproduce it locally on tip. It doesn't reproduce on earlier versions of go.

(To reproduce locally you'll have to set your PATH to include tip go because go/packages starts go list as a subprocess)

@dmitshur
Copy link
Contributor Author

dmitshur commented Sep 16, 2019

Unfortunately, I can't reproduce it locally yet.

I figured out why it was failing to reproduce for me. I made the mistake of doing this:

PATH="$HOME/gotip/bin:$PATH" go version && go test -v ./cmd/callgraph

Instead of something else that would do what I actually intended:

PATH="$HOME/gotip/bin:$PATH" go version && \
  PATH="$HOME/gotip/bin:$PATH" go test -v ./cmd/callgraph

🤦‍♂

Oh well, mystery solved. Thanks @matloob, your comment helped me figure it out.

@jayconrod jayconrod changed the title x/tools/cmd/callgraph: TestCallgraph has started to fail with Go tip cmd/go: 'go list -test' prints main package twice Sep 16, 2019
@jayconrod jayconrod added the NeedsFix The path to resolution is known, but the work has not been done. label Sep 16, 2019
@jayconrod jayconrod modified the milestones: Unreleased, Go1.13.1 Sep 16, 2019
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 16, 2019
@jayconrod
Copy link
Contributor

This was caused by CL 195279. Sorry for the break! (I wish we ran tools tests as a presubmit).

When the go command loads test packages, it needs to recompile packages that transitively import internal test packages. There are a couple extra rules for main packages imported by tests: they can't be compiled with -p main because that causes link errors.

After CL 195279, it appears that internal test main packages were being recompiled twice, resulting in an extra entry in the go list -test -deps output.

@gopherbot
Copy link

Change https://golang.org/cl/195677 mentions this issue: cmd/go: don't split internal test main packages twice

@jayconrod
Copy link
Contributor

@gopherbot Please backport to 1.13.1.

CL 195281 is a cherry-pick of CL 195279, which will introduce this issue. These CLs should be cherry-picked together.

@gopherbot
Copy link

Backport issue(s) opened: #34328 (for 1.13).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@bcmills bcmills modified the milestones: Go1.13.1, Go1.13.2 Sep 25, 2019
@katiehockman katiehockman modified the milestones: Go1.13.2, Go1.13.3 Oct 17, 2019
@golang golang locked and limited conversation to collaborators Oct 16, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Development

No branches or pull requests

6 participants