cmd/go: list with -export and -covermode=atomic fails to build #65264

maxsours · 2024-01-24T18:26:24Z

Go version

go version go1.21.6 darwin/arm64

Output of `go env` in your module/workspace:

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='~/Library/Caches/go-build'
GOENV='~/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='~/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='~/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.21.6'
GCCGO='gccgo'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/y8/kc0q4v_x1454fr__4l32j26w0000gp/T/go-build2219730342=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

$ go list -export -cover -race cmd/nm
# cmd/nm
/usr/local/go/src/cmd/nm/doc.go:41:37: could not import sync/atomic (open : no such file or directory)
cmd/nm

What did you see happen?

$ go list -export -cover -race cmd/nm
# cmd/nm
/usr/local/go/src/cmd/nm/doc.go:41:37: could not import sync/atomic (open : no such file or directory)
cmd/nm

What did you expect to see?

go build -cover -race works as expected. We would expect the go list command to succeed as well.
When running go list ... with -work, we can see sync/atomic is correctly added to the *.cover.go files, but it is not included in importcfg:

# import config
packagefile bufio=~/Library/Caches/go-build/b9/b98806a14c72980fa955342a2ffef07b871d95ac024cd8fd668021668be97816-d
packagefile cmd/internal/objfile=~/Library/Caches/go-build/b9/b9262c3ee809db084e17516b1c55810e3f3150e22831e4f4eafbf1a888855364-d
packagefile flag=~/Library/Caches/go-build/13/139eccd9f3144df8ee6480494307f2b06aa9bf19ba2f74cb4c7f9d14115d7819-d
packagefile fmt=~/Library/Caches/go-build/a3/a37fb0a0c11e799c4992ab511760fee037811f5481a1f7c4cf63ce7e08c3a0e6-d
packagefile log=~/Library/Caches/go-build/f1/f18a69602c944d8e4e1364a2d89a5c45cb101025319fb7c4cbe6a196fe249832-d
packagefile os=~/Library/Caches/go-build/d6/d64010892cba9d91f2fd56ee1177c4e1a48b5012f9ae1f58b8825412d853c590-d
packagefile sort=~/Library/Caches/go-build/0d/0daf4569d16ea11206c9465ae9f23f50a11e2ac546adbf32f8dc054a301e08d5-d
packagefile runtime=~/Library/Caches/go-build/e1/e1997716a043c5092d7ab1465a0a8e05c834335e26d446fedf0f9c4ac0c349d7-d
packagefile runtime/coverage=~/Library/Caches/go-build/e3/e30bc0cbef27b51d08c1ea7c44994cc02766dd15dd6435600a5ede665bb052c0-d

The text was updated successfully, but these errors were encountered:

bcmills · 2024-01-24T22:25:29Z

cmd/go adds the implicit dependency on sync/atomic here:
https://cs.opensource.google/go/go/+/master:src/cmd/go/internal/load/pkg.go;l=3507-3510;drc=36b14a78b58924a8aea22c8949c3b8a4b7045d8b

The call chain is via:

So the question is: is this a typestate bug (with the error happening before the call to PrepareForCoverageBuild, or is something in that call chain not reaching where we expect it to?

(attn @thanm)

cixel · 2024-01-25T19:17:07Z

My very hazy understanding is that it's an issue in runList:

EnsureImport updates a package's Internal.Imports and is adding the sync/atomic package.

This needs to happen before AutoAction creates the Action for cmd/nm, because it reads nm's Internal.Imports to populate Deps. These are what are used when printing the importcfg.

Locally, I was able to get the repro command to work by calling PrepareForCoverageBuild before creating the Action:

diff --git a/src/cmd/go/internal/list/list.go b/src/cmd/go/internal/list/list.go
index 92020da946..3fd697f3df 100644
--- a/src/cmd/go/internal/list/list.go
+++ b/src/cmd/go/internal/list/list.go
@@ -723,6 +723,9 @@ func runList(ctx context.Context, cmd *base.Command, args []string) {
                b.IsCmdList = true
                b.NeedExport = *listExport
                b.NeedCompiledGoFiles = *listCompiled
+               if cfg.Experiment.CoverageRedesign && cfg.BuildCover {
+                       load.PrepareForCoverageBuild(pkgs)
+               }
                a := &work.Action{}
                // TODO: Use pkgsFilter?
                for _, p := range pkgs {
@@ -730,9 +733,6 @@ func runList(ctx context.Context, cmd *base.Command, args []string) {
                                a.Deps = append(a.Deps, b.AutoAction(work.ModeInstall, work.ModeInstall, p))
                        }
                }
-               if cfg.Experiment.CoverageRedesign && cfg.BuildCover {
-                       load.PrepareForCoverageBuild(pkgs)
-               }
                b.Do(ctx, a)
        }

mknyszek · 2024-01-31T21:50:02Z

In triage, I think we're waiting for @thanm to take a look. Don't mean to assign you work though, so feel free to comment and unassign. Thanks! :)

gopherbot · 2024-02-01T19:17:21Z

Change https://go.dev/cl/560236 mentions this issue: cmd/go: fix build config before creating actions for 'go list -cover'

cixel · 2024-02-01T19:28:24Z

Change https://go.dev/cl/560236 mentions this issue: cmd/go: fix build config before creating actions for 'go list -cover'

I tried my hand at submitting a patch. It's quite small. Happy to take direction if my CL is incomplete or incorrect, or else close it and get out of the way to let a maintainer handle it. 😅

When -covermode is set to atomic, instrumented packages need to import sync/atomic. If this is not already imported by a package being instrumented, the build needs to ensure that sync/atomic is compiled whenever 'go list' is run in a way that triggers package builds. The build config was already being made to ensure the import, but only after the action graph had been created, so there was no guarantee that sync/atomic would be built when needed. Fixes golang#65264. Change-Id: Ib3f1e102ce2ef554ea08330d9db69a8c98790ac5 Reviewed-on: https://go-review.googlesource.com/c/go/+/560236 Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>

bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go compiler/runtime Issues related to the Go compiler and/or runtime. labels Jan 24, 2024

mknyszek added this to the Backlog milestone Jan 31, 2024

mknyszek assigned thanm Jan 31, 2024

gopherbot closed this as completed in ac08c05 Feb 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: list with -export and -covermode=atomic fails to build #65264

cmd/go: list with -export and -covermode=atomic fails to build #65264

maxsours commented Jan 24, 2024

bcmills commented Jan 24, 2024

cixel commented Jan 25, 2024 •

edited

mknyszek commented Jan 31, 2024

gopherbot commented Feb 1, 2024

cixel commented Feb 1, 2024

cmd/go: list with -export and -covermode=atomic fails to build #65264

cmd/go: list with -export and -covermode=atomic fails to build #65264

Comments

maxsours commented Jan 24, 2024

Go version

Output of go env in your module/workspace:

What did you do?

What did you see happen?

What did you expect to see?

bcmills commented Jan 24, 2024

cixel commented Jan 25, 2024 • edited

mknyszek commented Jan 31, 2024

gopherbot commented Feb 1, 2024

cixel commented Feb 1, 2024

Output of `go env` in your module/workspace:

cixel commented Jan 25, 2024 •

edited