Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: list -json includes assembly files among CompiledGoFiles #28749

Closed
alandonovan opened this issue Nov 12, 2018 · 11 comments
Closed

cmd/go: list -json includes assembly files among CompiledGoFiles #28749

alandonovan opened this issue Nov 12, 2018 · 11 comments
Assignees
Labels
FrozenDueToAge GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@alandonovan
Copy link
Contributor

alandonovan commented Nov 12, 2018

$ go version
go version devel +a4e958ff2d Mon Nov 12 13:44:06 2018 -0500 linux/amd64
$ go list -json -compiled reflect
{
        "Dir": "/usr/local/google/home/adonovan/goroot/src/reflect",
        "ImportPath": "reflect",
        "Name": "reflect",
...
        "GoFiles": [
                "deepequal.go",
                "makefunc.go",
                "swapper.go",
                "type.go",
                "value.go"
        ],
        "CompiledGoFiles": [
                "deepequal.go",
                "makefunc.go",
                "swapper.go",
                "type.go",
                "value.go",
                "asm_amd64.s"   // !
        ],
        "SFiles": [
                "asm_amd64.s"
        ],
...
@gopherbot
Copy link

Change https://golang.org/cl/149237 mentions this issue: go/packages: remove .s files from go list's CompiledGoFiles

gopherbot pushed a commit to golang/tools that referenced this issue Nov 13, 2018
This is a workaround for a go list regression that broke
go/packages but went unnoticed by because of a missing
call to packages.PrintErrors, added here.

Updates golang/go#28749

Change-Id: I1819a6143134a422791106ac037d3458ef864322
Reviewed-on: https://go-review.googlesource.com/c/149237
Reviewed-by: Ian Cottrell <iancottrell@google.com>
@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Nov 19, 2018
@bcmills bcmills added this to the Go1.13 milestone Nov 19, 2018
@bcmills bcmills added the GoCommand cmd/go label Jan 10, 2019
@bcmills bcmills removed their assignment Jan 10, 2019
@josharian
Copy link
Contributor

.cc files too. This is pretty clearly a bug in cmd/go:

  • cmd/go/internal/work/exec.go, around line 590:
	var srcfiles []string // .go and non-.go
	srcfiles = append(srcfiles, gofiles...)
	srcfiles = append(srcfiles, sfiles...)
	srcfiles = append(srcfiles, cfiles...)
	srcfiles = append(srcfiles, cxxfiles...)
	b.cacheSrcFiles(a, srcfiles)
  • cacheSrcFiles eventually does: c.PutBytes(cache.Subkey(a.actionID, "srcfiles"), buf.Bytes()).

  • Later, loadCachedSrcFiles does: list, _, err := c.GetBytes(cache.Subkey(a.actionID, "srcfiles")) and then a.Package.CompiledGoFiles = files.

  • This makes its way into the json that cmd/go emits, which go/packages trusts absolutely.

We should make cmd/go filter out non-Go files.


There's a comment in the go/packages source code, where it manually skips .s files:

		// Workaround for https://golang.org/issue/28749.
		// TODO(adonovan): delete before go1.12 release.

If we don't fix this for 1.12, we'll have to keep the workaround in go/packages for a long time. And expand it to cover .cc, .c, .S, etc.

I'll send a CL to expand the go/packages workaround, but I think it might be good to know how risky it is to fix this yet in cmd/go.

@gopherbot
Copy link

Change https://golang.org/cl/163597 mentions this issue: go/packages: expand CompiledGoFiles workaround

gopherbot pushed a commit to golang/tools that referenced this issue Feb 27, 2019
Updates golang/go#28749

Change-Id: I76eb264f61b511fec7e05cef1bdd35e1c7fd603b
Reviewed-on: https://go-review.googlesource.com/c/163597
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@gopherbot
Copy link

Change https://golang.org/cl/313131 mentions this issue: cmd/go: don't add generated SWIG C++ files to CompiledGoFiles

gopherbot pushed a commit that referenced this issue Apr 26, 2021
Also include SWIG C++ files in cgo hash.

For #28749
Fixes #37098

Change-Id: I6d912db2788200c2abdf328e382d4fbefda0a9ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/313131
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@mvdan
Copy link
Member

mvdan commented Sep 23, 2022

Hello from the future - I just ran into this :) Would a patch be welcome?

Edit: I am also confused by the workaround in go/packages; it seems to imply this was fixed in cmd/go in Go 1.12, but I still see the same problem. For example:

$ go version
go version go1.19.1 linux/amd64
$ go list -compiled -f '{{ range $_, $file := .CompiledGoFiles}}{{$file}}{{"\n"}}{{end}}' runtime | grep -v '\.go$'
asm.s
asm_amd64.s
duff_amd64.s
memclr_amd64.s
memmove_amd64.s
preempt_amd64.s
rt0_linux_amd64.s
sys_linux_amd64.s
time_linux_amd64.s

@mvdan
Copy link
Member

mvdan commented Sep 23, 2022

For those wanting a workaround which isn't tied to go/packages, this worked for me:

out := pkg.CompiledGoFiles[:0]
for _, path := range pkg.CompiledGoFiles {
	switch filepath.Ext(path) {
	case "": // e.g. a generated Go file inside the build cache
	case ".go":
	default: // e.g. an assembly file
		continue
	}
	out = append(out, path)
}
pkg.CompiledGoFiles = out

@gopherbot
Copy link

Change https://go.dev/cl/451285 mentions this issue: cmd/go: don't report non-go files in CompiledGoFiles

@bcmills
Copy link
Contributor

bcmills commented Nov 18, 2022

Looking at the fix CL, I believe this bug was introduced in https://go.dev/cl/148904 (ironically, authored by @alandonovan 😅).

@adonovan
Copy link
Member

Heh, that fellow must be feeling pretty stupid. Thanks for the fix.

@bcmills bcmills modified the milestones: Backlog, Go1.20 Nov 18, 2022
@gopherbot
Copy link

Change https://go.dev/cl/452059 mentions this issue: go/packages: change workaround for issue 28749 to not require NeedFiles

@mvdan
Copy link
Member

mvdan commented Nov 18, 2022

Looking forward to deleting my workaround in a year or two :)

gopherbot pushed a commit to golang/tools that referenced this issue Nov 21, 2022
Before this change, the workaround for golang/go#28749 filtered out the files
from OtherFiles from CompiledGoFiles to remove non-go files that are
appearing in CompiledGoFiles. But OtherFiles requires NeedFiles, so the
value of NeedFiles is affecting CompiledGoFiles which should only be
controlled by NeedCompliedGoFiles. Instead remove all files that don't
look like Go files.

Fixes golang/go#56632

Change-Id: Ib5e310f39f2470ee3b3488b6c4b74b70e3164d4c
Reviewed-on: https://go-review.googlesource.com/c/tools/+/452059
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Matloob <matloob@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
@golang golang locked and limited conversation to collaborators Nov 18, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

9 participants