cmd/go: error isn't properly reported when running go list on cgo package with missing pkg-config #35882

muirdm · 2019-11-27T21:54:36Z

What version of Go are you using (`go version`)?

$ go version
go version go1.13.3 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/muir/Library/Caches/go-build"
GOENV="/Users/muir/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/muir/projects//go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/_q/1xpb0k992ll803xsldpx0qyr0000gn/T/go-build252385107=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I opened a non-cgo package in our project. The package does not transitively depend on any cgo packages.

What did you expect to see?

I expected gopls to load the package with no issue.

What did you see instead?

gopls failed to create the view due to a go list "missing pkg-config" error.

LSP :: Error loading workspace folders (expected 1, got 0)
failed to load view for file:///Users/muir/projects/<project>/go/src: error loading packages: go [list -e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./...]: exit status 2: # pkg-config --cflags  -- <bunch of libraries>
pkg-config: exec: "pkg-config": executable file not found in $PATH

Elsewhere, our project has a cgo package that uses the cgo pkg-config directive (e.g. // #cgo pkg-config: some-library). My laptop doesn't currently have pkg-config installed.

I assume the error comes from the go list ./... we do now when creating a view.

The text was updated successfully, but these errors were encountered:

gopherbot · 2019-11-27T21:54:41Z

Thank you for filing a gopls issue! Please take a look at the Troubleshooting guide, and make sure that you have provided all of the relevant information here.

stamblerre · 2019-11-27T22:01:52Z

Thanks for reporting. This should probably be handled more gracefully in go/packages. Assigning to @matloob.

matloob · 2019-12-06T18:37:24Z

I can reproduce the underlying go list issue with the following layout:

foo/go.mod:

module foo

go1.13

foo/foo.go:

package foo

// #cgo pkg-config: bar
import "C"

output of go list -json -compiled . in directory foo:

{
	"Dir": "/Users/matloob/Desktop/foo",
	"ImportPath": "foo",
	"Name": "foo",
	"Root": "/Users/matloob/Desktop/foo",
	"Module": {
		"Path": "foo",
		"Main": true,
		"Dir": "/Users/matloob/Desktop/foo",
		"GoMod": "/Users/matloob/Desktop/foo/go.mod",
		"GoVersion": "1.13"
	},
	"Match": [
		"."
	],
	"Stale": true,
	"StaleReason": "build ID mismatch",
	"CgoFiles": [
		"foo.go"
	],
	"CgoPkgConfig": [
		"bar"
	],
	"Imports": [
		"C",
		"unsafe",
		"runtime/cgo",
		"syscall"
	],
	"Deps": [
		"errors",
		"internal/bytealg",
		"internal/cpu",
		"internal/oserror",
		"internal/race",
		"internal/reflectlite",
		"runtime",
		"runtime/cgo",
		"runtime/internal/atomic",
		"runtime/internal/math",
		"runtime/internal/sys",
		"sync",
		"sync/atomic",
		"syscall",
		"unsafe"
	]
}

go list exits with an exit status of 2.

I'm going to send a cl to suppress the error in go/packages. There are still two problems in go list. The first is that -e isn't being respected. The second is that no error is being generated in the package output by go list.

gopherbot · 2019-12-06T18:43:16Z

Change https://golang.org/cl/210202 mentions this issue: go/packages: suppress go list -e error when pkg-config can't be run

matloob · 2019-12-06T18:45:35Z

I'm going to repurpose this bug to fix the issue in go list.

Note that even when the issue is fixed in go list, if go/packages or gopls are operating on a project that has a dependency on a cgo package that cgo can't successfully be run on, they won't have access to the full set of sources (which need to be generated by cgo). So they might still have problems for those projects.

Of course if none of the packages you're working on have a dependency on the offending package, things should work.

muirdm · 2019-12-06T18:47:56Z

Also note that even after installing pkg-config, go/packages still failed in a similar way because the C library wasn't installed.

matloob · 2019-12-06T19:08:05Z

What was the error it failed with? we're probably going to have to put another special case in go/packages to ignore that error...

muirdm · 2019-12-06T19:18:18Z

2019/12/06 11:16:40 failed to load snapshot: go [list -e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./...]: exit status 2: # pkg-config --cflags  -- libavdevice libavformat libavfilter libavcodec libswscale libavutil libavdevice libavformat libavfilter libavcodec libswscale libavutil libavdevice libavformat libavfilter libavcodec libswscale libavutil libavdevice libavformat libavfilter libavcodec libswscale libavutil
Package libavdevice was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavdevice.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavdevice' found
Package libavformat was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavformat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavformat' found
Package libavfilter was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavfilter.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavfilter' found
Package libavcodec was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavcodec.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavcodec' found
Package libswscale was not found in the pkg-config search path.
Perhaps you should add the directory containing `libswscale.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libswscale' found
Package libavutil was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavutil.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavutil' found
Package libavdevice was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavdevice.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavdevice' found
Package libavformat was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavformat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavformat' found
Package libavfilter was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavfilter.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavfilter' found
Package libavcodec was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavcodec.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavcodec' found
Package libswscale was not found in the pkg-config search path.
Perhaps you should add the directory containing `libswscale.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libswscale' found
Package libavutil was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavutil.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavutil' found
Package libavdevice was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavdevice.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavdevice' found
Package libavformat was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavformat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavformat' found
Package libavfilter was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavfilter.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavfilter' found
Package libavcodec was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavcodec.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavcodec' found
Package libswscale was not found in the pkg-config search path.
Perhaps you should add the directory containing `libswscale.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libswscale' found
Package libavutil was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavutil.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavutil' found
Package libavdevice was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavdevice.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavdevice' found
Package libavformat was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavformat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavformat' found
Package libavfilter was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavfilter.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavfilter' found
Package libavcodec was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavcodec.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavcodec' found
Package libswscale was not found in the pkg-config search path.
Perhaps you should add the directory containing `libswscale.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libswscale' found
Package libavutil was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavutil.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavutil' found
pkg-config: exit status 1
# pkg-config --cflags  -- libavdevice libavformat libavfilter libavcodec libswscale libavutil libavdevice libavformat libavfilter libavcodec libswscale libavutil libavdevice libavformat libavfilter libavcodec libswscale libavutil libavdevice libavformat libavfilter libavcodec libswscale libavutil
Package libavdevice was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavdevice.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavdevice' found
Package libavformat was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavformat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavformat' found
Package libavfilter was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavfilter.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavfilter' found
Package libavcodec was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavcodec.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavcodec' found
Package libswscale was not found in the pkg-config search path.
Perhaps you should add the directory containing `libswscale.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libswscale' found
Package libavutil was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavutil.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavutil' found
Package libavdevice was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavdevice.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavdevice' found
Package libavformat was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavformat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavformat' found
Package libavfilter was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavfilter.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavfilter' found
Package libavcodec was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavcodec.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavcodec' found
Package libswscale was not found in the pkg-config search path.
Perhaps you should add the directory containing `libswscale.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libswscale' found
Package libavutil was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavutil.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavutil' found
Package libavdevice was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavdevice.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavdevice' found
Package libavformat was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavformat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavformat' found
Package libavfilter was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavfilter.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavfilter' found
Package libavcodec was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavcodec.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavcodec' found
Package libswscale was not found in the pkg-config search path.
Perhaps you should add the directory containing `libswscale.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libswscale' found
Package libavutil was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavutil.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavutil' found
Package libavdevice was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavdevice.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavdevice' found
Package libavformat was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavformat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavformat' found
Package libavfilter was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavfilter.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavfilter' found
Package libavcodec was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavcodec.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavcodec' found
Package libswscale was not found in the pkg-config search path.
Perhaps you should add the directory containing `libswscale.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libswscale' found
Package libavutil was not found in the pkg-config search path.
Perhaps you should add the directory containing `libavutil.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libavutil' found
pkg-config: exit status 1

	directory = file:///Users/muir/projects/<project>/go/src
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x16a69d5]

goroutine 136 [running]:
golang.org/x/tools/internal/lsp/cache.(*snapshot).View(0x0, 0xc00018cd80, 0xc0000ce300)
	/Users/muir/projects/tools/internal/lsp/cache/snapshot.go:61 +0x5
golang.org/x/tools/internal/lsp.(*Server).diagnoseSnapshot(0xc000278f60, 0x19cc4c0, 0x0)
	/Users/muir/projects/tools/internal/lsp/diagnostics.go:20 +0x69
created by golang.org/x/tools/internal/lsp.(*Server).addFolders
	/Users/muir/projects/tools/internal/lsp/general.go:173 +0x600

matloob · 2019-12-06T19:34:41Z

I'm abandoning the change. We've decided that the cost of the change is greater than the benefit. Because it's impossible to determine which package the error belongs to, we'd have to drop the error (or add errors to all packages) to work around it.

In general it's hard for go/packages to work correctly for code that doesn't build.

muirdm · 2019-12-06T19:52:37Z

When "go list" produces the cgo error and exits uncleanly it still also dumps what seems to be the normal JSON results (minus the cgo generated files). What would happen if we ignored the error and continued using the results?

If that is not tenable, is there some way we can at least make gopls able to start in the case of unbuildable cgo package?

stamblerre · 2019-12-06T20:05:02Z

Did CL 209420 help with this at all, or does gopls still fail in this case? If it does, I can add some workarounds in gopls to handle it.

muirdm · 2019-12-06T20:57:53Z

I uninstalled ffmpeg and still got the above panic (on master). Should be easy to reproduce by adding something like this to a file and then restarting gopls:

// #cgo pkg-config: some-library
import "C"

stamblerre · 2019-12-06T20:58:49Z

Ah, I just fixed that panic in CL 210215.

muirdm · 2019-12-06T21:18:25Z

Hot dog! Yes, seems to start up fine now.

stamblerre · 2019-12-06T21:33:23Z

Perfect! So it sounds like we will wait for this fix to be implemented in go list.

jayconrod · 2019-12-09T15:51:46Z

I think this is one instance of a broader problem: execution errors (from running the compiler, linker, or other tools) are not connected back to go list -e output.

Fixing that would be too complicated for 1.14 at this point in the freeze. We should consider it for 1.15 though.

ianlancetaylor · 2020-06-15T21:53:10Z

Moving to backlog.

gopherbot added this to the Unreleased milestone Nov 27, 2019

gopherbot added Tools This label describes issues relating to any tools in the x/tools repository. gopls Issues related to the Go language server, gopls. labels Nov 27, 2019

stamblerre assigned matloob Nov 27, 2019

stamblerre modified the milestones: Unreleased, gopls v1.0 Dec 4, 2019

matloob changed the title ~~x/tools/gopls: gopls can't start without pkg-config~~ cmd/go: error isn't properly reported when running go list on cgo package with missing pkg-config Dec 6, 2019

matloob added the GoCommand cmd/go label Dec 6, 2019

stamblerre removed Tools This label describes issues relating to any tools in the x/tools repository. gopls Issues related to the Go language server, gopls. labels Dec 6, 2019

stamblerre modified the milestones: gopls v1.0, Unreleased Dec 6, 2019

stamblerre unassigned matloob Dec 6, 2019

jayconrod added the NeedsFix The path to resolution is known, but the work has not been done. label Dec 9, 2019

jayconrod removed this from the Unreleased milestone Dec 9, 2019

jayconrod added this to the Go1.15 milestone Dec 9, 2019

ianlancetaylor modified the milestones: Go1.15, Backlog Jun 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: error isn't properly reported when running go list on cgo package with missing pkg-config #35882

cmd/go: error isn't properly reported when running go list on cgo package with missing pkg-config #35882

muirdm commented Nov 27, 2019

gopherbot commented Nov 27, 2019

stamblerre commented Nov 27, 2019

matloob commented Dec 6, 2019 •

edited

gopherbot commented Dec 6, 2019

matloob commented Dec 6, 2019

muirdm commented Dec 6, 2019

matloob commented Dec 6, 2019

muirdm commented Dec 6, 2019

matloob commented Dec 6, 2019

muirdm commented Dec 6, 2019

stamblerre commented Dec 6, 2019

muirdm commented Dec 6, 2019

stamblerre commented Dec 6, 2019

muirdm commented Dec 6, 2019

stamblerre commented Dec 6, 2019

jayconrod commented Dec 9, 2019

ianlancetaylor commented Jun 15, 2020

cmd/go: error isn't properly reported when running go list on cgo package with missing pkg-config #35882

cmd/go: error isn't properly reported when running go list on cgo package with missing pkg-config #35882

Comments

muirdm commented Nov 27, 2019

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

gopherbot commented Nov 27, 2019

stamblerre commented Nov 27, 2019

matloob commented Dec 6, 2019 • edited

gopherbot commented Dec 6, 2019

matloob commented Dec 6, 2019

muirdm commented Dec 6, 2019

matloob commented Dec 6, 2019

muirdm commented Dec 6, 2019

matloob commented Dec 6, 2019

muirdm commented Dec 6, 2019

stamblerre commented Dec 6, 2019

muirdm commented Dec 6, 2019

stamblerre commented Dec 6, 2019

muirdm commented Dec 6, 2019

stamblerre commented Dec 6, 2019

jayconrod commented Dec 9, 2019

ianlancetaylor commented Jun 15, 2020

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

matloob commented Dec 6, 2019 •

edited