cmd/go: go list -export -e outputs errors to stderr and has non-zero exit code #25842

myitcv · 2018-06-12T08:29:00Z

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

go version devel +6b3e343408 Tue Jun 12 03:42:37 2018 +0000 linux/amd64

Does this issue reproduce with the latest release?

No; this is only available in tip.

What operating system and processor architecture are you using (`go env`)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myitcv/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/tmp/tmp.4mkm9vJCVB"
GORACE=""
GOROOT="/home/myitcv/gos"
GOTMPDIR=""
GOTOOLDIR="/home/myitcv/gos/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build091981204=/tmp/go-build -gno-record-gcc-switches"

What did you do?

cd `mktemp -d`
export GOPATH=$PWD
mkdir -p src/example.com
cd src/example.com
mkdir p1
cat <<EOD > p1/p1.go
package p1

const Name = "p1"
EOD
mkdir p2
cat <<EOD > p2/main.go
package main

import "fmt"
import "example.com/p1"

func main() {
        fmt.Println(p1.Name == 5)
}
EOD

go list behaves as expected (there are no syntax errors but there is a compile error):

$ go list ./...
example.com/p1
example.com/p2

But if we add the -export and -e flags we see:

$ go list -e -export ./...
p2/main.go:7:29: cannot convert p1.Name (type untyped string) to type int
p2/main.go:7:29: invalid operation: p1.Name == 5 (mismatched types string and int)
example.com/p1
example.com/p2
$ echo $?
2

I wonder whether my expectations about -e are correct in the context of -export?

The -e flag changes the handling of erroneous packages, those that cannot be found or are malformed.

What did you expect to see?

$ go list -e -export ./...
example.com/p1
example.com/p2
$ echo $?
0

What did you see instead?

Per the repro steps:

$ go list -e -export ./...
p2/main.go:7:29: cannot convert p1.Name (type untyped string) to type int
p2/main.go:7:29: invalid operation: p1.Name == 5 (mismatched types string and int)
example.com/p1
example.com/p2
$ echo $?
2

/cc @rsc

The text was updated successfully, but these errors were encountered:

bcmills · 2018-06-12T16:06:10Z

It seems to me that go list -e -export should return zero only if it successfully wrote up-to-date export data. It would be reasonable for go list -e -export to return zero if there is an error in the package that does not affect its export data, if such a thing is possible.

That does suggest that we should refine the documentation, though. It's not quite accurate to say that “[w]ith the -e flag, the list command never prints errors to standard error”.

myitcv · 2018-06-12T16:50:25Z

It seems to me that go list -e -export should return zero only if it successfully wrote up-to-date export data.

To make it concrete, the issue I see with this is that as the caller of go list -e -export (and I'm guessing go/packages would also be such a caller?) I can't distinguish between:

go list failing for some truly fatal reason
go list failing because one or more of the packages I asked it to list -export (and I would typically do this with -json) failed to compile

We could use some sort of semantic that says "if there is any output on stdout then use it as an indicator of success, compile errors will be reported to stderr", but this seems somewhat brittle.

@bcmills do you see another way of me distinguishing between cases 1 and 2 above?

Which is why I felt the semantics of -e worked quite well for -export, with output to stderr being reserved for situations where there are indeed errors that will result in a non-zero exit code (or I guess where a -verbose-like flag is applied):

$ go list -json asdf
can't load package: package asdf: cannot find package "asdf" in any of:
        /home/myitcv/gos/src/asdf (from $GOROOT)
        /home/myitcv/.mountpoints/gopherize.me/src/myitcv.io/gopherize.me/_vendor/src/asdf (from $GOPATH)
        /home/myitcv/.mountpoints/gopherize.me/src/asdf
$ echo $?
1

vs:

$ go list -e -json asdf
{
        "ImportPath": "asdf",
        "Stale": true,
        "StaleReason": "build ID mismatch",
        "Incomplete": true,
        "Error": {
                "ImportStack": [
                        "asdf"
                ],
                "Pos": "",
                "Err": "cannot find package \"asdf\" in any of:\n\t/home/myitcv/gos/src/asdf (from $GOROOT)\n\t/home/myitcv/.mountpoints/gopherize.me/src/myitcv.io/gopherize.me/_vendor/src/asdf (from $GOPATH)\n\t/home/myitcv/.mountpoints/gopherize.me/src/asdf"
        }
}
$ echo $?
0

With go list -e -export, packages that fail to compile would simply omit the .Export field, and instead contain an .Error field similar to the above (with the compile error, although this is probably not essential).

bcmills · 2018-06-12T16:59:40Z

Hmm, that's a good point. -export is fairly useless without either -json or -f, and with one of those flags it seems reasonable to set Incomplete instead of Export for any package that we couldn't build.

rsc · 2018-08-09T15:29:11Z

May not fix for the release but I'd at least like to understand it better and talk to the go/packages team.

rsc · 2018-08-17T15:20:54Z

In general go/packages must expect that the subcommand it runs ("go list" or any other helper like bazel) can exit unsuccessfully and print errors to standard output/error. That should make Load fail, returning an err including the output. The usual way to generate such an error is:

out, err := cmd.CombinedOutput()
if err != nil {
	return nil, fmt.Errorf("%s: %v\n%s", strings.Join(cmdline, " "), err, out)
}

If errors happen during the build for -export, I agree that go list -e should ideally report them in the Package structs. But go/packages must be able to handle the current failure mode regardless, even if it's not quite right for the question that was asked.

Leaving for Go 1.12.

gopherbot · 2022-09-30T21:21:10Z

Change https://go.dev/cl/437298 mentions this issue: cmd/go: do not exit with non-zero code from go list -e -export

gopherbot · 2022-10-10T22:48:16Z

Change https://go.dev/cl/441879 mentions this issue: go/analysis: update tests for different go list error behavior

golang.org/cl/437298 updates go list to not exit with a non-zero error code in some cases and instead place the error on the Package struct. Also, as a cleanup, some places where exit status 2 was returned will return exit status 1 (bringing it in line with other errors returned by the go command). TestEncodeDecode in facts_test.go has been updated to fix a missing function body that is not relevant to the test, but that was causing an error because the Package struct now has an error on it. TestRunDespiteErrors in checker_test.go has been updated to reflect that in some cases an analysis with RunDespiteErrors will fail to run because a build error returned by go list when it's run to get export data is not recognized as being a parse/typechecker error (the kind of error allowed by TestRunDespiteError). TestIntegration in unitchecker_test.go has been updated to reflect that go vet running unitchecker will now fail with exit status 1 instead of 2 (so it just checks for a zero or non-zero status). For golang/go#25842 Change-Id: Idbbd19b5de661e6e5f49e0475c5bc918d8e33803 Reviewed-on: https://go-review.googlesource.com/c/tools/+/441879 Reviewed-by: Alan Donovan <adonovan@google.com> Reviewed-by: Michael Matloob <matloob@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> gopls-CI: kokoro <noreply+kokoro@google.com>

For golang#25842

go list -e -export puts errors running build actions on the load.Package corresponding to the failed action rather than exiting with a non zero exit code. For golang#25842 Change-Id: I1fea85cc5a0557f514fe9d4ed3b6a858376fdcde Reviewed-on: https://go-review.googlesource.com/c/go/+/437298 Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>

gopherbot · 2022-11-16T21:30:22Z

Change https://go.dev/cl/451218 mentions this issue: cmd/go/internal/work: make formatOutput return an error that includes the import path

… the import path This refines the error output that was previously adjusted in CL 437298. Longer term, we should consider unraveling the call chains involving formatOutput to avoid passing so many parameters through so many different formatting functions. Updates #25842. Change-Id: I3b9d03bf5968902d8ccc4841ab4dbe114a2239e4 Reviewed-on: https://go-review.googlesource.com/c/go/+/451218 Reviewed-by: Bryan Mills <bcmills@google.com> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Than McIntosh <thanm@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>

ldez · 2022-12-26T00:36:29Z

Hello,

I don't know if it's something expected but there is a side effect on packages.Load()

Related to golangci/golangci-lint#3414 (comment)

bcmills · 2023-06-09T21:33:15Z

@gopherbot, please backport to Go 1.19. This is needed to fix #60650.

gopherbot · 2023-06-09T21:33:44Z

Backport issue(s) opened: #60710 (for 1.19).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

gopherbot · 2023-06-09T21:43:10Z

Change https://go.dev/cl/502195 mentions this issue: [release-branch.go1.19] cmd/go: do not exit with non-zero code from go list -e -export

gopherbot · 2023-06-09T21:43:13Z

Change https://go.dev/cl/502196 mentions this issue: [release-branch.go1.19] cmd/go/internal/work: make formatOutput return an error that includes the import path

…o list -e -export go list -e -export puts errors running build actions on the load.Package corresponding to the failed action rather than exiting with a non zero exit code. Fixes #60710. Fixes #60650. Updates #25842. Change-Id: I1fea85cc5a0557f514fe9d4ed3b6a858376fdcde Reviewed-on: https://go-review.googlesource.com/c/go/+/437298 Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-on: https://go-review.googlesource.com/c/go/+/502195 TryBot-Bypass: Bryan Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>

…n an error that includes the import path This refines the error output that was previously adjusted in CL 437298. Longer term, we should consider unraveling the call chains involving formatOutput to avoid passing so many parameters through so many different formatting functions. Updates #60710. Updates #60650. Updates #25842. Change-Id: I3b9d03bf5968902d8ccc4841ab4dbe114a2239e4 Reviewed-on: https://go-review.googlesource.com/c/go/+/451218 Reviewed-by: Bryan Mills <bcmills@google.com> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Than McIntosh <thanm@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-on: https://go-review.googlesource.com/c/go/+/502196 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Bypass: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>

bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 12, 2018

bcmills added this to the Go1.11 milestone Jun 12, 2018

myitcv mentioned this issue Jul 10, 2018

cmd/go: list: errors in build caused by -export are printed to stderr, despite -e #26319

Closed

myitcv mentioned this issue Aug 2, 2018

cmd/go: go list -compiled -e fails completely if any package does not compile #26755

Closed

rsc added the release-blocker label Aug 9, 2018

rsc modified the milestones: Go1.11, Go1.12 Aug 17, 2018

myitcv mentioned this issue Oct 31, 2018

cmd/go: go list -e -json with syntax error behaves differently in test and non-test files #28491

Closed

bcmills self-assigned this Nov 14, 2018

rsc modified the milestones: Go1.12, Go1.13 Dec 6, 2018

bcmills added the GoCommand cmd/go label Dec 12, 2018

bcmills removed the release-blocker label May 16, 2019

andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019

rsc modified the milestones: Go1.14, Backlog Oct 9, 2019

dominikh mentioned this issue May 11, 2020

cmd/go: go list -e -compiled -json does not populate Error when cgo preprocessing fails #38990

Closed

dominikh mentioned this issue Feb 14, 2022

cmd/go: go list -e -export -json -gcflags=-bogus math prints to stderr #46817

Closed

rsc unassigned bcmills Jun 23, 2022

sxllwx added a commit to sxllwx/go that referenced this issue Oct 13, 2022

os/exec: exit cmd.Run() when context becomes done

1d66847

For golang#25842

ldez mentioned this issue Dec 26, 2022

go1.20 support golangci/golangci-lint#3414

Merged

gopherbot mentioned this issue Jun 9, 2023

cmd/go: go list -export -e outputs errors to stderr and has non-zero exit code [1.19 backport] #60710

Closed

bcmills mentioned this issue Jun 9, 2023

cmd/go, x/tools: many tests are broken with go1.19.10 #60650

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: go list -export -e outputs errors to stderr and has non-zero exit code #25842

cmd/go: go list -export -e outputs errors to stderr and has non-zero exit code #25842

myitcv commented Jun 12, 2018

bcmills commented Jun 12, 2018

myitcv commented Jun 12, 2018

bcmills commented Jun 12, 2018

rsc commented Aug 9, 2018

rsc commented Aug 17, 2018

gopherbot commented Sep 30, 2022

gopherbot commented Oct 10, 2022

gopherbot commented Nov 16, 2022

ldez commented Dec 26, 2022

bcmills commented Jun 9, 2023

gopherbot commented Jun 9, 2023

gopherbot commented Jun 9, 2023

gopherbot commented Jun 9, 2023

cmd/go: go list -export -e outputs errors to stderr and has non-zero exit code #25842

cmd/go: go list -export -e outputs errors to stderr and has non-zero exit code #25842

Comments

myitcv commented Jun 12, 2018

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

bcmills commented Jun 12, 2018

myitcv commented Jun 12, 2018

bcmills commented Jun 12, 2018

rsc commented Aug 9, 2018

rsc commented Aug 17, 2018

gopherbot commented Sep 30, 2022

gopherbot commented Oct 10, 2022

gopherbot commented Nov 16, 2022

ldez commented Dec 26, 2022

bcmills commented Jun 9, 2023

gopherbot commented Jun 9, 2023

gopherbot commented Jun 9, 2023

gopherbot commented Jun 9, 2023

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?