Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: duplicate symbols when including two main packages in -coverpkg in module mode #30374

Closed
lshulyay opened this issue Feb 24, 2019 · 25 comments
Labels
FrozenDueToAge GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@lshulyay
Copy link

Note that I am not sure if this is a bug or expected behaviour, only that this does not happen in the latest stable 1.11 release and does on 1.12rc1.

What version of Go are you using (go version)?

$ go version
go version go1.12rc1 linux/amd64

Does this issue reproduce with the latest release?

Not the latest stable release (1.11.5)

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/lazer/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/lazer/go"
GOPROXY=""
GORACE=""
GOROOT="/home/lazer/sdk/go1.12rc1"
GOTMPDIR=""
GOTOOLDIR="/home/lazer/sdk/go1.12rc1/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/lazer/Dev/gosnaillife/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build721704492=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I ran the following command:

go1.12rc1 test -race -v -coverpkg gitlab.com/drakonka/gosnaillife/cmd/snaillifecli,gitlab.com/drakonka/gosnaillife/cmd/snaillifesrv, -coverprofile allcoverage-rc.out ./...

Both of the above are main packages producing two binaries.

What did you expect to see?

I expected the tests to run and pass, with coverage output, as they do on Go 1.11.5

What did you see instead?

The following error:

2019/02/24 11:39:04 duplicate symbol runtime/debug.modinfo (types 30 and 30) in gitlab.com/drakonka/gosnaillife/cmd/snaillifecli and $WORK/b293/_pkg_.a(_go_.o)

I run Go RC and Go 1.11 side by side as part of my CI, which is how I noticed this issue. I am able to reproduce it locally as well. Full logs for the successful run of Go 1.11 and the failure run of Go 1.12rc1 can be found here:

@mvdan
Copy link
Member

mvdan commented Feb 25, 2019

I found the same regression just a few days ago, but hadn't filed an issue yet. This is impeding my daily development, and I'd consider it a regression introduced in 1.12.

The 1.12 release will be out any day now though, so it's probably more realistic to target a backport to 1.12.1. /cc @bcmills @ianlancetaylor @hyangah since this very likely relates to the recent modinfo changes in the cycle.

@mvdan mvdan added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 25, 2019
@mvdan mvdan added this to the Go1.12 milestone Feb 25, 2019
@bcmills
Copy link
Contributor

bcmills commented Feb 25, 2019

I'd consider it a regression introduced in 1.12.

This is almost certainly a bad interaction between #23910, //go:linkname, and the fix for #27584. Note that the former dates to at least Go 1.10, and can be a problem for any program that happens to use //go:linkname in package main (although those are hopefully rare).

Before the fix for #27584, the error (and redefinition) still existed, but because the variable was initialized in an init function, instead of a compile-time error one of the values of debug.modinfo would be overwritten at run-time.

@bcmills
Copy link
Contributor

bcmills commented Feb 25, 2019

Given that we do not currently merge coverage profiles from multiple processes (#28235), I'm not sure why you would want to include two different main packages in the same -coverpkg argument (except, perhaps, to simplify the patterns passed to -coverpkg).

The workaround for now is to include at most one main package in the -coverpkg arguments, or to run coverage in GOPATH mode.

@bcmills
Copy link
Contributor

bcmills commented Feb 25, 2019

CC @jayconrod

@bcmills bcmills changed the title Duplicate symbol error when running go test with two main packages in -coverpkg cmd/go: duplicate symbols when including two main packages in -coverpkg in module mode Feb 25, 2019
@jayconrod jayconrod self-assigned this Feb 25, 2019
@andybons andybons modified the milestones: Go1.12, Go1.13 Feb 25, 2019
@schoentoon
Copy link

Seems very similar to the issues I had when Golang 1.11 came out #23910. I guess I'm just disabling -coverpkg on CI for now.

@lshulyay
Copy link
Author

Seems very similar to the issues I had when Golang 1.11 came out #23910. I guess I'm just disabling -coverpkg on CI for now.

Depending on the details of your setup that seems like it might be overkill when you can just remove the conflicting main packages from -coverpkg and still get coverage reporting, right?

@markus-wa
Copy link
Contributor

Same issue on my end https://travis-ci.org/markus-wa/demoinfocs-golang/jobs/499992529

I see the milestone is 1.13, does that mean it won't be fixed in 1.12.x?

@mvdan
Copy link
Member

mvdan commented Feb 28, 2019

@bcmills I didn't know this was silently broken in 1.11; thanks for pointing this out. I think temporarily working around the issue for now is acceptable, but perhaps we should provide a simple example or one-liner that people could switch to.

@bcmills
Copy link
Contributor

bcmills commented Feb 28, 2019

@mvdan , the one-liner is GO111MODULE=off. (There is a reason I wasn't comfortable turning it on-by-default in 1.12, and bugs like this one are that reason: they're unusual but not esoteric.)

@mvdan
Copy link
Member

mvdan commented Feb 28, 2019

Thanks for the tip! If the bug(s) can be fixed in time for 1.13, then the timing with GO111MODULE=on would be ideal :)

@markus-wa
Copy link
Contributor

the one-liner is GO111MODULE=off

@bcmills but with that set to off, go, understandably, isn't able to find dependencies. - .travis.yml

Am I missing something obvious here?

I'm not sure why you would want to include two different main packages in the same -coverpkg argument

In my case I have a couple of examples for my lib and each of them has it's own main, then I just do go test -coverpkg=./... ./...

@jayconrod jayconrod added NeedsFix The path to resolution is known, but the work has not been done. GoCommand cmd/go and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Mar 1, 2019
@gopherbot
Copy link

Change https://golang.org/cl/164877 mentions this issue: cmd/go: avoid link errors when -coverpkg covers main packages

@sjansen
Copy link

sjansen commented Mar 1, 2019

I was bitten by the same problem: multiple main packages in subdirectories below examples/.

If you're slightly insane, here's a work around:

go test -race \
    -coverpkg .,$(
        ls -1d */ \
        | sed -E \
            -e :a \
            -e '$!N ; s|\n|,| ; ta' \
            -e 's#dist/|docs/|examples/|scripts/##g' \
            -e 's/,,/,/g ; s/^,// ; s/,$//' \
            -e 's|/|/...|g'
    ) \
    -coverprofile=dist/coverage.txt \
    -tags integration ./...

@markus-wa
Copy link
Contributor

@sjansen You can also use go list instead, if that's more sane.

go test -v -coverprofile=coverage.out -coverpkg=$(go list ./... | grep -v 'examples/' | awk -vORS=, '{ print $1 }' | sed 's/,$/\n/')

I'm still a bit bummed to lose coverage on the examples though 😄.

@lshulyay
Copy link
Author

lshulyay commented Mar 2, 2019

I've been doing it like this:

CVPKG=$(go list ./... | grep -v -e mocks -e snaillifecli -e snaillifesrv | tr '\n' ',') 

go test -race -v -coverpkg $CVPKG -coverprofile allcoverage-stable.out ./... 

@bcmills
Copy link
Contributor

bcmills commented Mar 8, 2019

@gopherbot, please backport to 1.12. This is a significant regression from GOPATH mode and the fix is straightforward.

@gopherbot
Copy link

Backport issue(s) opened: #30684 (for 1.12).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@jba
Copy link
Contributor

jba commented Mar 18, 2019

I can still reproduce this in 1.12.1:

> docker run -it golang:1.12.1
# cd ..
# git clone https://github.com/google/go-cloud
# cd go-cloud
# export GO111MODULE=on
# go test -coverpkg=./... ./...
...
2019/03/18 12:52:58 duplicate symbol runtime/debug.modinfo (types 30 and 30) in gocloud.dev/samples/blobdownload and $WORK/b466/_pkg_.a(_go_.o)  

(Note: I had to run the go test line several times because I would get occasional "Could not resolve host: github.com" errors. Not sure whether that's specific to my setup.)

@jayconrod
Copy link
Contributor

Reproduced. Trying to narrow it down.

It's possible this is a new issue. The _go_.o part of the message is suspicious. I don't think that file should be getting linked.

@jba
Copy link
Contributor

jba commented Mar 18, 2019

It might be useful to know that this repo has nested modules.

@gopherbot
Copy link

Change https://golang.org/cl/168200 mentions this issue: cmd/go: avoid link error when -coverpkg covers main packages (more)

@jayconrod
Copy link
Contributor

@gopherbot Please backport to 1.12. This is a more complete fix for an issue that was supposed to be fixed in 1.12.1.

@dmitshur
Copy link
Contributor

For posterity, the second backport issue was created manually: #30937 (for 1.12).

@imsodin
Copy link

imsodin commented May 17, 2019

I am encountering errors on go1.12.5 just like previously reported.

$ go version
go version go1.12.5 linux/amd64
go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/simon/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/media/ext4_data/Coding/go"
GOPROXY=""
GORACE=""
GOROOT="/media/ext4_data/Linux/source/go"
GOTMPDIR=""
GOTOOLDIR="/media/ext4_data/Linux/source/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/simon/tmp/syncthing/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build248314634=/tmp/go-build -gno-record-gcc-switches"

Running all tests with -coverpkg on all packages

git clone https://github.com/syncthing/syncthing.git
cd syncthing
go test -short -coverprofile coverage.out -coverpkg ./cmd/...,./lib/... ./cmd/... ./lib/...

results in errors like previously reported:

2019/05/15 16:02:50 duplicate symbol go.constinfo.main (types 45 and 45) in github.com/syncthing/syncthing/cmd/strelaypoolsrv and /home/simon/.cache/go-build/87/87afda35dba0b8de44e2f9d449eda7ce4caef7ed781612883b4e457eab1ee3d5-d(_go_.o)

The same also happens with just ./.... I cleaned out ~/.cache/go-build for good measure, didn't change anything.

To make sure it isn't some weird local problem of mine I also ran it on Syncthing's CI and reproduced the issue there as well: https://build.syncthing.net/viewLog.html?buildId=39242&buildTypeId=Syncthing_Coverage&tab=buildLog

@bcmills
Copy link
Contributor

bcmills commented May 20, 2019

@imsodin, please file a separate issue — there may be a similar bug involving the -coverprofile flag.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests