Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: diamond CGO dependency produces a symbol collision #16463

Closed
colinmarc opened this issue Jul 22, 2016 · 3 comments
Closed

cmd/go: diamond CGO dependency produces a symbol collision #16463

colinmarc opened this issue Jul 22, 2016 · 3 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Go1.8

Comments

@colinmarc
Copy link

colinmarc commented Jul 22, 2016 •
edited

Please answer these questions before submitting your issue. Thanks!

  1. What version of Go are you using (go version)?

    go version go1.6.3 darwin/amd64

  2. What operating system and processor architecture are you using (go env)?

    GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/colin/golang"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GO15VENDOREXPERIMENT="1"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common"
CXX="clang++"
CGO_ENABLED="1"

  3. What did you do?

  4. What did you expect to see?

    Successful build output!

  5. What did you see instead?

    It's failing to build on the last step with duplicate symbols:

      duplicate symbol _snappy_compress in:
      /var/folders/3d/ymqb8kcx5jv4s2gsqy7rd83h0000gn/T/go-link-410495653/000000.o
      /var/folders/3d/ymqb8kcx5jv4s2gsqy7rd83h0000gn/T/go-link-410495653/000003.o

I can also reproduce on linux; here's a travis build.

The issue seems to be that I have a diamond dependency on a local static library. I vendor and build a version of snappy, which is in turn used by sparkey, which I also build, and that is in turn used by a vendored version of go-sparkey, which is used in my app.

That had all been working fine (you can check out the 1.0 branch to see it in action). The issue arose when I introduced another dependency on snappy, in sequencefile/snappy.go. Then the build started breaking.

To summarize, the dependency graph looks like this (see the Makefile for reference):

Everything was working before I added the brach on the right; the linked commit contains the full set of changes that break the build.

Thanks in advance for the help! Sorry this is gnarly.
@ianlancetaylor ianlancetaylor changed the title Diamond CGO dependency produces a symbol collision cmd/go: diamond CGO dependency produces a symbol collision Jul 22, 2016
@ianlancetaylor ianlancetaylor added this to the Go1.8 milestone Jul 22, 2016
@quentinmit quentinmit added the NeedsFix The path to resolution is known, but the work has not been done. label Oct 6, 2016
@rsc
Copy link
Contributor

rsc commented Oct 21, 2016

I think the problem is the CGO_LDFLAGS being set, as seen in the Travis log:

CGO_LDFLAGS="/home/travis/gopath/src/github.com/stripe/sequins/build/lib/libsparkey.a /home/travis/gopath/src/github.com/stripe/sequins/build/lib/libsnappy.a /home/travis/gopath/src/github.com/stripe/sequins/build/lib/libzookeeper_mt.a -lrt -lm -lstdc++"

One step of the cgo build is to take the .o files that have been generated so far for the package (containing the Go-specific wrappers) and merge them into one by using (I am simplifying a little):

$CC -o _all.o *.o $CGO_LDFLAGS

The $CGO_LDFLAGS in this invocation has -l arguments stripped out, precisely to avoid this problem, but not explicit paths like foo.a. I believe that if you change your CGO_LDFLAGS to:

CGO_LDFLAGS="-L/home/travis/gopath/src/github.com/stripe/sequins/build/lib -lsparkey -lsnappy -lzookeeper_mt -lrt -lm -lstdc++"

then your build should work. We know to remove *.dylib, *.so, and *.dll from the CGO_LDFLAGS. Perhaps we should also remove *.a.

@ianlancetaylor, what do you think about removing *.a in src/cmd/go/build.go's builder.collect?

@ianlancetaylor
Copy link
Contributor

I removing .o and .a files from CGO_LDFLAGS in builder.collect makes sense.

@gopherbot
Copy link

CL https://golang.org/cl/32642 mentions this issue.

@golang golang locked and limited conversation to collaborators Nov 3, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Go1.8
Development

No branches or pull requests
5 participants
@rsc @quentinmit @colinmarc @ianlancetaylor @gopherbot