Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: executable created using linkshared option is not linking .so created by buildmode=shared with go modules enabled #38499

Closed
ramanvis opened this issue Apr 17, 2020 · 6 comments
Closed

cmd/go: executable created using linkshared option is not linking .so created by buildmode=shared with go modules enabled #38499

ramanvis opened this issue Apr 17, 2020 · 6 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@ramanvis
Copy link

What version of Go are you using (go version)?

root@71fe8ef0fc55:/go_src# go version
go version go1.14.2 linux/amd64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/go_src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build091221661=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Refer to: https://github.com/ramanvis/buildsharedwithmod
I did the following:

  1. Generated a shared library of my package using buildmode=shared for library "github.com/ramanvis/buildsharedwithmod/lib/advmath".
root@71fe8ef0fc55:/go_src# go install -buildmode=shared std
root@71fe8ef0fc55:/go_src# go install -buildmode=shared -linkshared github.com/ramanvis/buildsharedwithmod/lib/advmath
root@71fe8ef0fc55:/go_src# ls
Dockerfile  README.md  cmd  go.mod  go.sum  lib  libgithub.com-ramanvis-buildsharedwithmod-lib-advmath.so
  1. When I create my executable using linkshared option, then I do see executable created successfully.
root@71fe8ef0fc55:/go_src# go build -o /bin/advmath -linkshared github.com/ramanvis/buildsharedwithmod/cmd/advmath
  1. ldd of the the executable does not link to the shared library.

How to reproduce:

  1. Use this docker image to reproduce the problem:
    docker pull ramanvis/buildsharedwithmod:latest
  2. docker run --rm -it ramanvis/buildsharedwithmod:latest /bin/bash
  3. Execute the following commands:
go install -buildmode=shared std 
go install -buildmode=shared -linkshared github.com/ramanvis/buildsharedwithmod/lib/advmath
go build -o /bin/advmath -linkshared github.com/ramanvis/buildsharedwithmod/cmd/advmath
  1. Verify with ldd
ldd /bin/advmath

What did you expect to see?

I was expecting to see the binary being dynamically linked to libgithub.com-ramanvis-buildsharedwithmod-lib-advmath.so

root@71fe8ef0fc55:/go_src# ldd /bin/advmath
	linux-vdso.so.1 (0x00007fff1b9fc000)
	libstd.so => /usr/local/go/pkg/linux_amd64_dynlink/libstd.so (0x00007f267554e000)
        libgithub.com-ramanvis-buildsharedwithmod-lib-advmath.so => (/go_src/libgithub.com-ramanvis-buildsharedwithmod-lib-advmath.so)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2675389000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f2675384000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f2675363000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f2677c7b000)

What did you see instead?

root@71fe8ef0fc55:/go_src# ldd /bin/advmath
	linux-vdso.so.1 (0x00007fff1b9fc000)
	libstd.so => /usr/local/go/pkg/linux_amd64_dynlink/libstd.so (0x00007f267554e000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2675389000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f2675384000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f2675363000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f2677c7b000)
@andybons andybons changed the title executable created using linkshared option is not linking .so created by buildmode=shared with go modules enabled cmd/link: executable created using linkshared option is not linking .so created by buildmode=shared with go modules enabled Apr 17, 2020
@andybons andybons added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 17, 2020
@andybons andybons added this to the Unplanned milestone Apr 17, 2020
@andybons
Copy link
Member

@thanm @cherrymui

@thanm
Copy link
Contributor

thanm commented Apr 17, 2020

I took a look at this, testing with tip of master.

First off, please be aware that "-linkshared" and "-buildmode=shared" are very obscure, not a lot of people use them, so they are not as well tested and/or supported. There are know issues with building things this way (for example, linker DWARF generation is weak).

I believe the problem here is not with the linker, though, it looks as though it is with the Go command.

If I use the Go command in GOPATH mode (avoid use of modules), I can get your example to work properly. This is because when you do

go install -buildmode=shared -linkshared github.com/ramanvis/buildsharedwithmod/lib/advmath

the generated shared library is installed into ${GOPATH}/pkg/linux_amd64_dynlink, which the final link can then pick up.

Without GOPATH (e.g. in modules mode) the Go command seems to be just dropping the shared library into the current directory.

When the final link of the command takes place, the Go command is not picking up the shared library created in the previous step, it's just getting a regular archive from the cache. From the "-x" output:

packagefile github.com/ramanvis/buildsharedwithmod/lib/advmath=/usr/local/google/home/thanm/.cache/go-build/7b/7b01d210cc8dc2591258e60ee56bdf7b9b3d0283bc2ddb8d6f58cf01d11f327f-d

Somebody who knows the internals of the Go command (and module mode in particular) might be able to say more about how this is supposed to be handled, or even if it is supported at all.

Maybe @bcmills might be the right person to ask?

@bcmills
Copy link
Contributor

bcmills commented Apr 18, 2020

This might be a duplicate of #37675.

This TODO (from CL 208233, for #35759) is almost certainly relevant:

go/src/cmd/go/internal/load/pkg.go

Lines 1663 to 1665 in d5e1b7c

// TODO(bcmills): The reliance on p.Target implies that -linkshared does
// not work for any package that lacks a Target — such as a non-main
// package in module mode. We should probably fix that.

CC @jayconrod @matloob

@bcmills bcmills changed the title cmd/link: executable created using linkshared option is not linking .so created by buildmode=shared with go modules enabled cmd/go: executable created using linkshared option is not linking .so created by buildmode=shared with go modules enabled Apr 18, 2020
@bcmills bcmills modified the milestones: Unplanned, Backlog Apr 18, 2020
@ramanvis
Copy link
Author

Thanks for the really quick turnaround.

I too see that it works as expected when modules are turned off.

Not using modules is not an option. I guess "buildmode=shared" and "linkshared" would not work because of this issue

My intentions are to build a "shared" library and using the same in another go project, so that I could share without revealing the source. I could probably explore buildmode c-shared and use cgo. Could you let me know if there are any popular ways of doing the same?

@bcmills
Copy link
Contributor

bcmills commented Apr 20, 2020

Could you let me know if there are any popular ways of doing the same?

The fact that I couldn't find an issue to mark this as a duplicate of suggests that there is no “popular” way to do what you are trying to do. (You can't really “share without revealing the source” either way, since someone could always disassemble the library and reconstruct the original program logic — at that point, a go:generate tool for source obfuscation seems about as effective as a shared library, and potentially much more efficient.)

At any rate: the plugin API is the current way to build things that can plug in to (or accept plugins from) other Go programs, but it has its share of bugs too (such as #31278 and #31354).

@seankhliao
Copy link
Member

Obsoleted by #47788

@golang golang locked and limited conversation to collaborators Oct 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
6 participants
@andybons @bcmills @gopherbot @seankhliao @thanm @ramanvis