cmd/go: #cgo pkg-config: add conditional --static flag to pkg-config #12058
Comments
ianlancetaylor
changed the title
|
It's not immediately obvious to me how we can know whether we are going to be linking statically or linking dynamically. Are you suggesting that we look for the presence of --static in the -ldflags option? |
|
@ianlancetaylor That is the biggest obstacle to me as I'm not an expert, that's why I filed an issue rather than sent a pull request. I looked around for ways to tell go to produce a static binary with cgo or gccgo, and it looks like they all end up with passing -static flag. |
|
At the very least, it makes sense to add |
|
One example I just came across: gosqlite package which says So, it's all there they way it should be. One would thing that everything that is needed to fix it would be this: But unfortunately it won't work either for static linking case. One either needs to add If there is any other way to fix it, please let me know. I am only aware of an overlinking workaround, which is to pass -Wl,--as-needed, but with it parameters to linker needs to be in a proper order ($FLAGS $OBJECTS $LIBS), and it can also break some specific cases. |
TL;DR: stop building static binary that may fail Linker flag --unresolved-symbols=ignore-in-shared-libs was added in commit 06d0843 two years ago for the static build case, presumably to avoid dealing with problem of missing libraries. For the record, this is what ld(1) man page says: > --unresolved-symbols=method > Determine how to handle unresolved symbols. There are four > possible values for method: > ......... > ignore-in-shared-libs > Report unresolved symbols that come from regular object files, > but ignore them if they come from shared libraries. This can > be useful when creating a dynamic binary and it is known that > all the shared libraries that it should be referencing are > included on the linker's command line. Here, the flag is not used for its purpose ("creating a dynamic binary") and does more harm than good. Instead of complaining about missing symbols as it should do if some libraries are missing from LIBS/LDFLAGS, it lets ld create a binary with unresolved symbols, ike this: $ readelf -s bundles/1.7.1/binary/docker-1.7.1 | grep -w UND ........ 21029: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND dlopen ......... Such binary is working just fine -- until code calls one of those functions, then it crashes (for apparently no reason, i.e. it is impossible to tell why from the diagnistics printed). In other words, adding this flag allows to build a static binary with missing libraries, hiding the problem from both a developer (who forgot to add a library to #cgo: LDFLAGS -- I was one such developer a few days ago when I was working on ploop graphdriver) and from a user (who expects the binary to work without crashing, and it does that until the code calls a function in one of those libraries). Removing the flag immediately unveils the problem (as it should): /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libsqlite3.a(sqlite3.o): In function `unixDlError': (.text+0x20971): undefined reference to `dlerror' /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libsqlite3.a(sqlite3.o): In function `unixDlClose': (.text+0x8814): undefined reference to `dlclose' The problem is, gosqlite package says: #cgo LDFLAGS: -lsqlite3 which is enough for dynamic linking, as indirect dependencies (i.e. libraries required by libsqlite3.so) are listed in .so file and will be resolved dynamically by ldd upon executing the binary. For static linking though, one has to list all the required libraries, both direct and indirect. For libraries with pkgconfig support the list of required libraries can be obtained with pkg-config: $ pkg-config --libs sqlite3 # dynamic linking case -lsqlite3 $ pkg-config --libs --static sqlite3 # static case -lsqlite3 -ldl -lpthread It seems that all one has to do is to fix gosqlite this way: -#cgo LDFLAGS: -lsqlite3 +#cgo pkg-config: sqlite3 Unfortunately, cmd/go doesn't know that it needs to pass --static flag to pkg-config in case of static linking (see golang/go#12058). So, for one, one has to do one of these things: 1. Patch sqlite.go like this: -#cgo LDFLAGS: -lsqlite3 +#cgo pkg-config: --static sqlite3 (this is exactly what I do in goploop, see kolyshkin/goploop@e9aa072f51) 2. Patch sqlite.go like this: -#cgo LDFLAGS: -lsqlite3 +#cgo LDFLAGS: -lsqlite3 -ldl -lpthread (I would submit this patch to gosqlite but it seems that https://code.google.com/p/gosqlite/ is deserted and not maintained, and patching it here is not right as it is "vendored") 3. Explicitly add -ldl for the static link case. This is what this patch does. 4. Fork sqlite to github and maintain it there. Personally I am not ready for that, as I'm neither a Go expert nor gosqlite user. Now, moby#3 doesn't look like a clear solution, but nevertheless it makes the build much better than it was before. Signed-off-by: Kir Kolyshkin <kir@openvz.org>
TL;DR: stop building static binary that may fail Linker flag --unresolved-symbols=ignore-in-shared-libs was added in commit 06d0843 two years ago for the static build case, presumably to avoid dealing with problem of missing libraries. For the record, this is what ld(1) man page says: > --unresolved-symbols=method > Determine how to handle unresolved symbols. There are four > possible values for method: > ......... > ignore-in-shared-libs > Report unresolved symbols that come from regular object files, > but ignore them if they come from shared libraries. This can > be useful when creating a dynamic binary and it is known that > all the shared libraries that it should be referencing are > included on the linker's command line. Here, the flag is not used for its purpose ("creating a dynamic binary") and does more harm than good. Instead of complaining about missing symbols as it should do if some libraries are missing from LIBS/LDFLAGS, it lets ld create a binary with unresolved symbols, ike this: $ readelf -s bundles/1.7.1/binary/docker-1.7.1 | grep -w UND ........ 21029: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND dlopen ......... Such binary is working just fine -- until code calls one of those functions, then it crashes (for apparently no reason, i.e. it is impossible to tell why from the diagnistics printed). In other words, adding this flag allows to build a static binary with missing libraries, hiding the problem from both a developer (who forgot to add a library to #cgo: LDFLAGS -- I was one such developer a few days ago when I was working on ploop graphdriver) and from a user (who expects the binary to work without crashing, and it does that until the code calls a function in one of those libraries). Removing the flag immediately unveils the problem (as it should): /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libsqlite3.a(sqlite3.o): In function `unixDlError': (.text+0x20971): undefined reference to `dlerror' /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libsqlite3.a(sqlite3.o): In function `unixDlClose': (.text+0x8814): undefined reference to `dlclose' The problem is, gosqlite package says: #cgo LDFLAGS: -lsqlite3 which is enough for dynamic linking, as indirect dependencies (i.e. libraries required by libsqlite3.so) are listed in .so file and will be resolved dynamically by ldd upon executing the binary. For static linking though, one has to list all the required libraries, both direct and indirect. For libraries with pkgconfig support the list of required libraries can be obtained with pkg-config: $ pkg-config --libs sqlite3 # dynamic linking case -lsqlite3 $ pkg-config --libs --static sqlite3 # static case -lsqlite3 -ldl -lpthread It seems that all one has to do is to fix gosqlite this way: -#cgo LDFLAGS: -lsqlite3 +#cgo pkg-config: sqlite3 Unfortunately, cmd/go doesn't know that it needs to pass --static flag to pkg-config in case of static linking (see golang/go#12058). So, for one, one has to do one of these things: 1. Patch sqlite.go like this: -#cgo LDFLAGS: -lsqlite3 +#cgo pkg-config: --static sqlite3 (this is exactly what I do in goploop, see kolyshkin/goploop@e9aa072f51) 2. Patch sqlite.go like this: -#cgo LDFLAGS: -lsqlite3 +#cgo LDFLAGS: -lsqlite3 -ldl -lpthread (I would submit this patch to gosqlite but it seems that https://code.google.com/p/gosqlite/ is deserted and not maintained, and patching it here is not right as it is "vendored") 3. Explicitly add -ldl for the static link case. This is what this patch does. 4. Fork sqlite to github and maintain it there. Personally I am not ready for that, as I'm neither a Go expert nor gosqlite user. Now, moby#3 doesn't look like a clear solution, but nevertheless it makes the build much better than it was before. Signed-off-by: Kir Kolyshkin <kir@openvz.org>
|
@kolyshkin Sorry, I didn't mean to suggest that you should fix this. I'm trying to understand what you are suggesting. Let me try this: when you say "Docker can be linked either statically or dynamically" what do you mean? How does one decide to link Docker statically? How does one decide to link it dynamically? |
|
@ianlancetaylor sure! Looking into Docker build scripts, it's something like this (simplified for clarity): |
|
s/pgk-config/pkg-config/g |
kolyshkin
changed the title
|
Sure it would be easier just to do something like |
TL;DR: stop building static binary that may fail Linker flag --unresolved-symbols=ignore-in-shared-libs was added in commit 06d0843 two years ago for the static build case, presumably to avoid dealing with problem of missing libraries. For the record, this is what ld(1) man page says: > --unresolved-symbols=method > Determine how to handle unresolved symbols. There are four > possible values for method: > ......... > ignore-in-shared-libs > Report unresolved symbols that come from regular object files, > but ignore them if they come from shared libraries. This can > be useful when creating a dynamic binary and it is known that > all the shared libraries that it should be referencing are > included on the linker's command line. Here, the flag is not used for its purpose ("creating a dynamic binary") and does more harm than good. Instead of complaining about missing symbols as it should do if some libraries are missing from LIBS/LDFLAGS, it lets ld create a binary with unresolved symbols, ike this: $ readelf -s bundles/1.7.1/binary/docker-1.7.1 | grep -w UND ........ 21029: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND dlopen ......... Such binary is working just fine -- until code calls one of those functions, then it crashes (for apparently no reason, i.e. it is impossible to tell why from the diagnistics printed). In other words, adding this flag allows to build a static binary with missing libraries, hiding the problem from both a developer (who forgot to add a library to #cgo: LDFLAGS -- I was one such developer a few days ago when I was working on ploop graphdriver) and from a user (who expects the binary to work without crashing, and it does that until the code calls a function in one of those libraries). Removing the flag immediately unveils the problem (as it should): /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libsqlite3.a(sqlite3.o): In function `unixDlError': (.text+0x20971): undefined reference to `dlerror' /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libsqlite3.a(sqlite3.o): In function `unixDlClose': (.text+0x8814): undefined reference to `dlclose' The problem is, gosqlite package says: #cgo LDFLAGS: -lsqlite3 which is enough for dynamic linking, as indirect dependencies (i.e. libraries required by libsqlite3.so) are listed in .so file and will be resolved dynamically by ldd upon executing the binary. For static linking though, one has to list all the required libraries, both direct and indirect. For libraries with pkgconfig support the list of required libraries can be obtained with pkg-config: $ pkg-config --libs sqlite3 # dynamic linking case -lsqlite3 $ pkg-config --libs --static sqlite3 # static case -lsqlite3 -ldl -lpthread It seems that all one has to do is to fix gosqlite this way: -#cgo LDFLAGS: -lsqlite3 +#cgo pkg-config: sqlite3 Unfortunately, cmd/go doesn't know that it needs to pass --static flag to pkg-config in case of static linking (see golang/go#12058). So, for one, one has to do one of these things: 1. Patch sqlite.go like this: -#cgo LDFLAGS: -lsqlite3 +#cgo pkg-config: --static sqlite3 (this is exactly what I do in goploop, see kolyshkin/goploop@e9aa072f51) 2. Patch sqlite.go like this: -#cgo LDFLAGS: -lsqlite3 +#cgo LDFLAGS: -lsqlite3 -ldl -lpthread (I would submit this patch to gosqlite but it seems that https://code.google.com/p/gosqlite/ is deserted and not maintained, and patching it here is not right as it is "vendored") 3. Explicitly add -ldl for the static link case. This is what this patch does. 4. Fork sqlite to github and maintain it there. Personally I am not ready for that, as I'm neither a Go expert nor gosqlite user. Now, #3 doesn't look like a clear solution, but nevertheless it makes the build much better than it was before. Signed-off-by: Kir Kolyshkin <kir@openvz.org>
|
@ianlancetaylor please let me know what do you think about this suggestion |
|
Why not just add a new cgo file protected by a build tag, say, static?
// file static_cgo.go
// +build static
/*
#cgo pkgconfig: --static libxml-2.0
#cgo LDFLAGS: -static
*/
import "C"
And use go build -tags static will build a static binary.
|
|
@kolyshkin I don't like the idea of adding a -static option to go build. Minux's suggestion SGTM. I would be OK with adding a -pkgconfigflags option to go build. This would provide a list of options that would be passed to pkg-config. |
|
http://linux.die.net/man/1/pkg-config
Do you know what is |
|
Right: the pkg-config command takes a --static option; see the man page. I'm not sure what you mean by The reference to private libraries is because when linking statically you need to explicitly list all the libraries you need. Shared libraries include a list of their dependencies, so you only have to link against the main shared library. Static libraries do not include such a list, so you need to link against all of them. |
|
Adding |
No, these are totally different things.
In this context "private libraries" means "all the libraries that this library itself uses/needs". Let me give you an example. If we want to build something dynamically with libsqlite, we add Now, if we want to build something statically, we need to list all the libraries explicitly in So, for dynamic build of something with libsqlite, we need |
This sort of works for me. The only problem I see is a need to ask everyone who's using If the above is impossible, I guess we can close this issue. |
|
On Mon, Oct 5, 2015 at 5:25 PM, Kirill Kolyshkin notifications@github.com
see #12058 (comment) |
Go (well, cgo) supports pkg-config directive so it's easier to figure out which libs we need to link against. Note that in case of static linking the list of libraries is larger (as we also need to include the libraries needed by libraries we use). Fortunately, there is --static flag to pkg-config to aim in this. Unfortunately, cgo doesn't know whether to pass this flag or not, neither do we, so we have to ask the user to indicate they are building statically by providing "static" build flag. This solves the problem of "overlinking" the non-static binary. Before: $ objdump -p goploop.test | grep NEEDED NEEDED libploop.so.1 NEEDED librt.so.1 NEEDED libpthread.so.0 NEEDED libxml2.so.2 NEEDED libz.so.1 NEEDED liblzma.so.5 NEEDED libm.so.6 NEEDED libc.so.6 After: $ objdump -p goploop.test | grep NEEDED NEEDED libploop.so.1 NEEDED libpthread.so.0 NEEDED libc.so.6 So, the libraries that are used by libploop are not explicitly listed now. For a discussion leading to this change, please see golang/go#12058 Signed-off-by: Kir Kolyshkin <kir@openvz.org>
@minux Sorry, I overlooked it. Looks like it's a good way to go! |
It's simpler than I initially thought! We just have to use -tags static, there's no need to pass linker flags as we can set LDFLAGS for cgo right from go source file. See golang/go#12058 (comment) for discussion about it. Signed-off-by: Kir Kolyshkin <kir@openvz.org>
bcmills
added
the
NeedsInvestigation
|
#26492 has the label Proposal-Accepted. Does that proposal subsume this one? |
|
When building a static binary, the osusergo build-tag should be used so that a pure go implementation of the `os/user` package is used. relates to: - golang/go#23265 os/user: add build tags to select cgo-vs-go implementation, like net package - golang/go@62f0127 - golang/go#24787 os/user: LookupId panics on Linux+glibc static build - golang/go#26492 cmd/go: build: add -static flag - golang/go#12058 cmd/go: #cgo pkg-config: add conditional --static flag to pkg-config Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When building a static binary, the osusergo build-tag should be used so that a pure go implementation of the `os/user` package is used. relates to: - golang/go#23265 os/user: add build tags to select cgo-vs-go implementation, like net package - golang/go@62f0127 - golang/go#24787 os/user: LookupId panics on Linux+glibc static build - golang/go#26492 cmd/go: build: add -static flag - golang/go#12058 cmd/go: #cgo pkg-config: add conditional --static flag to pkg-config - moby/moby#39994 homedir: add cgo or osusergo buildtag constraints for unix Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When building a static binary, the osusergo build-tag should be used so that a pure go implementation of the `os/user` package is used. relates to: - golang/go#23265 os/user: add build tags to select cgo-vs-go implementation, like net package - golang/go@62f0127 - golang/go#24787 os/user: LookupId panics on Linux+glibc static build - golang/go#26492 cmd/go: build: add -static flag - golang/go#12058 cmd/go: #cgo pkg-config: add conditional --static flag to pkg-config - moby/moby#39994 homedir: add cgo or osusergo buildtag constraints for unix Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
I found this case when working on Docker. Docker can be linked either statically or dynamically. For static linking, one has to specify all the libraries explicitly, including those required indirectly (i.e. by libraries that you use). For dynamic linking, one needs to specify only the libraries one uses directly. For example, when linking against libxml2 on Linux, for the case of dynamic linking you have to specify only -lxml2, while for the static linking case you have to also specify all the libraries used by libxml2 (like -lz -lm -llzma).
Fortunately, pkg-config supports this:
I was expecting Go to add --static flag when the result is going to be statically linked, unfortunately it never happens. My current workaround is to do this:
It solves my issues, but the problem with this approach is this leads to "overlinking" in the dynamic case, as described e.g. here: https://wiki.openmandriva.org/en/Overlinking_issues_in_packaging
Is it possible to implement conditionally adding --static to pkg-config invocation?
The text was updated successfully, but these errors were encountered: