cmd/cgo, cmd/link: with zig as CC/CXX, relocation target not defined, fatal error, SIGSEGV when compiling net/http #44695
Comments
|
Can you confirm how are you building for windows and darwin? |
|
Yes, for Windows I get: Darwin actually did fail, it just wasn't clear. |
|
But, I think that my Windows example isn't actually using cgo; I did a |
|
There's a issue for it not working on windows. #43886 |
|
Yes, thanks. I had seen that, but I wanted to start with my Linux example. We can just ignore my Darwin and Windows comments 🙂 |
dmitshur
changed the title
dmitshur
added
the
NeedsInvestigation
|
Another datapoint: I can't use cgo+zig to cross-compile from amd64-linux to amd64-darwin: EDIT: created #46644. |
|
I encountered this error myself in the past few days and ended up looking into it. I think this is that
If I set the linkmode myself it works: |
|
Maybe I'm misunderstanding, but when I do Which is a real 2.9MB file that exists. |
|
That was a different problem I was seeing and unrelated to this, sorry. |
|
As @andrew-daly observed, this behaviour occurs when compiling binaries where cgo is enabled but C is not actually being used. For example, the following will fail to build: package main
import _ "runtime/cgo"
func main() {}However, simply importing the C package resolves the issue: package main
import "C"
import _ "runtime/cgo"
func main() {}Alternatively, configuring For context the host is For Bazel Should this issue be closed? It looks like we've found a resolution and appears to be more of a Zig thing, rather than Go? Please correct me if I'm wrong. Thank you all. |
|
@uhthomas it is unusual for user code to import runtime/cgo directly like that. I'm not familiar with Bazel, but if Bazel adds an import like that, I'd suggest we revisit that. In the past PIE linking requires cgo, but at least on some platforms (including linux/amd64 and darwin/amd64) that has not been the case quite a few releases ago, and the Go linker can link a PIE binary just fine (if cgo is not otherwise used). So if Bazel forces it to use cgo for PIE, maybe it shouldn't now. |
|
Apologies, I used It looks like |
|
Thanks. It looks like when external linking is used it works, but it doesn't work when it is not? Perhaps zcc produces object files that the Zig linker understands, but the Go linker doesn't? Does zcc produce standard ELF/Mach-O/PE/etc. object files, or it has its own custom object file? |
|
@cherrymui Sorry for the late reply, yes this is correct. Compilation with Zig CC is not possible without external linking. This may be expected behaviour, I'm not sure. Unfortunately I am not aware of the specifics of how Zig CC works but as I understand it shouldn't behave any differently than glibc or musl. |
When building object files, `zig cc` will instruct lld to remove unused sections via `--gc-sections`. This is problematic for cgo. Briefly, go builds cgo executables as follows*: 1. build `_cgo_.o`, which links (on linux_amd64 systems) to `/usr/local/go/src/runtime/race/race_linux_amd64.syso`. 2. That `.syso` contains references to symbols from libc. If the actual Go program uses at least one libc symbol, it will link correctly. However, if Go is building a cgo executable, but without any C code, the sections from `.syso` file will be garbage-collected, leaving a `_cgo_.o` without any references to libc. I assume the `gc_sections` is an optimization. If yes, then it makes sense for the final executable, but not for the intermediate object. If not, please correct me. Quoting @andrewrk in [1]: > The C source code for the temporary executable needs to have dummy > calls to getuid, pthread_self, sleep, and every other libc function > that cgo/race code wants to call. I agree in this case. However, while we could potentially fix it for go, I don't know how many other systems do that, which compilcates use of `zig cc` for other projects. If we consider `zig cc` a drop-in clang replacement (except for `-fsanitize=undefined`, which I tend to agree with), then it should not be optimizing the intermediate object files. I assume this was added as an optimization. If that's correct, let's optimize the final executable, but not the intermediate objects. Fixes ziglang#11398 Fixes golang/go#44695 Fixes golang/go#52690 [*]: Empirically observed with `CGO_ENABLED=1 go test -race -x -v` [1]: golang/go#52690 (comment)
|
I created ziglang/zig#11577 for further discussion and possible resolution. With today's zig and go 1.18 you can compile the original file as such: |
When building object files, `zig cc` will instruct lld to remove unused sections via `--gc-sections`. This is problematic cgo builds that don't explicitly use C code. Briefly, go builds cgo executables as follows*: 1. build `_cgo_.o`, which links (on linux_amd64 systems) to `/usr/local/go/src/runtime/race/race_linux_amd64.syso`. 2. That `.syso` contains references to symbols from libc. If the user program uses at least one libc symbol, it will link correctly. However, if Go is building a cgo executable, but without C code, the sections from `.syso` file will be garbage-collected, leaving a `_cgo_.o` without any references to libc, causing the final linking step to not link libc. Until now, this could be worked around by `-linkmode external` flag to `go build`. This causes Go to link the final executable using the external linker (which implicitly links libc). However, that flag brings in a whole different world of worms. I assume the `gc_sections` is an optimization; I tried to re-add `--gc-sections` to the final executable, but that didn't go well. I know removing such an optimization may be contentious, so let's start the discussion here. Quoting @andrewrk in [1] (it was about `--as-needed`, but the point remains the same): > The C source code for the temporary executable needs to have dummy > calls to getuid, pthread_self, sleep, and every other libc function > that cgo/race code wants to call. I agree this is how it *should* work. However, while we could fix it for go, I don't know how many other systems rely on that, and we'll never know we've fixed the last one. The point is, GCC/Clang does not optimize sections by default, and downstream tools rely on that. If we want to consider `zig cc` a drop-in clang replacement (except for `-fsanitize=undefined`, which I tend to agree with), then it should not be optimizing the intermediate object files. Or at least have a very prominent fine-print that this is happening, with ways to work around it. Fixes ziglang#11398 Fixes golang/go#44695 Fixes golang/go#52690 [*]: Empirically observed with `CGO_ENABLED=1 go test -race -x -v` [1]: golang/go#52690 (comment)
When building object files, `zig cc` will instruct lld to remove unused sections via `--gc-sections`. This is problematic cgo builds that don't explicitly use C code. Briefly, go builds cgo executables as follows*: 1. build `_cgo_.o`, which links (on linux_amd64 systems) to `/usr/local/go/src/runtime/race/race_linux_amd64.syso`. 2. That `.syso` contains references to symbols from libc. If the user program uses at least one libc symbol, it will link correctly. However, if Go is building a cgo executable, but without C code, the sections from `.syso` file will be garbage-collected, leaving a `_cgo_.o` without any references to libc, causing the final linking step to not link libc. Until now, this could be worked around by `-linkmode external` flag to `go build`. This causes Go to link the final executable using the external linker (which implicitly links libc). However, that flag brings in a whole different world of worms. I assume the `gc_sections` is an optimization; I tried to re-add `--gc-sections` to the final executable, but that didn't go well. I know removing such an optimization may be contentious, so let's start the discussion here. Quoting @andrewrk in [1] (it was about `--as-needed`, but the point remains the same): > The C source code for the temporary executable needs to have dummy > calls to getuid, pthread_self, sleep, and every other libc function > that cgo/race code wants to call. I agree this is how it *should* work. However, while we could fix it for go, I don't know how many other systems rely on that, and we'll never know we've fixed the last one. The point is, GCC/Clang does not optimize sections by default, and downstream tools rely on that. If we want to consider `zig cc` a drop-in clang replacement (except for `-fsanitize=undefined`, which I tend to agree with), then it should not be optimizing the intermediate object files. Or at least have a very prominent fine-print that this is happening, with ways to work around it. Fixes ziglang#11398 Fixes golang/go#44695 Fixes golang/go#52690 [*]: Empirically observed with `CGO_ENABLED=1 go test -race -x -v` [1]: golang/go#52690 (comment)
|
FWIW @motiejus even with Using PIE seems to fix things as usual. |
We have updated bazel-zig-cc in our monorepo and can't see this error any more. Please bring this up in the mailing list, so we can investigate this properly. I am very interested on why it would still be failing. If you can reproduce it like in this test, it would give most data (and a ready test case to put to bazel-zig-cc). |
What version of Go are you using (
go version)?Does this issue reproduce with the latest release?
Yes, on tip too.
What operating system and processor architecture are you using (
go env)?go envOutputWhat did you do?
In my PATH, I have two scripts to work around #43078:
Then, I compiled with
CC="zcc" CXX="zxx" go build.What did you expect to see?
A successful compilation.
What did you see instead?
If I change my code to just import
netonly, the panic goes away, but it still fails:I don't quite know the issue prefix for the title, or if this is two issues together (the relocation failing and the linker panicking/segfaulting). It's not clear to me if this is an issue with Go or with zig either; others have gotten this to work just fine (given the other thread and blog posts about it), but I have not.
Compiling with GOOS=windows or GOOS=darwin with this setup works; only Linux appears to have troubles.The text was updated successfully, but these errors were encountered: