Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: allow configuration of dsymutil and strip for darwin/mach-o linking #47316

Closed
chancila opened this issue Jul 21, 2021 · 8 comments
Closed
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@chancila
Copy link

chancila commented Jul 21, 2021

What version of Go are you using (go version)?

1.16.6

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/chancila/.cache/go-build"
GOENV="/home/chancila/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/chancila/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/chancila/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/snap/go/7954"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/snap/go/7954/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.6"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1637780390=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I am cross-compiling a darwin executable from linux with cgo enabled. I have successfully compiled everything but the link step fails due to xcrun missing since the go link command assumes xcrun exists if targeting darwin/mach when it really should probably only assume that if targeting darwin and host is darwin. I'm currently planning to work around this by creating a shim xcrun however having the ability to directly set these tools would be cleaner and easier to integrate with other build systems (namely bazel which I'm using in this case). The underlying tools have equivalents in open source llvm so they are not exclusive to macos hosts.

The code in question https://github.com/golang/go/blob/master/src/cmd/link/internal/ld/lib.go#L1643 should allow configuration of the tool in some way so that this use-case can be covered.

What did you expect to see?

Successful build

What did you see instead?

Linker error from ld/lib.go: Exitf("%s: running dsymutil failed: %v\n%s", os.Args[0], err, out) due to missing xcrun

@thanm thanm added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 21, 2021
@thanm thanm added this to the Backlog milestone Jul 21, 2021
@thanm
Copy link
Contributor

thanm commented Jul 21, 2021

@cherrymui

@thanm
Copy link
Contributor

thanm commented Jul 21, 2021

If as you say you are planning on writing a shim xcrun, would this not take care of the problems you describe?

Putting additoinal knobs/controls into the linker would mainly just make it more complicated and hard to understand (for example, for the 99% of folks not doing cross compilation, they might think that they need to specify a dsymutil path, or something to this effect).

@chancila
Copy link
Author

chancila commented Jul 21, 2021

I mean it would solve the issue in a very annoying way, in general this is a somewhat odd inconsistency given how the rest of the external c toolchain tools are configurable and hardcoding these tools kind of makes the toolchain non-hermetic, or at least requires some very non-obvious work to make hermetic, even if host is also darwin.

@chancila
Copy link
Author

trying to force xcrun in the path, I'm now running into another issue... #43783

due to how bazel works pretty much everything is relative to a execution root...so the go invocation sees that the xcrun which is added in the c++ toolchain is relative and complains...

@cherrymui
Copy link
Member

@chancila could you share more information about your C cross toolchain? What tools are included and how are they named? Thanks.

We used to just run dsymutil and strip from PATH. Some user reported that they installed GNU tools to PATH and it doesn't work.

Maybe we could use $CC --print-prog-name dsymutil. Would that work for you?

@cherrymui cherrymui changed the title allow configuration of dsymutil and strip for darwin/mach-o linking cmd/link: allow configuration of dsymutil and strip for darwin/mach-o linking Jul 22, 2021
@chancila
Copy link
Author

chancila commented Jul 22, 2021

@cherrymui

  • hosted on linux (ubuntu 20.04)
  • targeting macos (amd64 and arm64)
  • plain llvm (12) release (from official llvm releases on github)
  • linux hosted apple ld64 (from https://github.com/tpoechtrager/cctools-port)
  • apple macos sysroot from xcode 12.1 (and 12.5 for arm64)

I'm exercising this whole thing through bazel and rules_go...the fundamental issue is that the go linker tries to use xcrun when targeting mach binaries but it probably needs a fallback path when the linker/toolchain is hosted on non-macos systems rather than blindly using xcrun.

I think --print-prog-name would work fine actually, verifying with my toolchain it does the right thing.

In this scenario the failures are xcrun dsymutil and xcrun strip

I have successfully managed to cgo cross compile for both arm64 and amd64 by adding a shim xcrun wrapper in my system to /usr/bin globally as a hack, I would prefer to not have to do this as the toolchain is otherwise hermetic to the build and handled by bazel without modifying the system tools at all.

@cherrymui
Copy link
Member

@chancila thanks! I think I'd go with --print-prog-name.

@gopherbot
Copy link

Change https://golang.org/cl/336769 mentions this issue: cmd/link: use "CC --print-prog-name" to locate tools

@golang golang locked and limited conversation to collaborators Sep 30, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

4 participants