cmd/link: cross compile from MacOS to Windows with CGO_ENABLED=1 and -buildmode=c-archive not working #59221

awsong · 2023-03-24T14:37:28Z

What version of Go are you using (`go version`)?

$ go version
go version go1.20.2 darwin/arm64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/wsong/Library/Caches/go-build"
GOENV="/Users/wsong/Library/Application Support/go/env"
GOEXE=".exe"
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/wsong/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="windows"
GOPATH="/Users/wsong/go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.20.2"
GCCGO="gccgo"
GOAMD64="v1"
AR="x86_64-w64-mingw32-gcc"
CC="x86_64-w64-mingw32-gcc"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/wsong/Development/test/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/var/folders/lr/s8_5072s6cs602ksv8tv5myh0000gn/T/go-build2849677897=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I have following files in my folder:

example/ go.mod   test.c   test.go
./example:
main.c

wsong@Work: ~/Development/test $ cat go.mod
module test

go 1.20
wsong@Work: ~/Development/test $ cat test.c
// Functions exported by Go.
extern int Test();

int test() {
	Test();
	return 0;
}
wsong@Work: ~/Development/test $ cat test.go
package main

//#include "errno.h"
import "C"

func main() {}

//export Test
func Test() C.int {
	return 0
}

wsong@Work: ~/Development/test $ cat example/main.c 
extern int test();

int main(void) {
	test();
	return 0;
}

I run go build like this:

CGO_ENABLED=1 CC=x86_64-w64-mingw32-gcc AR=x86_64-w64-mingw32-gcc GOOS=windows GOARCH=amd64 go build -v -a -buildmode=c-archive .

Then go into example folder and run:

x86_64-w64-mingw32-gcc main.c ../test.a

What did you expect to see?

a.exe is generated.

What did you see instead?

/opt/homebrew/Cellar/mingw-w64/10.0.0_5/toolchain-x86_64/bin/x86_64-w64-mingw32-ld: /var/folders/lr/s8_5072s6cs602ksv8tv5myh0000gn/T//ccHcSIoc.o:main.c:(.text+0xe): undefined reference to `test'
collect2: error: ld returned 1 exit status

The text was updated successfully, but these errors were encountered:

heschi · 2023-03-24T17:26:33Z

cc @golang/compiler

cherrymui · 2023-03-24T18:44:20Z

What version of C toolchain do you use? Thanks.

cc @thanm

awsong · 2023-03-24T23:23:42Z

It's mingw-w64 from homebrew, on M1 Mac.

wsong@Work: ~/Development/test/example $ x86_64-w64-mingw32-gcc -v
Using built-in specs.
COLLECT_GCC=x86_64-w64-mingw32-gcc
COLLECT_LTO_WRAPPER=/opt/homebrew/Cellar/mingw-w64/10.0.0_5/toolchain-x86_64/libexec/gcc/x86_64-w64-mingw32/12.2.0/lto-wrapper
Target: x86_64-w64-mingw32
Configured with: ../configure --target=x86_64-w64-mingw32 --with-sysroot=/opt/homebrew/Cellar/mingw-w64/10.0.0_5/toolchain-x86_64 --prefix=/opt/homebrew/Cellar/mingw-w64/10.0.0_5/toolchain-x86_64 --with-bugurl=https://github.com/Homebrew/homebrew-core/issues --enable-languages=c,c++,fortran --with-ld=/opt/homebrew/Cellar/mingw-w64/10.0.0_5/toolchain-x86_64/bin/x86_64-w64-mingw32-ld --with-as=/opt/homebrew/Cellar/mingw-w64/10.0.0_5/toolchain-x86_64/bin/x86_64-w64-mingw32-as --with-gmp=/opt/homebrew/opt/gmp --with-mpfr=/opt/homebrew/opt/mpfr --with-mpc=/opt/homebrew/opt/libmpc --with-isl=/opt/homebrew/opt/isl --with-zstd=no --disable-multilib --disable-nls --enable-threads=posix
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 12.2.0 (GCC)

thanm · 2023-03-27T14:12:48Z

@awsong Thanks for the report.

If you could dump out the symbols from your test.a and post it here, that would be helpful, e.g. something like

$ CGO_ENABLED=1 go build -buildmode=c-archive 
$ llvm-objdump-14 -t test.a | fgrep test | fgrep .text 
0000000000011200 l     F .text	00000000000000ef runtime.gcTrigger.test
000000000003ade0 l     F .text	00000000000001f0 runtime.testAtomic64
0000000000000000 g     F .text	000000000000000c test
$

awsong · 2023-03-27T15:05:59Z

wsong@Work: ~/Development/test $ x86_64-w64-mingw32-objdump -t test.a | fgrep test | fgrep .text 
wsong@Work: ~/Development/test $ x86_64-w64-mingw32-objdump -t test.a | fgrep test 
In archive test.a:
[228](sec  1)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000012900 runtime.gcTrigger.test
[697](sec  1)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x000000000003e620 runtime.testAtomic64
[1544](sec  4)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000053678 runtime.test_z64
[1545](sec  4)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x0000000000053670 runtime.test_x64
[1569](sec  4)(fl 0x00)(ty    0)(scl   2) (nx 0) 0x00000000000534a9 runtime.testingWER
[  0](sec -2)(fl 0x00)(ty    0)(scl 103) (nx 1) 0x0000000000000000 test.cgo2.c
[  0](sec -2)(fl 0x00)(ty    0)(scl 103) (nx 1) 0x0000000000000000 test.c
[  2](sec  1)(fl 0x00)(ty   20)(scl   2) (nx 1) 0x0000000000000000 test

thanm · 2023-03-27T15:13:46Z

Thanks. No red flags there, not immediately sure what the issue is. I'll work on setting up a test machine of some sort to see if I can repro. Stay tuned.

thanm · 2023-03-27T23:59:01Z

OK, I think I see what the issue is. I'll send a CL tomorrow morning.

If you need a workaround for the time being, you can do this:

$ ARPATH=`x86_64-w64-mingw32-gcc --print-prog-name ar`
$ CGO_ENABLED=1 CC=x86_64-w64-mingw32-gcc AR=x86_64-w64-mingw32-gcc GOOS=windows GOARCH=amd64 go build -ldflags="-extar=$ARPATH" -buildmode=c-archive .
$

Problem here is that the Go linker is invoking the system "ar" instead of the cross-compiler's "ar".

awsong · 2023-03-28T00:06:57Z

Verified that the workaround works on my system.

Thanks

gopherbot · 2023-03-28T13:27:40Z

Change https://go.dev/cl/479775 mentions this issue: cmd/link: use "ar" path derived from querying CC for -buildmode=c-archive

…mode When external linking with -buildmode=c-archive, the Go linker eventually invokes the "ar" tool to create the final archive library. Prior to this patch, if the '-extar' flag was not in use, we would just run "ar". This works well in most cases but breaks down if we're doing cross-compilation targeting Windows (macos system "ar" apparently doesn't create the windows symdef section correctly). To fix the problem, capture the output of "cc --print-prog-name ar" and invoke "ar" using the path returned by that command. Fixes golang#59221. Change-Id: I9de66e98947c42633b16fde7208c2958d62fe7cc Reviewed-on: https://go-review.googlesource.com/c/go/+/479775 Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Than McIntosh <thanm@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>

thanm · 2023-03-28T20:04:15Z

Hmm, I've had to send a revert for https://go.dev/cl/479775 since it causes a failure on our ios-arm64-corellium builder... hopefully after I figure out what is going on I can resubmit CL 479775.

gopherbot · 2023-03-30T19:23:34Z

Change https://go.dev/cl/480675 mentions this issue: env/corellium/ios: special case "--print-prog-name=ar" in clang wrapper

Intercept the "--print-prog-name=ar" flag in the clang wrapper and return "ar" from the system path, as opposed to the "ar" selected by clang. This is needed to avoid bypassing the ar wrapper installed in /var/root/bin, which is the one we want. Updates golang/go#59221. Change-Id: I907a6d6be2e6439901b36d6f7f8669939fd82e94 Reviewed-on: https://go-review.googlesource.com/c/build/+/480675 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>

gopherbot · 2023-04-25T13:16:37Z

Change https://go.dev/cl/488576 mentions this issue: cmd/link: use path from "cc --print-prog-name ar" for c-archive buildmode

mknyszek · 2023-06-09T18:33:42Z

Hey @thanm, any update here? Do you still plan to do this for Go 1.21? Thanks.

thanm · 2023-06-09T18:46:33Z

I've moved to backlog. The main sticking points here are the problems with the ios correllium builders-- needs more investigation as to why the change doesn't work with their custom wrappers.

cherrymui · 2023-06-09T19:27:58Z

If we want to do something for this cycle, we could do it only for Windows.

awsong changed the title ~~affected/package: Cross compile from MacOS to Windows with CGO_ENABLED and -buildmode=c-archive not working~~ affected/package: Cross compile from MacOS to Windows with CGO_ENABLED=1 and -buildmode=c-archive not working Mar 24, 2023

heschi changed the title ~~affected/package: Cross compile from MacOS to Windows with CGO_ENABLED=1 and -buildmode=c-archive not working~~ cmd/compile: cross compile from MacOS to Windows with CGO_ENABLED=1 and -buildmode=c-archive not working Mar 24, 2023

heschi added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 24, 2023

heschi added this to the Go1.21 milestone Mar 24, 2023

gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Mar 24, 2023

thanm self-assigned this Mar 27, 2023

thanm changed the title ~~cmd/compile: cross compile from MacOS to Windows with CGO_ENABLED=1 and -buildmode=c-archive not working~~ cmd/link: cross compile from MacOS to Windows with CGO_ENABLED=1 and -buildmode=c-archive not working Mar 28, 2023

thanm added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Mar 28, 2023

gopherbot closed this as completed in 422f448 Mar 28, 2023

cherrymui reopened this Mar 28, 2023

thanm modified the milestones: Go1.21, Backlog Jun 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/link: cross compile from MacOS to Windows with CGO_ENABLED=1 and -buildmode=c-archive not working #59221

cmd/link: cross compile from MacOS to Windows with CGO_ENABLED=1 and -buildmode=c-archive not working #59221

awsong commented Mar 24, 2023

heschi commented Mar 24, 2023

cherrymui commented Mar 24, 2023

awsong commented Mar 24, 2023 •

edited

thanm commented Mar 27, 2023

awsong commented Mar 27, 2023

thanm commented Mar 27, 2023

thanm commented Mar 27, 2023

awsong commented Mar 28, 2023 •

edited

gopherbot commented Mar 28, 2023

thanm commented Mar 28, 2023

gopherbot commented Mar 30, 2023

gopherbot commented Apr 25, 2023

mknyszek commented Jun 9, 2023

thanm commented Jun 9, 2023

cherrymui commented Jun 9, 2023

cmd/link: cross compile from MacOS to Windows with CGO_ENABLED=1 and -buildmode=c-archive not working #59221

cmd/link: cross compile from MacOS to Windows with CGO_ENABLED=1 and -buildmode=c-archive not working #59221

Comments

awsong commented Mar 24, 2023

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

heschi commented Mar 24, 2023

cherrymui commented Mar 24, 2023

awsong commented Mar 24, 2023 • edited

thanm commented Mar 27, 2023

awsong commented Mar 27, 2023

thanm commented Mar 27, 2023

thanm commented Mar 27, 2023

awsong commented Mar 28, 2023 • edited

gopherbot commented Mar 28, 2023

thanm commented Mar 28, 2023

gopherbot commented Mar 30, 2023

gopherbot commented Apr 25, 2023

mknyszek commented Jun 9, 2023

thanm commented Jun 9, 2023

cherrymui commented Jun 9, 2023

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

awsong commented Mar 24, 2023 •

edited

awsong commented Mar 28, 2023 •

edited