cmd/go: cgo builds don't work with "zig cc" #43078

andrewrk · 2020-12-08T19:03:54Z

Hello, I am filing this issue on behalf of an issue reported upstream here: ziglang/zig#7342

@lu4p writes:

I would like to use "zig cc" for easily cross compiling cgo dependent go code, however the "drop-in" c compiler replacement doesn't work.

Example cgo code

main.go

package main

//int Add(int a, int b){
//    return a+b;
//}
import "C"
import "fmt"

func main() {
	a := C.int(10)
	b := C.int(20)
	c := C.Add(a, b)
	fmt.Println(c) // 30
}

other compilers

go build main.go //build works fine (uses gcc)
CC="clang" go build main.go //build works fine (uses clang)

zig cc

$ CC="zig cc" go build main.go
# runtime/cgo
info: Usage: zig [command] [options]

Commands:

  build            Build project from build.zig
  build-exe        Create executable from source or object files
  build-lib        Create library from source or object files
  build-obj        Create object from source or assembly
  cc               Use Zig as a drop-in C compiler
  c++              Use Zig as a drop-in C++ compiler
  env              Print lib path, std path, compiler id and version
  fmt              Parse file and render in canonical zig format
  init-exe         Initialize a `zig build` application in the cwd
  init-lib         Initialize a `zig build` library in the cwd
  libc             Display native libc paths file or validate one
  run              Create executable and run immediately
  translate-c      Convert C code to Zig code
  targets          List available compilation targets
  test             Create and run a test build
  version          Print version number and exit
  zen              Print zen of zig and exit

General Options:

  --help           Print command-specific usage

error: unknown command: -E

go version go1.15.5 linux/amd64

So this looks like go is inserting flags before it passes cc. The user will have to work around this by creating a shell script like this:

#!/bin/sh
zig cc $@

And then passing that as CC. This way zig gets invoked with zig cc -E ... instead of zig -E ...

This issue is a feature request for go to honor the entire command line in CC, appending flags rather than prepending them. Or, at least adding explicit detection for zig cc and passing the cc as the first arg.

The text was updated successfully, but these errors were encountered:

bcmills · 2020-12-08T20:53:27Z

This seems closely related to #41400, but with the extra caveat that the first argument is not actually the full C compiler command.

bcmills · 2020-12-08T20:53:59Z

CC (ha!) @ianlancetaylor @jayconrod @matloob

andrewrk · 2020-12-08T21:27:39Z

Related topic, can anyone help shed some light on why zig cc gets EACCES when it tries to create a project-local zig-cache/ directory?

mkdirat(AT_FDCWD, "zig-cache", 0755 = -1 EACCES (Permission denied)

Edit: hmm, it looks like go changes the cwd when invoking the c compiler for some reason.

AlexRouSg · 2020-12-08T21:55:26Z

@andrewrk run go build -x to show the build commands
It should be cding to a temp dir before running cc

When I do CC="gcc gcc2" go build -x (ignore the invalid CC string)
I get ... TERM='dumb' gcc -I "c:\\go\\src\\runtime\\cgo" gcc2 -m64 -mthreads -fmessage-length=0 "-fdebug-prefix-map=$WORK\\b057=/tmp/go-build" -gno-record-gcc-switches -I "$WORK\\b057\\" -g -O2 -Wall -Werror -o "$WORK\\b057\\_x001.o" -c _cgo_export.c

After skimming the CL for #41400 it does look like the fix would be in the same places

andrewrk · 2020-12-08T22:18:39Z

@andrewrk run go build -x to show the build commands

Hmm I got this:

[nix-shell:~/misc/gozig]$ CC="zcc" go build -x main.go
WORK=/run/user/1000/go-build735552327
mkdir -p $WORK/b001/
cat >$WORK/b001/importcfg.link << 'EOF' # internal
packagefile command-line-arguments=/home/andy/.cache/go-build/24/2472355c95ff5bd2c8379130112402791828be5fa411080b10c62f6c57556a65-d
packagefile fmt=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/fmt.a
packagefile runtime/cgo=/home/andy/.cache/go-build/9b/9b3a75664ee02a7fb873a1dc93cf7958c170dedef5fc1c46e2de82b40856bdec-d
packagefile syscall=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/syscall.a
packagefile runtime=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/runtime.a
packagefile errors=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/errors.a
packagefile internal/fmtsort=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/fmtsort.a
packagefile io=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/io.a
packagefile math=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/math.a
packagefile os=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/os.a
packagefile reflect=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/reflect.a
packagefile strconv=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/strconv.a
packagefile sync=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/sync.a
packagefile unicode/utf8=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/unicode/utf8.a
packagefile internal/bytealg=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/bytealg.a
packagefile internal/oserror=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/oserror.a
packagefile internal/race=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/race.a
packagefile internal/unsafeheader=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/unsafeheader.a
packagefile internal/cpu=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/cpu.a
packagefile runtime/internal/atomic=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/runtime/internal/atomic.a
packagefile runtime/internal/math=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/runtime/internal/math.a
packagefile runtime/internal/sys=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/runtime/internal/sys.a
packagefile internal/reflectlite=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/reflectlite.a
packagefile sort=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/sort.a
packagefile math/bits=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/math/bits.a
packagefile internal/poll=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/poll.a
packagefile internal/syscall/execenv=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/syscall/execenv.a
packagefile internal/syscall/unix=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/syscall/unix.a
packagefile internal/testlog=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/testlog.a
packagefile sync/atomic=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/sync/atomic.a
packagefile time=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/time.a
packagefile unicode=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/unicode.a
EOF
mkdir -p $WORK/b001/exe/
cd .
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=YCcuUkfsSqMNZpM94Hzz/WNU9rIi2KCHzwXiokjH7/pwW5amiz-sk8p-Zpyuhd/YCcuUkfsSqMNZpM94Hzz -extld=zcc /home/andy/.cache/go-build/24/2472355c95ff5bd2c8379130112402791828be5fa411080b10c62f6c57556a65-d
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/buildid -w $WORK/b001/exe/a.out # internal
# command-line-arguments
warning: unsupported linker arg: --compress-debug-sections=zlib-gnu
cp $WORK/b001/exe/a.out main
rm -r $WORK/b001/

I don't see the zig cc build commands

AlexRouSg · 2020-12-08T22:20:18Z

@andrewrk if it's been build cached you'd need to run go build -x -a to force a rebuild

gopherbot · 2020-12-08T23:01:47Z

Change https://golang.org/cl/276412 mentions this issue: cmd/go: Retain CC/CXX arg order

andrewrk · 2020-12-08T23:11:21Z

OK here's what I'm seeing:

~/my-project-directory$ CC="zcc" go build -x -a main.go
...
cd /nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/src/syscall
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/asm -p syscall -trimpath "$WORK/b029=>" -I $WORK/b029/ -I /nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/include -D GOOS_linux -D GOARCH_amd64 -o $WORK/b029/asm_linux_amd64.o ./asm_linux_amd64.s
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/pack r $WORK/b029/_pkg_.a $WORK/b029/asm_linux_amd64.o # internal
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/buildid -w $WORK/b029/_pkg_.a # internal
# runtime/cgo
attempt to unwrap error: ReadOnlyFileSystem
...

User runs command with cwd set to $HOME/my-project-directory
Go changes the cwd to /share/go/src/syscall
Zig does this:

mkdirat(AT_FDCWD, "zig-cache", 0755) = -1 EROFS (Read-only file system)

Although go does its own caching, it would be best for the user experience to allow zig to do its own caching as well, for a couple reasons:

There is a global cache as well, which speeds up linking for subsequent builds for the same target, even for different projects. This is for things such as libc and compiler_rt. Actually this part is working fine though, it is the local cache that is problematic.
zig cc does in fact allow .zig files to be thrown into the mix, and zig does its own caching of that part of the compilation. In this use case the user would do something like CC=zig cc foo.zig.
Zig disables its own per-file caching when the -MD flags (et all) are used, which Go is passing, so it actually integrates kind of perfectly.

I can open another issue for this but some informal guidance on how the go project sees this use case would be useful too. My question is, is this to be expected that the cwd should be completely ignored by the C compiler?

benesch · 2020-12-10T06:25:15Z

Just a passing thought, but assuming that the compiler's working directory is writeable might be flawed? At least, it seems reasonable enough to execute a C compiler from a non-writeable directory (maybe you're compiling sources in someone else's home directory?) as long as the output file you've specified is writeable. Maybe it would generally make sense for zig cc to gracefully handle cases where it can't create a local cache in the working directory in general, and not just for cgo?

andrewrk · 2020-12-10T23:55:11Z

I solved the cache directory issue with ziglang/zig@f7d6006. So, as soon as https://golang.org/cl/276412 lands, I believe this issue can be considered to be solved.

bcmills · 2020-12-11T20:53:08Z

@andrwrk, as a workaround in the meantime you could probably set CC to a small script that invokes zig cc with the same arguments.

benesch · 2020-12-12T04:32:43Z

@bcmills that workaround is mentioned in the original issue description. :)

seankhliao · 2021-01-26T19:57:29Z

resolved with #43808 and CL 284780

AlexRouSg · 2021-01-26T19:59:14Z

@seankhliao #43808 is a separate issue?

seankhliao · 2021-01-26T20:00:41Z

oops

LaPingvino · 2021-03-02T13:26:26Z

resolved with #43808 and CL 284780

yeah running with CL 284780 and that definitely does NOT solve this issue.

gopherbot · 2021-07-15T00:08:31Z

Change https://golang.org/cl/334732 mentions this issue: [dev.cmdgo] cmd: support space and quotes in CC and CXX

The CC and CXX environment variables now support spaces and quotes (both double and single). This fixes two issues: first, if CC is a single path that contains spaces (like 'c:\Program Files\gcc\bin\gcc.exe'), that should now work if the space is quoted or escaped (#41400). Second, if CC or CXX has multiple arguments (like 'gcc -O2'), they are now split correctly, and the arguments are passed before other arguments when invoking the C compiler. Previously, strings.Fields was used to split arguments, and the arguments were placed later in the command line. (#43078). Fixes #41400 Fixes #43078 Change-Id: I2d5d89ddb19c94adef65982a8137b01f037d5c11 Reviewed-on: https://go-review.googlesource.com/c/go/+/334732 Trust: Jay Conrod <jayconrod@google.com> Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>

gopherbot · 2021-08-12T22:42:54Z

Change https://golang.org/cl/341936 mentions this issue: cmd: support space and quotes in CC and CXX

anacrolix · 2021-08-24T12:11:08Z

I can confirm this fixes the issue.

andrewrk mentioned this issue Dec 8, 2020

"zig cc" doesn't work with cgo (golang) ziglang/zig#7342

Closed

bcmills changed the title ~~cgo doesn't work with "zig cc"~~ cmd/go: cgo builds don't work with "zig cc" Dec 8, 2020

bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 8, 2020

bcmills added this to the Backlog milestone Dec 8, 2020

ianlancetaylor added the help wanted label Dec 8, 2020

This comment has been minimized.

Sign in to view

andrewrk mentioned this issue Dec 10, 2020

CLI: explicit integration with cgo ziglang/zig#7377

Closed

bcmills modified the milestones: Backlog, Go1.17 Jan 14, 2021

jayconrod mentioned this issue Jan 21, 2021

cmd/go: CC variable misparsed when set to a path containing spaces #41400

Closed

AlexRouSg mentioned this issue Jan 24, 2021

cmd/cgo: rt0_go failure on Windows when using zig cc #43886

Closed

seankhliao closed this as completed Jan 26, 2021

seankhliao reopened this Jan 26, 2021

This was referenced Jan 26, 2021

build: setting CC to "ccache clang" does not work #11685

Closed

cmd/cgo: ignores arguments in $CC #14551

Closed

zikaeroh mentioned this issue Feb 28, 2021

cmd/cgo, cmd/link: with zig as CC/CXX, relocation target not defined, fatal error, SIGSEGV when compiling net/http #44695

Closed

ianlancetaylor modified the milestones: Go1.17, Backlog May 4, 2021

Andoryuuta mentioned this issue May 8, 2021

Proposal: standard wrappers for zig cc and zig c++ ziglang/zig#8716

Closed

motiejus mentioned this issue Jun 8, 2021

cmd/link: with CC=zig: SIGSERV when cross-compiling to darwin/amd64 #46644

Closed

gopherbot closed this as completed in 742dcba Aug 16, 2021

dmitshur modified the milestones: Backlog, Go1.18 May 26, 2022

dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels May 26, 2022

golang locked and limited conversation to collaborators May 26, 2023

gopherbot added the FrozenDueToAge label May 26, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: cgo builds don't work with "zig cc" #43078

cmd/go: cgo builds don't work with "zig cc" #43078

andrewrk commented Dec 8, 2020

bcmills commented Dec 8, 2020

bcmills commented Dec 8, 2020 •

edited

andrewrk commented Dec 8, 2020 •

edited

AlexRouSg commented Dec 8, 2020

This comment has been minimized.

andrewrk commented Dec 8, 2020

AlexRouSg commented Dec 8, 2020 •

edited

gopherbot commented Dec 8, 2020

andrewrk commented Dec 8, 2020 •

edited

benesch commented Dec 10, 2020

andrewrk commented Dec 10, 2020

bcmills commented Dec 11, 2020

benesch commented Dec 12, 2020

seankhliao commented Jan 26, 2021

AlexRouSg commented Jan 26, 2021

seankhliao commented Jan 26, 2021

LaPingvino commented Mar 2, 2021

gopherbot commented Jul 15, 2021

gopherbot commented Aug 12, 2021

anacrolix commented Aug 24, 2021

cmd/go: cgo builds don't work with "zig cc" #43078

cmd/go: cgo builds don't work with "zig cc" #43078

Comments

andrewrk commented Dec 8, 2020

Example cgo code

other compilers

zig cc

bcmills commented Dec 8, 2020

bcmills commented Dec 8, 2020 • edited

andrewrk commented Dec 8, 2020 • edited

AlexRouSg commented Dec 8, 2020

This comment has been minimized.

andrewrk commented Dec 8, 2020

AlexRouSg commented Dec 8, 2020 • edited

gopherbot commented Dec 8, 2020

andrewrk commented Dec 8, 2020 • edited

benesch commented Dec 10, 2020

andrewrk commented Dec 10, 2020

bcmills commented Dec 11, 2020

benesch commented Dec 12, 2020

seankhliao commented Jan 26, 2021

AlexRouSg commented Jan 26, 2021

seankhliao commented Jan 26, 2021

LaPingvino commented Mar 2, 2021

gopherbot commented Jul 15, 2021

gopherbot commented Aug 12, 2021

anacrolix commented Aug 24, 2021

bcmills commented Dec 8, 2020 •

edited

andrewrk commented Dec 8, 2020 •

edited

AlexRouSg commented Dec 8, 2020 •

edited

andrewrk commented Dec 8, 2020 •

edited