cmd/go: build fails requiring C compiler for runtime/cgo #47215

lyderic · 2021-07-15T09:24:09Z

What version of Go are you using (`go version`)?

I am running go 1.16.5, but I am also testing go 1.17rc1:

$ go1.17rc1 version
go version go1.17rc1 linux/amd64
$ go version
go version go1.16.5 linux/amd64

Does this issue reproduce with the latest release?

It's ok with the latest stable release (1.16.5), not with 1.17rc1

What operating system and processor architecture are you using (`go env`)?

GOARCH="amd64"
GOOS="linux"
I am running Archlinux, fully up to date as of July 15 2021, 10:15 BST. I am running in an LXD container (LXD 4.16)

go env Output

$ go1.17rc1 env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/unix/.cache/go-build"
GOENV="/home/unix/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/unix/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/unix/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/unix/sdk/go1.17rc1"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/unix/sdk/go1.17rc1/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17rc1"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build640958217=/tmp/go-build -gno-record-gcc-switches"

What did you do?

This is the full script of what I ran:

I [unix@go117 foo]$ go1.17rc1 mod init foo
go: creating new go.mod: module foo
I [unix@go117 foo]$ ed main.go
main.go: No such file or directory
a
package main

import ( 
  . "github.com/lyderic/tools"
)

func main(){
  Cyanln("Hello, World!")
}
.
wq
98
I [unix@go117 foo]$ go1.17rc1 mod tidy
go: finding module for package github.com/lyderic/tools
go: downloading github.com/lyderic/tools v0.3.4
go: found github.com/lyderic/tools in github.com/lyderic/tools v0.3.4
go: downloading github.com/fatih/color v1.10.0
go: downloading github.com/sendgrid/sendgrid-go v3.10.0+incompatible
go: downloading github.com/wayneashleyberry/terminal-dimensions v1.0.0
go: downloading github.com/mattn/go-colorable v0.1.8
go: downloading github.com/mattn/go-isatty v0.0.12
go: downloading github.com/sendgrid/rest v2.6.4+incompatible
go: downloading github.com/stretchr/testify v1.7.0
go: downloading golang.org/x/sys v0.0.0-20210423082822-04245dca01da
go: downloading golang.org/x/net v0.0.0-20210614182718-04defd469f4e
go: downloading github.com/davecgh/go-spew v1.1.0
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
I [unix@go117 foo]$ go1.17rc1 build
\# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH
I [unix@go117 foo]$ go build
I [unix@go117 foo]$ ./foo
Hello, World!

What did you expect to see?

With go 1.16.5, the above program compiles with no error. I use only one external lib: github.com/lyderic/tools

What did you see instead?

With go 1.17rc1, I get the following error:

$ go1.17rc1 build
# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH

Note: if I install gcc, the error (obviously) goes away. However, the program compiles with no gcc installed if I compile with go 1.16.5, so there might be a bug.

Thanks
L.

The text was updated successfully, but these errors were encountered:

seankhliao · 2021-07-15T12:07:38Z

How did you install 1.16.5 and 1.17rc1 ?

lyderic · 2021-07-15T12:11:55Z

I installed go 1.16.5 as follows:

$ sudo pacman -S go

Then I installed go 1.17rc1 as follows:

$ go get golang.org/dl/go1.17rc1
$ go1.17rc1 download

Then I set up my PATH to include ${HOME}/go/bin

lyderic · 2021-07-15T12:28:00Z

To rule out any Archlinux / Pacman effect, I did a manual install of go 1.16.6 :

$ curl -LOs https://golang.org/dl/go1.16.6.linux-amd64.tar.gz
$ sudo tar xzf go1.16.6.linux-amd64.tar.gz -C /usr/local
$ export PATH=${PATH}:/usr/local/go/bin
$ go version
go version go1.16.6 linux/amd64
$ go get golang.org/dl/go1.17rc1
go: downloading golang.org/dl v0.0.0-20210713194856-38ddc79c2163
$ ~/go/bin/go1.17rc1 download
Downloaded   0.0% (     3273 / 134784935 bytes) ...
[...]
Downloaded  91.8% (123780176 / 134784935 bytes) ...
Downloaded  93.4% (125844544 / 134784935 bytes) ...
Downloaded 100.0% (134740992 / 134784935 bytes) ...
Downloaded 100.0% (134784935 / 134784935 bytes)
Unpacking /home/unix/sdk/go1.17rc1/go1.17rc1.linux-amd64.tar.gz ...
Success. You may now run 'go1.17rc1'
$ export PATH=${PATH}:${HOME}/go/bin
$ go1.17rc1 version
go version go1.17rc1 linux/amd64

It didn't fix the problem. The most notable thing is that the simple example program I use compiles fine with 1.16 but not with go 1.17. So there is a regression issue, I guess.

Many thanks

L.

lyderic · 2021-07-15T12:42:45Z

I tried the above manual install on a fresh ubuntu 20.04 LXC container: same error:

$ go1.17rc1 build
# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH

However:

$ go version
go version go1.16.6 linux/amd64
$ go build
$ ./foo 
Hello, World!

seankhliao · 2021-07-15T12:48:16Z

It looks like go doesn't skip runtime/cgo when a C compiler isn't available and CGO_ENABLED isn't explicitly disabled

[root@1ba9e2afedf2 x]# go1.16.5 build -x runtime/cgo
WORK=/tmp/go-build3995079167
[root@1ba9e2afedf2 x]# go1.17rc1 build -x runtime/cgo
WORK=/tmp/go-build3635095243
mkdir -p $WORK/b001/
cd /root/sdk/go1.17rc1/src/runtime/cgo
TERM='dumb' CGO_LDFLAGS='"-g" "-O2" "-lpthread"' /root/sdk/go1.17rc1/pkg/tool/linux_amd64/cgo -objdir $WORK/b001/ -importpath runtime/cgo -import_runtime_cgo=false -import_syscall=false -- -I $WORK/b001/ -g -O2 -Wall -Werror ./cgo.go
# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH

cc @bcmills @jayconrod @matloob

lyderic · 2021-07-15T12:54:30Z

Yes indeed. If I disable CGO_ENABLED, it works:

$ go1.17rc1 build
# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH
$ CGO_ENABLED=0 go1.17rc1 build
$ ./foo
Hello, World!

jayconrod · 2021-07-15T15:13:08Z

Closing as a duplicate of #33840. #37158, #27303, #26988 are also related.

We need a much better diagnostic here, and we should reconsider whether CGO_ENABLED should default to 1 when no C compiler is installed.

lyderic · 2021-07-15T15:36:46Z

The issue is that compilation works independently of how CGO_ENABLED is set when the same program is compiled with 1.16. So it looks like a regression problem to me...

$ CGO_ENABLED=0 go build
$ CGO_ENABLED=1 go build
$ CGO_ENABLED=0 go1.17rc1 build
$ CGO_ENABLED=1 go1.17rc1 build
# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH

jayconrod · 2021-07-15T15:47:11Z

@lyderic I expect this is because of the build cache. If you run go clean -cache then CGO_ENABLED=1 go build, does it still succeed with Go 1.16 and gcc not installed?

lyderic · 2021-07-15T15:50:06Z

Yes it does:

$ go clean -cache
$ CGO_ENABLED=1 go build

jayconrod · 2021-07-15T16:11:04Z

Interesting. Reopening to investigate this more. I wonder if we changed how the cache key is calculated?

jayconrod · 2021-07-16T18:57:17Z

We did change how the cache key is calculated.

I've opened #47251 with an explanation and possible solution.

bcmills · 2021-07-19T15:14:14Z

Ok, so here's an idea for 1.18 (if we don't do #47257 or #47251 instead).

What if we bake the C compiler ID used to build the standard library into cmd/go itself, and use that as the default ID for the default CC if exec.LookPath fails to locate a matching binary?

I guess that would allow CGO_ENABLED=1 builds to succeed, but only under very narrow circumstances. They would succeed if (and only if) both:

the user doesn't set any other flags that would invalidate cache entries, and
the user's code doesn't depend on any third-party packages with cgo build constraints, which would presumably fail to build anyway if no C compiler is present.

But that seems too fragile to me. I prefer #47251 (don't default to CGO_ENABLED=1 without a C compiler) and/or #47257 (don't ship precompiled .a files; always require a C compiler for builds that depend on cgo code).

jayconrod · 2021-07-29T17:18:07Z

Verified that the workaround in CL 335409 fixes this.

Discussion continues in #47257. If that's approved and implemented, it would still be an error to build a package that needs cgo (like runtime/cgo) if no C compiler is installed. However, net and os/user would not need cgo.

gopherbot · 2022-11-22T03:21:58Z

Change https://go.dev/cl/452677 mentions this issue: cmd/go: remove special case for prebuilt cgo library cache keys

This was an oversight from CL 452457 that I noticed while investigating #56889. This change essentially undoes CL 335409, which is no longer needed after CL 450739 because we no longer attempt to use cgo by default when no C compiler is present. Updates #47257. Updates #40042. Updates #47215. Change-Id: I29c7ce777a9ec7ba5820dc1d836b12a61b86bc37 Reviewed-on: https://go-review.googlesource.com/c/go/+/452677 Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com>

seankhliao changed the title ~~go 1.17rc1: "gcc" not found (works with go 1.16.5)~~ cmd/go: build fails requiring C compiler for runtime/cgo Jul 15, 2021

seankhliao added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 15, 2021

jayconrod closed this as completed Jul 15, 2021

jayconrod reopened this Jul 15, 2021

jayconrod self-assigned this Jul 15, 2021

jayconrod mentioned this issue Jul 16, 2021

cmd/go: default to CGO_ENABLED=0 when CC not set and default compiler not found #47251

Closed

cherrymui added this to the Go1.17 milestone Jul 16, 2021

jayconrod mentioned this issue Jul 16, 2021

cmd/dist: remove precompiled .a files from binary distributions [freeze exception] #47257

Closed

jayconrod closed this as completed Jul 29, 2021

lyderic mentioned this issue Dec 15, 2021

cmd/go: varying GOROOT_FINAL invalidates precompiled C dependencies of "net" #50183

Closed

rsc unassigned jayconrod Jun 23, 2022

golang locked and limited conversation to collaborators Nov 22, 2023

gopherbot added the FrozenDueToAge label Nov 22, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: build fails requiring C compiler for runtime/cgo #47215

cmd/go: build fails requiring C compiler for runtime/cgo #47215

lyderic commented Jul 15, 2021

seankhliao commented Jul 15, 2021

lyderic commented Jul 15, 2021

lyderic commented Jul 15, 2021 •

edited

lyderic commented Jul 15, 2021

seankhliao commented Jul 15, 2021

lyderic commented Jul 15, 2021

jayconrod commented Jul 15, 2021

lyderic commented Jul 15, 2021

jayconrod commented Jul 15, 2021

lyderic commented Jul 15, 2021

jayconrod commented Jul 15, 2021

jayconrod commented Jul 16, 2021

bcmills commented Jul 19, 2021 •

edited

jayconrod commented Jul 29, 2021

gopherbot commented Nov 22, 2022

cmd/go: build fails requiring C compiler for runtime/cgo #47215

cmd/go: build fails requiring C compiler for runtime/cgo #47215

Comments

lyderic commented Jul 15, 2021

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

seankhliao commented Jul 15, 2021

lyderic commented Jul 15, 2021

lyderic commented Jul 15, 2021 • edited

lyderic commented Jul 15, 2021

seankhliao commented Jul 15, 2021

lyderic commented Jul 15, 2021

jayconrod commented Jul 15, 2021

lyderic commented Jul 15, 2021

jayconrod commented Jul 15, 2021

lyderic commented Jul 15, 2021

jayconrod commented Jul 15, 2021

jayconrod commented Jul 16, 2021

bcmills commented Jul 19, 2021 • edited

jayconrod commented Jul 29, 2021

gopherbot commented Nov 22, 2022

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

lyderic commented Jul 15, 2021 •

edited

bcmills commented Jul 19, 2021 •

edited