Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: GO111MODULE=on go build requires gcc #26988

Closed
nqv opened this issue Aug 14, 2018 · 20 comments
Closed

cmd/go: GO111MODULE=on go build requires gcc #26988

nqv opened this issue Aug 14, 2018 · 20 comments
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Milestone

Comments

@nqv
Copy link
Contributor

nqv commented Aug 14, 2018

What version of Go are you using (go version)?

go version go1.11rc1 linux/amd64

Does this issue reproduce with the latest release?

Not sure

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/nqv/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/nqv/go"
GOPROXY=""
GORACE=""
GOROOT="/home/nqv/Downloads/go"
GOTMPDIR=""
GOTOOLDIR="/home/nqv/Downloads/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build126805613=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Build a minimal http server failed with GO111MODULE=on go build

$ cd ~/go/src/httpd
$ cat httpd.go
package main

import "net/http"

func main() {
	http.ListenAndServe(":8000", nil)
}
$ GO111MODULE=on go mod init
go: creating new go.mod: module httpd
$ GO111MODULE=on go build
# net
exec: "gcc": executable file not found in $PATH

What did you expect to see?

GO111MODULE=on go build produces binary.

What did you see instead?

Build failed due to missing gcc.
Either of these commands works fine though:

$ go build
$ GO111MODULE=on CGO_ENABLED=0 go build
@mvdan
Copy link
Member

mvdan commented Aug 14, 2018

It's surprising that GO111MODULE=on go build is trying to build the net package, or any other part of the standard library. How did you install Go? Does go build std do nothing, like it should?

cc @myitcv

@myitcv
Copy link
Member

myitcv commented Aug 14, 2018

Edit: fixed bad copy-paste.

Reproduced:

docker run --rm -i -t ubuntu:16.04 bash
apt-get -y update < /dev/null > /dev/null
apt-get -y install wget < /dev/null > /dev/null
cd /tmp
wget -q https://dl.google.com/go/go1.11rc1.linux-amd64.tar.gz
tar -zxf go1.11rc1.linux-amd64.tar.gz
export PATH=/tmp/go/bin:$PATH
cd
mkdir -p go/src/httpd
cd go/src/httpd
cat <<EOD > httpd.go
package main

import "net/http"

func main() {
http.ListenAndServe(":8000", nil)
}
EOD
GO111MODULE=on go mod init
GO111MODULE=on go build

exits non-zero with:

# net
exec: "gcc": executable file not found in $PATH

whereas:

go build

just works.

/cc @rsc @bcmills @ianlancetaylor

@andybons andybons added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. modules labels Aug 14, 2018
@andybons andybons added this to the Go1.11 milestone Aug 14, 2018
@nqv
Copy link
Contributor Author

nqv commented Aug 14, 2018

I installed go1.11 from archive

$ cd ~/Downloads
$ curl -L -O https://dl.google.com/go/go1.11rc1.linux-amd64.tar.gz
$ tar xf go1.11rc1.linux-amd64.tar.gz

FYI, go build -v outputs

$ go build -v
httpd

$ rm -r ~/.cache/go-build/
$ GO111MODULE=on go build -v
golang_org/x/crypto/cryptobyte/asn1
golang_org/x/net/dns/dnsmessage
golang_org/x/crypto/internal/chacha20
golang_org/x/crypto/poly1305
golang_org/x/crypto/cryptobyte
golang_org/x/crypto/curve25519
golang_org/x/crypto/chacha20poly1305
golang_org/x/text/transform
golang_org/x/text/unicode/bidi
golang_org/x/net/http2/hpack
net
# net
exec: "gcc": executable file not found in $PATH
golang_org/x/text/unicode/norm
golang_org/x/text/secure/bidirule
golang_org/x/net/idna

$ rm -r ~/.cache/go-build/
$ GO111MODULE=on CGO_ENABLED=0 go build -v
golang_org/x/crypto/cryptobyte/asn1
golang_org/x/net/dns/dnsmessage
golang_org/x/crypto/curve25519
golang_org/x/crypto/internal/chacha20
golang_org/x/crypto/cryptobyte
golang_org/x/crypto/poly1305
golang_org/x/text/transform
golang_org/x/crypto/chacha20poly1305
golang_org/x/text/unicode/bidi
golang_org/x/text/unicode/norm
net
golang_org/x/net/http2/hpack
golang_org/x/text/secure/bidirule
golang_org/x/net/idna
golang_org/x/net/http/httpproxy
net/textproto
crypto/x509
golang_org/x/net/http/httpguts
crypto/tls
net/http/httptrace
net/http
httpd

@rsc
Copy link
Contributor

rsc commented Aug 17, 2018

It's not that surprising, and it's fine for Go 1.11.

@pda
Copy link

pda commented Aug 28, 2018

I find it surprising. It's a pain when there's no gcc.
Here's a minimal reproduction:

$ docker run --rm -w /notgo golang:1.11-alpine sh -c 'go mod init x && go build std'
go: creating new go.mod: module x
# net
exec: "gcc": executable file not found in $PATH

It looks like setting CGO_ENABLED=0 avoids the error, but I'm not sure what the implications are…

$ docker run --rm -w /x golang:1.11-alpine sh -c 'go mod init x && CGO_ENABLED=0 go build std'
go: creating new go.mod: module x

@tylercloke
Copy link

@rsc Can you expand a bit? It was certainly a surprise for me :) I started seeing this after upgrade to 1.11. Feels like a breaking change to the end-user if my build worked before upgrade and doesn't work now. Just want to understand what changed in more detail and what proper remediation is. Thanks!

@thepudds
Copy link
Contributor

thepudds commented Sep 1, 2018

Regarding this issue, Russ commented:

Because the pre-built packages are non-module builds and can’t be reused. Sorry. Disable cgo for now or install gcc.

Note this issue is specific to opting in to modules (and related to the fact that Go 1.11 ships non-module based packages because the modules system overall is still opt-in only in Go 1.11).

This particular issue should not be affecting non-module based builds as far as I understand it.

@tylercloke
Copy link

tylercloke commented Sep 4, 2018

@thepudds Thanks! Can you link me to the original context for that comment? Also, would we expect this to affect builds that pull in deps that might use modules? We did not opt into modules and were seeing issues on 1.11.

@bcmills
Copy link
Contributor

bcmills commented Sep 4, 2018

We did not opt into modules and were seeing issues on 1.11.

The original report of this issue required GO111MODULE=on in order to trigger. If you're seeing these symptoms outside of module mode, please file a separate issue for them.

(You can check whether module mode is active by running go env: if you're in module mode, it will have a non-empty GOMOD line.)

@tylercloke
Copy link

tylercloke commented Sep 4, 2018

There are two issues marked as dups of this that make no mention of modules. I can also repro without modules enabled:

#27303
#27285

I'll open another issue when I have time if you want but what I'm seeing looks pretty similar to the above.

@mfridman
Copy link

Noticed a similar issue in our pipeline when switching from 1.10.x to 1.11.0. Specifically with the 1.11.0-alpine3.7 docker image.

We explicitly set GO111MODULE=off and were seeing similar error as reported above.

exec: "gcc": executable file not found in $PATH

The solution, as mentioned before, was to explicitly set CGO_ENABLED=0. I suspect many folks will come across this particular issue when using lightweight containers such as alpine.

@thepudds
Copy link
Contributor

@mfridman if you look at the symptoms in #27285, do those line up with what you are seeing? #27285 has an initial CL.

@mvdan
Copy link
Member

mvdan commented Oct 3, 2018

We ran into the same issue when upgrading from 1.10.3-alpine to 1.11.1-alpine - errors due to gcc not being installed.

To clarify, this is when building a package inside GOPATH and without setting any Go env vars other than GOPATH and GOCACHE:

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/go/src/brank.as/brankas-api/.go_cache"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build759719652=/tmp/go-build -gno-record-gcc-switches"
$ go test -p 8 -parallel 8 -coverprofile=coverage.out -count=1 -v ./...
# runtime/cgo
exec: "gcc": executable file not found in $PATH

For now I'll just globally set CGO_ENABLED=0, but I presume there's something to fix here since this worked in 1.10.

@thepudds
Copy link
Contributor

thepudds commented Oct 6, 2018

Hi @mvdan , given you are seeing that behavior outside of modules (including an empty GOMOD reported by go env), what would you think about opening a new issue that covers the problem you saw in your immediately prior comment?

I think that was the suggestion from @bcmills several comments back:

The original report of this issue required GO111MODULE=on in order to trigger. If you're seeing these symptoms outside of module mode, please file a separate issue for them.

As far as I am following things, it sounds like there might be at least three distinct behaviors:

  1. This original issue reported here in cmd/go: GO111MODULE=on go build requires gcc #26988, which might be surprising to a user but sounds like it is currently "expected" behavior with 1.11 due to the opt in nature of modules and because "the pre-built packages are non-module builds and can’t be reused".

  2. Issue cmd/go: 'go build' in module mode rebuilds vendored dependencies in GOROOT #27285, which sounds like it is related to the -i flag (and which has one pending CL, but sounds like the solution might be to make -i a no-op in module mode)

  3. The issue you are reporting, which sounds like it happened when modules are disabled (and some chance might be the same thing that @mfridman and @tylercloke reported, but that might be easiest to discuss and determine in a new/separate issue. Edit: @mvdan has now opened cmd/go: 'go test' on alpine and ubuntu requires gcc in GOPATH mode #28065 for this issue 3).

@mvdan
Copy link
Member

mvdan commented Oct 6, 2018

Agreed, will open a new issue. I posted here first, in case I was missing something obvious or misunderstanding how GO11MODULE affected this issue.

@thepudds
Copy link
Contributor

thepudds commented Oct 6, 2018

Hi @mvdan, makes sense that you posted here first. Overall, not 100% clear what's going on with the more recent reports here, and some chance there might be multiple underlying causes that might initially seem to have similar symptoms...

@rsc
Copy link
Contributor

rsc commented Oct 24, 2018

This is expected: we can only ship one pre-compiled copy of the standard library, and the pre-compiled copy we ship is for non-modules (GOPATH) mode. If we need standard library packages in module mode then we may need to rebuild them if there are any changes needed. In this case the slightly different effect of the vendor folder means that changes are needed.

One way to fix this for Go 1.12 would be to rename the top-level vendor folder to something different (internal/...) and fix the imports to match. We should probably do that.

@rsc rsc added the NeedsFix The path to resolution is known, but the work has not been done. label Oct 24, 2018
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 24, 2018
@rsc
Copy link
Contributor

rsc commented Oct 24, 2018

Note the fix is to change the import path of the vendor folder.

@gopherbot
Copy link

Change https://golang.org/cl/147443 mentions this issue: vendor/golang_org: move to internal/golang_org

@bcmills
Copy link
Contributor

bcmills commented Nov 29, 2018

CL 147443 fixed the known cause of these symptoms. Please open a new issue if you continue to observe them with a go command built after commit 2012227.

blgm added a commit to pivotal-cf/cf-rabbitmq-smoke-tests-release that referenced this issue Dec 4, 2018
When the smoke tests release was updated to Go v1.11.2, the BOSH release
pipelines all passed, but pipelines that consumed the final release
failed because `gcc` was not installed.  This is a very similar issue to
golang/go#26988.  Experiments showed that
adding `CGO_ENABLED=0 GO111MODULE=off` to disable `cgo` and Go moudles
allowed the smoke tests to pass.  These variables should be removed when
we recieved an updated version of Go (after 1.11.2).

[#159658309]
@golang golang locked and limited conversation to collaborators Nov 29, 2019
@rsc rsc unassigned bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Projects
None yet
Development

No branches or pull requests