Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: setting CC to "ccache clang" does not work #11685

Closed
tucnak opened this issue Jul 12, 2015 · 11 comments
Closed

build: setting CC to "ccache clang" does not work #11685

tucnak opened this issue Jul 12, 2015 · 11 comments

Comments

@tucnak
Copy link

tucnak commented Jul 12, 2015

I am not able to build Go from Git (tag: go1.5beta1) with bootstrapping go1.4.2 on OSX 10.10.4, cos of wrong handling of $CC environmental variable.

Steps to reproduce:

  1. Get a custom CC, e.g. with ccache: export CC="ccache clang"
  2. ./all.bash

Expected result: built Go with exit status 1
Observed result: https://gist.github.com/tucnak/77c6d21d299f3c73ee7e

Looks like build script cuts off everything after space char in $CC, so instead of calling ccache clang -E -dM ... it does ccache -E -dM ... which is wrong. There must be a regression!

@ianlancetaylor ianlancetaylor changed the title OSX: Go from Git won't get built with custom $CC build: setting CC to "ccache clang" does not work Jul 12, 2015
@ianlancetaylor ianlancetaylor added this to the Go1.6 milestone Jul 12, 2015
@dominikh
Copy link
Member

This affects building any package with cgo and isn't specific to building Go itself. It's a regression that was introduced in commit b38fa89

@ianlancetaylor
Copy link
Contributor

The build doesn't discard the options after the space character--you can see them in your gist in GOGCCFLAGS. But you are correct that the build assumes that the first word in $CC is special, and that it doesn't need to keep the second work immediately after the first word as is true for something like "ccache clang".

I doubt this will be fixed for the 1.5 release. For now you will have to work around the problem, either by using a shell script or by omitting "ccache". Frankly, "ccache" isn't buying you anything here, so I recommend omitting it until this is fixed.

@dominikh
Copy link
Member

For what it is worth, this was already semi-broken in Go 1.4, where it executes the following commands for CC="myccache gcc":

myccache gcc -E -dM -xc -m64 -I /tmp/go-build460939936/command-line-arguments/_obj/ -
myccache gcc -w -Wno-error -o/tmp/go-build460939936/command-line-arguments/_obj//_cgo_.o -gdwarf-2 -c -xc -I /tmp/go-build460939936/command-line-arguments/_obj/ -m64 -
myccache gcc -w -Wno-error -o/tmp/go-build460939936/command-line-arguments/_obj//_cgo_.o -gdwarf-2 -c -xc -I /tmp/go-build460939936/command-line-arguments/_obj/ -m64 -
myccache -I /tmp gcc -fPIC -m64 -pthread -fmessage-length=0 -print-libgcc-file-name

Where the last command isn't correct. So this is only partly a regression towards 1.4. It used to work, however, in 2012, where 7ae6872 specifically added support for it.

@ghost
Copy link

ghost commented Oct 14, 2015

In my opinion, this isn't a golang bug.

cgo should just print a warning when CC contains spaces, so that the user can fix the actual cause of the problem.

Solution:

$ cat ~/bin/ccache-gcc 
#!/bin/bash
exec ccache /usr/bin/gcc "$@"

$ cat ~/bin/ccache-g++
#!/bin/bash
exec ccache /usr/bin/g++ "$@"

$ cat ~/.bashrc  (relevant lines only)
export PATH="${HOME}/bin:${PATH}"
export CC="ccache-gcc"
export CXX="ccache-g++"

@ianlancetaylor
Copy link
Contributor

@atom-symbol I don't think I agree. I think you should be able to use spaces in $CC, and my understanding is that it used to work. It's not a major issue--as you say, you can always use a shell script--but I think we should fix it so that it works.

@ghost
Copy link

ghost commented Oct 14, 2015

@ianlancetaylor CC="xxx yyy zzz" is ambiguous because it is uncertain whether the executable is "xxx" or "xxx yyy zzz". One way to decide is first to look whether there exists "xxx yyy zzz" executable, if it doesn't than for check existence of "xxx yyy", etc. go build does not know beforehand whether the name has spaces in it or not.

In the absolute, fixing this issue is impossible because solving it would require go build to know the mind's meaning of spaces in "xxx yyy zzz". Even if the build checks for the existence of "$PATH/xxx yyy zzz", it doesn't guarantee that the person really meant "$PATH/xxx yyy zzz" because the existence of "$PATH/xxx yyy zzz" may just be a coincidence that the person did not took into account.

@tucnak
Copy link
Author

tucnak commented Oct 14, 2015

@atom-symbol CC="xxx yyy zzz" doesn't seem ambiguous to me, it's pretty clear for those familiar with unix. I also can't workaround CC with spaces, since it's a must-have for my build environment in Qt. IMHO, any sort of valid CC command should be interpreted by Go fine. And as @ianlancetaylor pointed out, it's been working inb4...

@ianlancetaylor
Copy link
Contributor

@atom-symbol I agree with @tucnak. It's not ambiguous. Spaces separate arguments. That is historically how the CC environment variable works on Unix systems. Programs like make and autoconf support CC with spaces, and people expect to be able to use them.

@mdempsky
Copy link
Member

mdempsky commented Dec 7, 2015

There's a handful of problems.

First, in cmd/go when it sees a multifield CC value like "ccache clang", it resets CC="ccache" and moves "clang" into GOGCCFLAGS. (It also assumes it's able to insert arguments after the first field like "ccache -I objdir clang".)

Second, cmd/cgo doesn't use GOGCCFLAGS, so it doesn't see the "clang" argument.

Third, when you run "go env", it runs mkEnv again based on the updated environment, so the "clang" string disappears completely from the output of CC="ccache clang" go env, contrary to the environment actually provided to subtools.

Lastly, cmd/dist/test.go is currently written to assume $CC (and go env CC's output; it uses both for some reason) is a single field string, and it does use $GOGCCFLAGS.

@mdempsky mdempsky modified the milestones: Go1.7, Go1.6 Dec 8, 2015
@mdempsky mdempsky self-assigned this Dec 8, 2015
@mdempsky
Copy link
Member

mdempsky commented Dec 8, 2015

This has never worked, and ccache isn't particularly useful for Go, so it doesn't seem like a blocker for Go 1.6. I'll look into it for Go 1.7.

@rsc rsc modified the milestones: Unplanned, Go1.7 May 17, 2016
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation, when the host and target
architectures are the same build a set of host binaries and tools with
CC_FOR_TARGET set to the host compiler.  Also, the go build system is not
compatible with ccache, so use HOSTCC_NOCCACHE.  See
golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation, when the host and target
architectures are the same build a set of host binaries and tools with
CC_FOR_TARGET set to the host compiler.  Also, the go build system is not
compatible with ccache, so use HOSTCC_NOCCACHE.  See
golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation, when the host and target
architectures are the same build a set of host binaries and tools with
CC_FOR_TARGET set to the host compiler.  Also, the go build system is not
compatible with ccache, so use HOSTCC_NOCCACHE.  See
golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation, when the host and target
architectures are the same build a set of host binaries and tools with
CC_FOR_TARGET set to the host compiler.  Also, the go build system is not
compatible with ccache, so use HOSTCC_NOCCACHE.  See
golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation, when the host and target
architectures are the same build a set of host binaries and tools with
CC_FOR_TARGET set to the host compiler.  Also, the go build system is not
compatible with ccache, so use HOSTCC_NOCCACHE.  See
golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 19, 2016
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 24, 2016
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation, when the host and target
architectures are the same build a set of host binaries and tools with
CC_FOR_TARGET set to the host compiler.  Also, the go build system is not
compatible with ccache, so use HOSTCC_NOCCACHE.  See
golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 24, 2016
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 26, 2016
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation build and install a set of
compiler and tool binaries built with CC_FOR_TARGET set to the host compiler.
Also, the go build system is not compatible with ccache, so use
HOSTCC_NOCCACHE.  See golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 26, 2016
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation build and install a set of
compiler and tool binaries built with CC_FOR_TARGET set to the host compiler.
Also, the go build system is not compatible with ccache, so use
HOSTCC_NOCCACHE.  See golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 26, 2016
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
woodsts pushed a commit to woodsts/buildroot that referenced this issue May 26, 2016
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation build and install a set of
compiler and tool binaries built with CC_FOR_TARGET set to the host compiler.
Also, the go build system is not compatible with ccache, so use
HOSTCC_NOCCACHE.  See golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 26, 2016
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
glevand added a commit to glevand/buildroot--buildroot that referenced this issue May 31, 2016
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
buildroot-auto-update pushed a commit to buildroot/buildroot that referenced this issue Jul 4, 2016
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
BWhitten pushed a commit to LairdCP/wb-buildroot that referenced this issue Dec 14, 2017
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation build and install a set of
compiler and tool binaries built with CC_FOR_TARGET set to the host compiler.
Also, the go build system is not compatible with ccache, so use
HOSTCC_NOCCACHE.  See golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
BWhitten pushed a commit to LairdCP/wb-buildroot that referenced this issue Dec 14, 2017
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Risca pushed a commit to Risca/CHIP-buildroot that referenced this issue Sep 30, 2018
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation build and install a set of
compiler and tool binaries built with CC_FOR_TARGET set to the host compiler.
Also, the go build system is not compatible with ccache, so use
HOSTCC_NOCCACHE.  See golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Risca pushed a commit to Risca/CHIP-buildroot that referenced this issue Sep 30, 2018
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Risca pushed a commit to Risca/CHIP-buildroot that referenced this issue Jun 14, 2020
The go build system doesn't have the notion of cross compiling, but just the
notion of architecture.  When the host and target architectures are different
it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
architectures are the same it will use CC_FOR_TARGET for both host and target
compilation.  To work around this limitation build and install a set of
compiler and tool binaries built with CC_FOR_TARGET set to the host compiler.
Also, the go build system is not compatible with ccache, so use
HOSTCC_NOCCACHE.  See golang/go#11685.

Fixes build errors like these:

  host/usr/bin/go: No such file or directory

  http://autobuild.buildroot.net/results/6664189a6f3a815978e8d0a1d7ef408ca47e2874/

Signed-off-by: Geoff Levand <geoff@infradead.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Risca pushed a commit to Risca/CHIP-buildroot that referenced this issue Jun 14, 2020
Use the host compiler when building host tools.

The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
here.  See golang/go#11685.

Signed-off-by: Geoff Levand <geoff@infradead.org>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
@seankhliao
Copy link
Member

seankhliao commented Jan 26, 2021

resolved with #43808 and CL 284780
tracking with #43078

@golang golang locked and limited conversation to collaborators Jan 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

7 participants