cmd/go: CC variable misparsed when set to a path containing spaces #41400

XamarinPOC · 2020-09-15T11:47:32Z

go build -buildmode=c-shared -o C:\Users\XAMARI~1\AppData\Local\Temp\gomobile-work-285666087\lib\armeabi-v7a\libbasic.so golang.org/x/mobile/example/basic failed: exit status 2

runtime/cgo

exec: "C:\Program": file does not exist

E:\GoWorkDir>go env

set GO111MODULE=auto
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Xamarin Developer\AppData\Local\go-build
set GOENV=C:\Users\Xamarin Developer\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=E:\GoWorkDir\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=E:\GoWorkDir
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\XAMARI~1\AppData\Local\Temp\go-build823121506=/tmp/go-build -gno-record-gcc-switches

davecheney · 2020-09-15T12:23:47Z

Your go installation is corrupt and/or the time stamps have been disrupted, possibly by unpacking one version of go over another

Please delete c:\go and reinstall from a fresh copy

AlexRouSg · 2020-09-15T12:54:33Z

I am actually able to repro this.

exact command: CC="C:\\path with spaces\\bin\\gcc" go build or CC="C:/path with spaces/bin/gcc" go build
works with any source file that uses cgo

It's specific to the CC/CXX class of variables so looks like something is wrong with how go parses them.

cc @ianlancetaylor

bcmills · 2020-09-15T13:17:55Z

@AlexRouSg, thanks for the detail. I can even reproduce it on Linux:

example.com/bin with spaces$ echo "$CC"
/tmp/tmp.jTseVJjT2L/example.com/bin with spaces/gcc

example.com/bin with spaces$ "$CC" --version
gcc (Debian 9.3.0-13) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


example.com/bin with spaces$ go env CC
/tmp/tmp.jTseVJjT2L/example.com/bin

AlexRouSg · 2020-09-15T13:31:46Z

So looks like there are multiple places that needs fixing ...

I've found

go/src/cmd/cgo/gcc.go

Line 1536 in 2963443

func (p *Package) gccBaseCmd() []string {

which only affects running the cgo binary direcly

go build and go tool cgo seems to have a different issue and I can't seem to figure out where that is. But likely it's related to calling strings.Fields or something along those lines.

bcmills · 2020-09-15T13:37:12Z

Yes, there are several places where we're calling strings.Fields to split this sort of command, which seems clearly incorrect to me.

But I think we also try to follow the usual C toolchain behavior for the CC environment variable, so I'm trying to figure out if there is a standard format for escaping paths containing spaces. (Perhaps @ianlancetaylor or @jayconrod knows?)

AlexRouSg · 2020-09-15T13:42:03Z

@XamarinPOC @chaitanyaevoke In the meantime for gomobile, you'll have to install the SDK to a path without spaces

XamarinPOC · 2020-09-15T13:58:39Z

I just uninstalled and installed fresh Go version 1.15.2 but no luck.

I'm trying to run build command => gomobile build -target=android golang.org/x/.............. to generate apk file

XamarinPOC · 2020-09-15T14:00:17Z

@AlexRouSg Which Sdk?? Can you . please explain it .

AlexRouSg · 2020-09-15T14:00:42Z

@XamarinPOC
You need to install the android sdk/ndk to a path without spaces like C:\androidsdk
Make sure you changed the variable ANDROID_HOME too

gopherbot · 2020-09-15T14:27:15Z

Change https://golang.org/cl/255018 mentions this issue: cmd/go: allow the CC path to contain spaces

XamarinPOC · 2020-09-15T14:55:52Z

Yup ,after removing spaces its working now. Can we get this fix in next release.

ianlancetaylor · 2020-09-15T19:24:08Z

Wait. Right now I can run

CC="gcc -O3" go build

and all my cgo files will be built with -O3.

Now, maybe that is a mistake and maybe we should change it. But we definitely shouldn't change that behavior casually. We should discuss it on the mailing list, we should find out whether it will break anybody.

In particular there is no documented way to pass C compiler flags when running make.bash other than setting CC to include the compiler and the flags. And the value of the CC environment variable used when running make.bash because the default value of the C compiler used when building programs.

In other words, we can't just treat this a minor bug that we should just fix. We have to think about how this has worked in the past and how it should work going forward.

bcmills · 2020-09-15T19:55:11Z

The CL that I mailed preserves the property that space-separated arguments following the executable name are treated as arguments: it uses exec.LookPath to identify the first space that could plausibly indicate a break between the command and its arguments, and falls back to the old strings.Fields behavior if none of the possible prefixes resolves to an executable.

I believe that preserves the existing behavior for all commands that would successfully invoke a compiler, and also fixes some number of cases that previously would have failed.

ianlancetaylor · 2020-09-15T23:01:36Z

OK, thanks. I'm a bit worried about the possibility of confusion. I guess it should work but it seems pretty complicated.

bcmills · 2020-09-16T13:10:10Z

Agreed. I was kind of hoping that there would be some existing standard or extension for quoting within CC, but I couldn't find one.

As an alternative, I suppose that we could allow shell-style quoting within the CC variable. (Do you think that would be better or worse?)

AlexRouSg · 2020-09-16T18:26:43Z

Shell style quoting sounds better to me. I can imagine people having args like -I/path with spaces/ which wouldn't be covered by the current CL. Tho that makes me wonder if cgo handles args with spaces correctly for #cgo CFLAGS family of comments or pkgconfig provided flags ...

EDIT:
Looks like #cgo comments understands shell quoting while pkgconfig is kinda weird.
pkgconfig escapes spaces with \ so it looks like -L/path\ with\ spaces which then causes cgo to error

ianlancetaylor · 2020-09-16T23:56:00Z

Shell quoting sounds a little better to me, I guess.

jayconrod · 2021-01-21T23:01:54Z

I'm looking for a narrow fix of #43808 that can be backported, and it will probably affect the eventual solution to this issue.

What I'm thinking right now is that we should support shell quoting using the code in go/build.splitQuoted. That's what we use for #cgo directives today, so it seems reasonable to reuse it here (in the absence of any broader standard).

For #43808, we could quote the CC and GCCGO variables passed from cmd/go to cmd/cgo if needed, then unquote them in cmd/cgo. For 1.17, we could fix this issue by expanding that to cmd/go and anywhere else we deal with CC. We should fix #43078 at the same time.

gopherbot · 2021-07-15T00:08:33Z

Change https://golang.org/cl/334731 mentions this issue: [dev.cmdgo] cmd/internal/str: add functions for quoting and splitting args

This will let cmd/cgo and cmd/link use this package for argument parsing. For #41400 Change-Id: I12ee21151bf3f00f3e8d427faaaab2453c823117 Reviewed-on: https://go-review.googlesource.com/c/go/+/334730 Trust: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>

… args JoinAndQuoteFields does the inverse of SplitQuotedFields: it joins a list of arguments with spaces into one string, quoting arguments that contain spaces or quotes. QuotedStringListFlag uses SplitQuotedFields and JoinAndQuoteFields together to define new flags that accept lists of arguments. For #41400 Change-Id: I4986b753cb5e6fabb5b489bf26aedab889f853f5 Reviewed-on: https://go-review.googlesource.com/c/go/+/334731 Trust: Jay Conrod <jayconrod@google.com> Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>

The CC and CXX environment variables now support spaces and quotes (both double and single). This fixes two issues: first, if CC is a single path that contains spaces (like 'c:\Program Files\gcc\bin\gcc.exe'), that should now work if the space is quoted or escaped (#41400). Second, if CC or CXX has multiple arguments (like 'gcc -O2'), they are now split correctly, and the arguments are passed before other arguments when invoking the C compiler. Previously, strings.Fields was used to split arguments, and the arguments were placed later in the command line. (#43078). Fixes #41400 Fixes #43078 Change-Id: I2d5d89ddb19c94adef65982a8137b01f037d5c11 Reviewed-on: https://go-review.googlesource.com/c/go/+/334732 Trust: Jay Conrod <jayconrod@google.com> Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>

gopherbot · 2021-08-12T22:42:52Z

Change https://golang.org/cl/341934 mentions this issue: cmd/internal/str: move package from cmd/go/internal/str

gopherbot · 2021-08-12T22:42:53Z

Change https://golang.org/cl/341935 mentions this issue: cmd/internal/str: add utilities for quoting and splitting args

gopherbot · 2021-08-12T22:42:53Z

Change https://golang.org/cl/341936 mentions this issue: cmd: support space and quotes in CC and CXX

This will let cmd/cgo and cmd/link use this package for argument parsing. For #41400 Change-Id: I12ee21151bf3f00f3e8d427faaaab2453c823117 Reviewed-on: https://go-review.googlesource.com/c/go/+/334730 Trust: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-on: https://go-review.googlesource.com/c/go/+/341934 Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>

JoinAndQuoteFields does the inverse of SplitQuotedFields: it joins a list of arguments with spaces into one string, quoting arguments that contain spaces or quotes. QuotedStringListFlag uses SplitQuotedFields and JoinAndQuoteFields together to define new flags that accept lists of arguments. For #41400 Change-Id: I4986b753cb5e6fabb5b489bf26aedab889f853f5 Reviewed-on: https://go-review.googlesource.com/c/go/+/334731 Trust: Jay Conrod <jayconrod@google.com> Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-on: https://go-review.googlesource.com/c/go/+/341935

gopherbot · 2022-04-04T19:10:19Z

Change https://go.dev/cl/398058 mentions this issue: cmd/go: quote fragments in CGO_ env variables reported by 'go env'

These fields have been parsed as quoted fields since CL 334732, but we missed the unparsing side in 'go env'. Certain scripts (notably make.ba{sh,t}) expect to be able to set the environment to exactly what 'go env' reports, so for round-trip purposes it is important to match the marshaling and unmarshaling functions. (Noticed while debugging #52009.) Updates #41400 Change-Id: I0ff39b7a6e1328111c285c97cd23f79b723f3c73 Reviewed-on: https://go-review.googlesource.com/c/go/+/398058 Trust: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>

gopherbot · 2023-02-09T14:04:25Z

Change https://go.dev/cl/466875 mentions this issue: cmd/go: handle escapes in pkg-config ldflags output

#16455 handled escapes in pkg-config output but only for cflags. The fix for #41400 left a note that we don't need to parse quotes and unescapes, but it is still necessary to handle spaces in pkg-config output. As cflags has already been processed correctly, we apply the same logic to ldflags here. Fixes #35262 Change-Id: Id01d422b103780f67f89e99ff1df0d8f51a7a137 GitHub-Last-Rev: c67e511 GitHub-Pull-Request: #58429 Reviewed-on: https://go-review.googlesource.com/c/go/+/466875 Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: David Chase <drchase@google.com>

golang#16455 handled escapes in pkg-config output but only for cflags. The fix for golang#41400 left a note that we don't need to parse quotes and unescapes, but it is still necessary to handle spaces in pkg-config output. As cflags has already been processed correctly, we apply the same logic to ldflags here. Fixes golang#35262 Change-Id: Id01d422b103780f67f89e99ff1df0d8f51a7a137 GitHub-Last-Rev: c67e511 GitHub-Pull-Request: golang#58429 Reviewed-on: https://go-review.googlesource.com/c/go/+/466875 Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: David Chase <drchase@google.com>

vault-thirteen · 2023-04-09T14:27:46Z

Why was this issue closed ?
Was the problem fixed ?

ericlagergren · 2023-04-09T14:42:55Z

Yes, see

@gopherbot gopherbot closed this as completed in 742dcba on Aug 16, 2021.

bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Sep 15, 2020

This comment has been minimized.

Sign in to view

bcmills removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Sep 15, 2020

bcmills changed the title ~~# runtime/cgo exec: "C:\\Program": file does not exist~~ cmd/go: exec error with cgo when CC variable contains spaces on Windows Sep 15, 2020

bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 15, 2020

bcmills added this to the Go1.16 milestone Sep 15, 2020

bcmills changed the title ~~cmd/go: exec error with cgo when CC variable contains spaces on Windows~~ cmd/go: CC variable misparsed when set to a path containing spaces Sep 15, 2020

bcmills self-assigned this Sep 15, 2020

bcmills mentioned this issue Dec 8, 2020

cmd/go: cgo builds don't work with "zig cc" #43078

Closed

jayconrod modified the milestones: Go1.16, Go1.17 Jan 6, 2021

AlexRouSg mentioned this issue Jan 20, 2021

cmd/go: handle space in path to C compiler #43808

Closed

bcmills assigned jayconrod and unassigned bcmills Jul 15, 2021

bcmills modified the milestones: Go1.17, Go1.18 Jul 15, 2021

gopherbot closed this as completed in 742dcba Aug 16, 2021

dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels May 26, 2022

rsc unassigned jayconrod Jun 23, 2022

wangfakang mentioned this issue Jan 26, 2023

Unable to build go code on Windows with current bazel setup envoyproxy/envoy#25144

Open

zlasd mentioned this issue Feb 9, 2023

cmd/go: handle escapes in pkg-config ldflags output #58429

Closed

seankhliao mentioned this issue Apr 9, 2023

cgo: C compiler "D:\\Program" not found. Is space symbol in CC variable bugged again ? #59512

Closed

golang locked and limited conversation to collaborators Apr 8, 2024

gopherbot added the FrozenDueToAge label Apr 8, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: CC variable misparsed when set to a path containing spaces #41400

cmd/go: CC variable misparsed when set to a path containing spaces #41400

XamarinPOC commented Sep 15, 2020

davecheney commented Sep 15, 2020

AlexRouSg commented Sep 15, 2020 •

edited

This comment has been minimized.

bcmills commented Sep 15, 2020

AlexRouSg commented Sep 15, 2020

bcmills commented Sep 15, 2020

AlexRouSg commented Sep 15, 2020 •

edited

XamarinPOC commented Sep 15, 2020

XamarinPOC commented Sep 15, 2020

AlexRouSg commented Sep 15, 2020 •

edited

gopherbot commented Sep 15, 2020

XamarinPOC commented Sep 15, 2020

ianlancetaylor commented Sep 15, 2020

bcmills commented Sep 15, 2020

ianlancetaylor commented Sep 15, 2020

bcmills commented Sep 16, 2020

AlexRouSg commented Sep 16, 2020 •

edited

ianlancetaylor commented Sep 16, 2020

jayconrod commented Jan 21, 2021

gopherbot commented Jul 15, 2021

gopherbot commented Aug 12, 2021

gopherbot commented Aug 12, 2021

gopherbot commented Aug 12, 2021

gopherbot commented Apr 4, 2022

gopherbot commented Feb 9, 2023

vault-thirteen commented Apr 9, 2023

ericlagergren commented Apr 9, 2023

cmd/go: CC variable misparsed when set to a path containing spaces #41400

cmd/go: CC variable misparsed when set to a path containing spaces #41400

Comments

XamarinPOC commented Sep 15, 2020

runtime/cgo

davecheney commented Sep 15, 2020

AlexRouSg commented Sep 15, 2020 • edited

This comment has been minimized.

bcmills commented Sep 15, 2020

AlexRouSg commented Sep 15, 2020

bcmills commented Sep 15, 2020

AlexRouSg commented Sep 15, 2020 • edited

XamarinPOC commented Sep 15, 2020

XamarinPOC commented Sep 15, 2020

AlexRouSg commented Sep 15, 2020 • edited

gopherbot commented Sep 15, 2020

XamarinPOC commented Sep 15, 2020

ianlancetaylor commented Sep 15, 2020

bcmills commented Sep 15, 2020

ianlancetaylor commented Sep 15, 2020

bcmills commented Sep 16, 2020

AlexRouSg commented Sep 16, 2020 • edited

ianlancetaylor commented Sep 16, 2020

jayconrod commented Jan 21, 2021

gopherbot commented Jul 15, 2021

gopherbot commented Aug 12, 2021

gopherbot commented Aug 12, 2021

gopherbot commented Aug 12, 2021

gopherbot commented Apr 4, 2022

gopherbot commented Feb 9, 2023

vault-thirteen commented Apr 9, 2023

ericlagergren commented Apr 9, 2023

AlexRouSg commented Sep 15, 2020 •

edited

AlexRouSg commented Sep 15, 2020 •

edited

AlexRouSg commented Sep 15, 2020 •

edited

AlexRouSg commented Sep 16, 2020 •

edited