cmd/go: handle pkg-config output containing spaces #16455

ktsakas · 2016-07-21T04:20:49Z

I am using Go v1.6.2 on Windows 10.
I tried to build the git2go package in MinGW-w64 using go build but it fails.
When I run go build -x, I get the following:

CGO_LDFLAGS="-g" "-O2" "-LC:/Program" "Files" "(x86)/libgit2/lib" "-lgit2" 
"C:\\Go\\pkg\\tool\\windows_amd64\\cgo.exe" -objdir 
"C:\\msys64\\tmp\\go-build610654106\\github.com\\libgit2\\git2go\\_obj\\" 
-importpath github.com/libgit2/git2go -- Files (x86)/libgit2/include 
-IC:/Program -I "C:\\msys64\\tmp\\go-build610654106\\github.com\\libgit2\\git2go\\_obj\\" 
blame.go blob.go branch.go checkout.go cherrypick.go clone.go commit.go 
config.go credentials.go describe.go diff.go git.go graph.go 
handles.go ignore.go index.go merge.go note.go object.go odb.go packbuilder.go 
patch.go refdb.go reference.go remote.go repository.go reset.go revparse.go settings.go 
signature.go status.go submodule.go tag.go tree.go walk.go
# github.com/libgit2/git2go
gcc: error: Files: No such file or directory
gcc: error: (x86)/libgit2/include: No such file or directory

Notice how "-LC:/Program" "Files" "(x86)/libgit2/lib" flag is split up on every space and the parts are put in quotes, which seems to cause the issue.
I understand that mingw has problems with spaces in paths, however this should be fixable in build.go.

The issue has been mentioned again in git2go#155.

The text was updated successfully, but these errors were encountered:

0xmohit · 2016-07-21T07:12:47Z

This appears to be the same as #7906.

mattn · 2016-07-21T07:50:50Z

This is related #11868.

quentinmit · 2016-07-29T20:55:08Z

It looks like those CGO_LDFLAGS are coming from git.go:

#cgo pkg-config: libgit2

Can you please try running pkg-config --debug --libs libgit2 and let me know the complete output?

I would like to know if we're misparsing the output of pkg-config, or if it is not outputting the correctly-escaped string.

rsc · 2016-10-21T15:29:08Z

We are certainly misparsing the output of pkg-config, since we use strings.Fields. There's no way that can possibly handle a path like C:/Program Files correctly. The question is how the output is defined so that we can parse it correctly.

On Linux, I get:

$ cat /tmp/whitespace.pc
prefix=/usr
exec_prefix=${prefix}
libdir="${exec_prefix}/white space/lib"
includedir="${prefix}/white space/include"

Name: Whitespace test
Description: Dummy pkgconfig test package for testing pkgconfig
Version: 1.0.0
Requires:
Libs: -L${libdir} -lfoo\ bar "-lbar baz" -r:foo 
Cflags: -I${includedir} -I$(top_builddir) -Iinclude\ dir "-Iother include dir" -Dlala=misc
$ PKG_CONFIG_PATH=/tmp pkg-config --libs whitespace
-r:foo -L/usr/white\ space/lib -lfoo\ bar -lbar\ baz  
$

That is, on Linux pkg-config seems to escape spaces with a backslash. We don't handle that (and we should), but if pkg-config on Windows were emitting those, we'd see them in the CGO_LDFLAGS print, and we don't. For example, if I continue on Linux:

$ cat x.go
package main

/*
#cgo pkg-config: whitespace
*/
import "C"

func main() {}

$ PKG_CONFIG_PATH=/tmp go build -x x.go
WORK=/tmp/go-build540380321
mkdir -p $WORK/command-line-arguments/_obj/
mkdir -p $WORK/command-line-arguments/_obj/exe/
cd /usr/local/google/home/rsc
pkg-config --cflags whitespace
pkg-config --libs whitespace
CGO_LDFLAGS="-g" "-O2" "-r:foo" "-L/usr/white\\" "space/lib" "-lfoo\\" "bar" "-lbar\\" "baz" ./go/pkg/tool/linux_amd64/cgo -objdir $WORK/command-line-arguments/_obj/ -importpath command-line-arguments -- -Dlala=misc "-I/usr/white\\" space/include -I$(top_builddir) "-Iinclude\\" dir "-Iother\\" "include\\" dir -I $WORK/command-line-arguments/_obj/ x.go
# command-line-arguments
gcc: error: space/include: No such file or directory
gcc: error: dir: No such file or directory
gcc: error: include\: No such file or directory
gcc: error: dir: No such file or directory
$

So it looks like, even if we were handling spaces correctly on Linux, handling them correctly on Windows would require different code.

The big question, then, is what pkg-config prints on Windows. It would be great if it printed one arg per line or something like that, so we could split at newline.

@ktsakas, can you please send the output of

pkg-config --cflags libgit2
pkg-config --libs libgit2

on your Windows machine?

Thanks.

quentinmit · 2016-10-24T23:11:56Z

@ktsakas I tried installing MinGW-w64 and MSys2 to debug this, only to discover that their libgit2 package is installed to /mingw64/include. How did you end up with libgit2 in C:\Program Files (x86)\libgit2?

mattn · 2016-10-25T00:06:53Z

As far as I try this on msys2 on my environment,

backslash separator

prefix=c:\Program Files (x86)\libgit
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: foo
Version: 1.2.3
Description: libfoo
URL: http://example.com
Libs: -L${libdir} -lfoo
Cflags: -I${includedir}

$ pkg-config --cflags foo
Files (x86)libgit/include -Ic:Program

slash separator

prefix=c:/Program Files (x86)/libgit
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: foo
Version: 1.2.3
Description: libfoo
URL: http://example.com
Libs: -L${libdir} -lfoo
Cflags: -I${includedir}

$ pkg-config --cflags foo
Files (x86)/libgit/include -Ic:/Program

backslash escaped

prefix=c:/Program\ Files\ (x86)/libgit
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: foo
Version: 1.2.3
Description: libfoo
URL: http://example.com
Libs: -L${libdir} -lfoo
Cflags: -I${includedir}

$ pkg-config --cflags foo
-Ic:/Program\ Files\ (x86)/libgit/include

quoted includedir

prefix=c:/Program Files (x86)/libgit
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir="${prefix}/include"

Name: foo
Version: 1.2.3
Description: libfoo
URL: http://example.com
Libs: -L${libdir} -lfoo
Cflags: -I${includedir}

$ pkg-config --cflags foo
-Ic:/Program\ Files\ (x86)/libgit/include

rsc · 2016-11-01T17:43:49Z

@ktsakas we are waiting on more info from you about pkg-config on your Windows system.

In the meanwhile, we will fix the Unix case and leave Windows using strings.Fields.

gopherbot · 2016-11-03T22:50:08Z

CL https://golang.org/cl/32735 mentions this issue.

quentinmit · 2016-11-07T20:30:34Z

It looks like this is caused by libgit2 generating an incorrect .pc file: https://github.com/libgit2/libgit2/blob/master/libgit2.pc.in

The includedir= argument needs to be quoted if the path contains a space, otherwise pkg-config treats it as multiple arguments.

quentinmit added this to the Go1.8 milestone Jul 29, 2016

quentinmit added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 29, 2016

broady changed the title ~~go build passes incorrect gcc parameters on mingw64~~ cmd/go: go build passes incorrect gcc parameters on mingw64 Aug 19, 2016

aronatkins mentioned this issue Sep 1, 2016

cmd/cgo: ${SRCDIR} does not handle paths with % #16959

Closed

rsc changed the title ~~cmd/go: go build passes incorrect gcc parameters on mingw64~~ cmd/go: handle pkg-config output containing spaces Oct 21, 2016

rsc added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Oct 21, 2016

quentinmit self-assigned this Oct 24, 2016

mattn mentioned this issue Nov 3, 2016

cmd/cgo: support clang on Windows #17014

Open

gopherbot closed this as completed in c2917af Nov 7, 2016

golang locked and limited conversation to collaborators Nov 7, 2017

gopherbot added the FrozenDueToAge label Nov 7, 2017

rsc unassigned quentinmit Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: handle pkg-config output containing spaces #16455

cmd/go: handle pkg-config output containing spaces #16455

ktsakas commented Jul 21, 2016

0xmohit commented Jul 21, 2016

mattn commented Jul 21, 2016

quentinmit commented Jul 29, 2016

rsc commented Oct 21, 2016

quentinmit commented Oct 24, 2016

mattn commented Oct 25, 2016

rsc commented Nov 1, 2016

gopherbot commented Nov 3, 2016

quentinmit commented Nov 7, 2016

cmd/go: handle pkg-config output containing spaces #16455

cmd/go: handle pkg-config output containing spaces #16455

Comments

ktsakas commented Jul 21, 2016

0xmohit commented Jul 21, 2016

mattn commented Jul 21, 2016

quentinmit commented Jul 29, 2016

rsc commented Oct 21, 2016

quentinmit commented Oct 24, 2016

mattn commented Oct 25, 2016

backslash separator

slash separator

backslash escaped

quoted includedir

rsc commented Nov 1, 2016

gopherbot commented Nov 3, 2016

quentinmit commented Nov 7, 2016