Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: flag validation fails when pkg-config --libs reports a path containing spaces #35262

Closed
CannibalVox opened this issue Oct 31, 2019 · 6 comments
Labels
FrozenDueToAge help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@CannibalVox
Copy link

CannibalVox commented Oct 31, 2019

What version of Go are you using (go version)?

$ go version
go version go1.13.1 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Stephen Baynham\AppData\Local\go-build
set GOENV=C:\Users\Stephen Baynham\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\Stephen Baynham\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Users\Stephen Baynham\source\repos\go-luajit\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\STEPHE~1\AppData\Local\Temp\go-build884938030=/tmp/go-build -gno-record-gcc-switches

What did you do?

Place the -L directory of a C lib into a directory with a space in it, and access it a pkg-config .pc file.

/*
#cgo pkg-config: luajit
#include "go_luajit.h"
*/
import "C"

func main() {}

Attempt to build:

$ pkg-config --cflags luajit
-IC:/Program\ Files/Git/usr/local/include/luajit-2.0

$ pkg-config --libs luajit
-LC:/Program\ Files/Git/usr/local/lib -lluajit-5.1

$ go build ./...
go build github.com/cannibalvox/luajitter: invalid flag in pkg-config --libs: Files/Git/usr/local/lib

What did you expect to see?

Successful build.

What did you see instead?

go build github.com/cannibalvox/luajitter: invalid flag in pkg-config --libs: Files/Git/usr/local/lib

Context

In 2016, this issue was brought up and was successfully solved, but only for CFlags: #16455

Commit: https://go-review.googlesource.com/c/go/+/32735/6/src/cmd/go/build.go

However, on windows where the (git bash) mingw /usr directory is in C:\Program Files, both the include and lib directories have spaces in them. In issue 16455, it was determined to wait on windows until more information was provided. However, the linked commit fixes linux as well as windows. The only problem is that the fix was only applied to CFLAGS and not LDFLAGS. Having LDFLAGS also use splitPkgConfigOutput instead of strings.Fields would solve this problem entirely. Are there any considerations that might prevent me from PRing a straight 1:1 replacement?

@bcmills
Copy link
Contributor

bcmills commented Nov 5, 2019

CC @ianlancetaylor @jayconrod

@bcmills bcmills changed the title Make pkg-config --libs also handle spaces in paths cmd/go: flag validation fails when pkg-config --libs reports a path containing spaces Nov 5, 2019
@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. help wanted labels Nov 5, 2019
@bcmills bcmills added this to the Backlog milestone Nov 5, 2019
@bcmills
Copy link
Contributor

bcmills commented Nov 5, 2019

Are there any considerations that might prevent me from PRing a straight 1:1 replacement?

I'm not aware of any. Send a PR (and, ideally, a regression test) and we can discuss on the code review?

CannibalVox added a commit to CannibalVox/go that referenced this issue Nov 30, 2019
In 2016, an issue was raised that #cgo pkg-config: annotations did not
work with paths that have spaces in them.  The relevant issue is golang#16455.
This was fixed, but only for CFLAGS.  Because a Windows user was not
available, the windows use case was overlooked (linux is unlikely to
have libs folders with spaces).

The same logic developed for golang#16455 is now applied to --libs output in
addition to --cflags output, and tests have been created for the
getPkgConfigFlags method in exec.go.

Fixes golang#35262
CannibalVox added a commit to CannibalVox/go that referenced this issue Nov 30, 2019
In 2016, an issue was raised that #cgo pkg-config: annotations did not
work with paths that have spaces in them.  The relevant issue is golang#16455.
This was fixed, but only for CFLAGS.  Because a Windows user was not
available, the windows use case was overlooked (linux is unlikely to
have libs folders with spaces).

The same logic developed for golang#16455 is now applied to --libs output in
addition to --cflags output, and tests have been created for the
getPkgConfigFlags method in exec.go.

Fixes golang#35262
CannibalVox added a commit to CannibalVox/go that referenced this issue Nov 30, 2019
In 2016, an issue was raised that #cgo pkg-config: annotations
did not work with paths that have spaces in them.  The relevant
issue is golang#16455. This was fixed, but only for CFLAGS.  Because
a Windows user was not available, the windows use case was
overlooked (linux is unlikely to have libs folders with spaces).

The same logic developed for golang#16455 is now applied to --libs
output in addition to --cflags output, and tests have been
created for the getPkgConfigFlags method in exec.go.

Fixes golang#35262
@gopherbot
Copy link

Change https://golang.org/cl/209437 mentions this issue: cmd/go: make cgo pkg-config work w/ lib paths that have spaces

@gopherbot
Copy link

Change https://go.dev/cl/466875 mentions this issue: cmd/go: handle escapes in pkg-config ldflags output

@gopherbot
Copy link

Change https://go.dev/cl/466864 mentions this issue: cmd/go: fuzz test splitPkgConfigOutput and fix minor bugs

johanbrandhorst pushed a commit to Pryz/go that referenced this issue Feb 12, 2023
golang#16455 handled escapes in pkg-config output but only for cflags. The fix
for golang#41400 left a note that we don't need to parse quotes and unescapes,
but it is still necessary to handle spaces in pkg-config output. As cflags
has already been processed correctly, we apply the same logic to ldflags
here.

Fixes golang#35262

Change-Id: Id01d422b103780f67f89e99ff1df0d8f51a7a137
GitHub-Last-Rev: c67e511
GitHub-Pull-Request: golang#58429
Reviewed-on: https://go-review.googlesource.com/c/go/+/466875
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: David Chase <drchase@google.com>
gopherbot pushed a commit that referenced this issue Feb 15, 2023
In reviewing CL 466875, I noticed that the implementation of
splitPkgConfigOutput from CL 86541 referred to another specific
implementation, and that implementation has had recent changes to fix
deviations from the POSIX specification for shell argument parsing.

Curious about those changes, I decided to fuzz the function to check
whether it agreed in practice with the way a real shell parses
arguments in POSIX mode. It turned out to deviate in several edge
cases, such as backslash-escapes within single quotes, quoted empty
strings, and carriage returns. (We do not expect to see carriage
returns in pkg-config output anyway, but the quote handling might
matter.)

This change updates the implementation to refer to the POSIX
documentation instead of another implementation, and confirms the
behavior with a fuzz test. It may introduce minor deviations from the
pkgconf implementation that was previously used as a reference, but if
so it is plausible that those could be fixed upstream in pkgconf
(like the other recent changes there).

For #35262.
Updates ##23373.

Change-Id: Ifab76e94af0ca9a6d826379f4a6e2028561e615c
Reviewed-on: https://go-review.googlesource.com/c/go/+/466864
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
johanbrandhorst pushed a commit to Pryz/go that referenced this issue Feb 22, 2023
In reviewing CL 466875, I noticed that the implementation of
splitPkgConfigOutput from CL 86541 referred to another specific
implementation, and that implementation has had recent changes to fix
deviations from the POSIX specification for shell argument parsing.

Curious about those changes, I decided to fuzz the function to check
whether it agreed in practice with the way a real shell parses
arguments in POSIX mode. It turned out to deviate in several edge
cases, such as backslash-escapes within single quotes, quoted empty
strings, and carriage returns. (We do not expect to see carriage
returns in pkg-config output anyway, but the quote handling might
matter.)

This change updates the implementation to refer to the POSIX
documentation instead of another implementation, and confirms the
behavior with a fuzz test. It may introduce minor deviations from the
pkgconf implementation that was previously used as a reference, but if
so it is plausible that those could be fixed upstream in pkgconf
(like the other recent changes there).

For golang#35262.
Updates #golang#23373.

Change-Id: Ifab76e94af0ca9a6d826379f4a6e2028561e615c
Reviewed-on: https://go-review.googlesource.com/c/go/+/466864
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
@gopherbot
Copy link

Change https://go.dev/cl/473616 mentions this issue: cmd/go: trim spaces in pkg-config ldflags output

gopherbot pushed a commit that referenced this issue Mar 6, 2023
Fixes #58889
Updates #35262

Change-Id: I1d51aa03f445faaf4f4e9cc412d5499cad526663
Reviewed-on: https://go-review.googlesource.com/c/go/+/473616
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
Auto-Submit: Quim Muntal <quimmuntal@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
@golang golang locked and limited conversation to collaborators Mar 5, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
3 participants