Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/tools: replace use of x/sys/execabs with os/exec #57304

Open
Manbeardo opened this issue Dec 14, 2022 · 8 comments
Open

x/tools: replace use of x/sys/execabs with os/exec #57304

Manbeardo opened this issue Dec 14, 2022 · 8 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone
Unreleased

Comments

@Manbeardo
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.19.4 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\James\AppData\Local\go-build
set GOENV=C:\Users\James\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\James\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\James\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.19.4
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\workspace\golang.org\x\sys\go.mod
set GOWORK=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=C:\Users\James\AppData\Local\Temp\go-build2317869406=/tmp/go-build -gno-record-gcc-switches

What did you do?

I'm trying to run gqlgen (https://github.com/99designs/gqlgen) as part of a Bazel build.

What did you expect to see?

There should be some way to make gqlgen run correctly in Bazel

What did you see instead?

gqlgen fails with the error: go resolves to executable in current directory (.\external\go_sdk\bin\go.exe)

I originally commented on the execabs proposal here: #43724 (comment)

Bazel makes it intentionally difficult to ever get an absolute path in order to make builds more hermetic. To that end, Bazel runs executable files in this arrangement:

  • CWD is a specially-built "execroot" directory which contains all the executable step's dependencies
  • PATH is empty unless rule authors explicitly add elements to it
  • rule authors are only able to add relative paths to PATH

This creates a somewhat-unusual situation where the paths to be found by exec.LookPath/execabs.LookPath aren't absolute, but they also aren't files directly in CWD. Since the paths aren't absolute, the current behavior of exec.LookPath/execabs.LookPath is to return an error. However, my understanding is that the LookPath security vulnerability is only relevant if the relative directory is conventionally listed on PATH variables in interactive sessions, but the only such conventionally-used relative path that I'm aware of is .. If it's true that . is the only relative path that regularly appears on users' PATH variables, I think it should be safe to update the enforcement check to allow relative paths outside of .:

if !filepath.IsAbs(path) && filepath.Dir(path) == '.' && godebug.Get("execerrdot") != "0" {
	return path, &Error{file, ErrDot}
}
@Manbeardo
Copy link
Author

This is the output I'm getting from Bazel right now. Note that the path is relative, but not to a file directly in ..

sad log

@Manbeardo
Copy link
Author

Also, I guess the error message is technically incorrect if you go edge-case hunting and put .. on your PATH for some reason?

@Manbeardo
Copy link
Author

Also, technically I can work around this by having Bazel run a wrapper around gqlgen that converts the elements of PATH from relative to absolute paths before actually running gqlgen, but that's a really ugly solution.

@seankhliao
Copy link
Member

Duplicate of #53962

@seankhliao seankhliao marked this as a duplicate of #53962 Dec 14, 2022
@seankhliao seankhliao closed this as not planned Won't fix, can't repro, duplicate, stale Dec 14, 2022
@Manbeardo
Copy link
Author

GODEBUG=execerrdot=0 does not solve this problem (it was set when I produced that screenshot) because:

  • golang.org/x/sys/execabs does not respect that debug flag
  • golang.org/x/tools/go/packages uses golang.org/x/sys/execabs to execute commands
  • github.com/99designs/gqlgen depends upon golang.org/x/tools/go/packages

Further, the GODEBUG flag was created in the spirit of a temporary migration measure, but there's no reasonable way to work around this problem as a consumer of golang.org/x/tools/go/packages because:

  • every successful code path in packages.Load calls execabs.Command or execabs.CommandContext
  • execabs.Command and execabs.CommandContext do not respect GODEBUG=execerrdot=0 and always return an error if the executable path is not absolute

The ONLY workaround that I see here is to add a wrapper in Bazel that makes the PATH absolute, which compromises the hermeticness of the Bazel build and doesn't seem strictly necessary because Bazel's behavior can be allowed while still disallowing the use of . in PATH.

cc @ianlancetaylor

@seankhliao seankhliao changed the title os/exec: LookPath always returns an error when called inside Bazel x/tools: replace use of x/sys/execabs with os/exec Dec 14, 2022
@seankhliao seankhliao added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 14, 2022
@seankhliao
Copy link
Member

cc @golang/tools-team

@seankhliao seankhliao reopened this Dec 14, 2022
@gopherbot gopherbot added the Tools This label describes issues relating to any tools in the x/tools repository. label Dec 14, 2022
@gopherbot gopherbot added this to the Unreleased milestone Dec 14, 2022
@ianlancetaylor
Copy link
Contributor

One way to support the Bazel use case is to make the GODEBUG=execerrdot setting permanent and to add support for GODEBUG=execerrdot to x/sys/execabs.

In general I don't think we want x/tools/go/packages to ignore exec.ErrDot errors; we explicitly decided to not do that in https://go.dev/blog/path-security.

@Manbeardo
Copy link
Author

If the route forward is to respect the GODEBUG flag inside x/sys/execabs, should there also be a separate issue to move/copy the godebug package from a stdlib internal package to something that's usable in x/sys/execabs?

slimsag added a commit to slimsag/autogold that referenced this issue Feb 16, 2023
@slimsag
Add ENABLE_BAZEL_HACKS=true for poor Bazel users
52c04f4 
Bazel build systems try to keep hermeticity by setting PATH="." - but Go does not like this as
it is a security concern; almost all Go tooling relies on golang.org/x/tools/go/packages.Load
which behind the scenes must invoke `go` and uses the secure version of x/sys/execabs, but
ultimately this means Go tools like autogold cannot be run in Bazel:

golang/go#57304

Autogold relies on `packages.Load` in order to determine the Go package name / path when writing
out a Go AST representation of the value passed in; but the issue above means autogold cannot be
used with Bazel without removing "." from your PATH, which Bazel claims breaks hermeticity (one
of the whole reasons people use Bazel.)

For Bazel users, we allow them to set ENABLE_BAZEL_HACKS=true which causes autogold to guess/infer
package names and paths using stack trace information and import paths. This is not perfect, it
doesn't respect packages whose import paths donot match their defined `package foo` statement for
example - but it's sufficient to enable autogold to be used in Bazel build environments where the
above Go/Bazel bug is found.

Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
@slimsag slimsag mentioned this issue Feb 16, 2023
Merged
1 task
slimsag added a commit to hexops/autogold that referenced this issue Feb 20, 2023
@slimsag
Add ENABLE_BAZEL_PACKAGES_LOAD_HACK=true for poor Bazel users (#40)
f2fed4f 
Bazel build systems try to keep hermeticity by setting PATH="." - but Go does not like this as
it is a security concern; almost all Go tooling relies on golang.org/x/tools/go/packages.Load
which behind the scenes must invoke `go` and uses the secure version of x/sys/execabs, but
ultimately this means Go tools like autogold cannot be run in Bazel:

golang/go#57304

Autogold relies on `packages.Load` in order to determine the Go package name / path when writing
out a Go AST representation of the value passed in; but the issue above means autogold cannot be
used with Bazel without removing "." from your PATH, which Bazel claims breaks hermeticity (one
of the whole reasons people use Bazel.)

For Bazel users, we allow them to set ENABLE_BAZEL_PACKAGES_LOAD_HACK=true which causes autogold to guess/infer
package names and paths using stack trace information and import paths. This is not perfect, it
doesn't respect packages whose import paths donot match their defined `package foo` statement for
example - but it's sufficient to enable autogold to be used in Bazel build environments where the
above Go/Bazel bug is found.

Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
4 participants
@Manbeardo @ianlancetaylor @gopherbot @seankhliao