cmd/go: cache results of exec.LookPath #36768

Zyxon123 · 2020-01-25T19:24:37Z

What version of Go are you using (`go version`)?

$ go version
go version go1.13.6 windows/amd64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (`go env`)?

windows 10 64 bit version 1809
intel i5-8265U

go env Output

$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\lidic\AppData\Local\go-build
set GOENV=C:\Users\lidic\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\lidic\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Go\src\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\lidic\AppData\Local\Temp\go-build756524190=/tmp/go-build -gno-record-gcc-switches
GOROOT/bin/go version: go version go1.13.6 windows/amd64
GOROOT/bin/go tool compile -V: compile version go1.13.6

What did you do?

Hi,

I followed the steps here after I installed Go. However, running go run hello.go takes around 10 - 20 seconds for it to output “hello, world”. Same thing with go build. I’ve also tried reinstalling and excluding my Go folder and Go temp folder from windows defender (the only antivirus I use). I've even disabled real time protection. How do I fix this? Any help is appreciated.

Thanks!

What did you expect to see?

I expect a hello world program to be compiled within seconds, other users are able to do so.

What did you see instead?

Takes 10-20 seconds to compile instead.

PS C:\Go\src\hello> Measure-Command {go run hello.go}


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 15
Milliseconds      : 582
Ticks             : 155821163
TotalDays         : 0.000180348568287037
TotalHours        : 0.00432836563888889
TotalMinutes      : 0.259701938333333
TotalSeconds      : 15.5821163
TotalMilliseconds : 15582.1163

go run -x output

C:\Go\src\hello>go run -x hello.go
WORK=C:\Users\lidic\AppData\Local\Temp\go-build803697114
mkdir -p $WORK\b001\
cat >$WORK\b001\importcfg.link << 'EOF' # internal
packagefile command-line-arguments=C:\Users\lidic\AppData\Local\go-build\a8\a8c4809a5f80405952f1e99a0a10c35826595987039565f2c4eed84d206e5060-d
packagefile fmt=c:\go\pkg\windows_amd64\fmt.a
packagefile runtime=c:\go\pkg\windows_amd64\runtime.a
packagefile errors=c:\go\pkg\windows_amd64\errors.a
packagefile internal/fmtsort=c:\go\pkg\windows_amd64\internal\fmtsort.a
packagefile io=c:\go\pkg\windows_amd64\io.a
packagefile math=c:\go\pkg\windows_amd64\math.a
packagefile os=c:\go\pkg\windows_amd64\os.a
packagefile reflect=c:\go\pkg\windows_amd64\reflect.a
packagefile strconv=c:\go\pkg\windows_amd64\strconv.a
packagefile sync=c:\go\pkg\windows_amd64\sync.a
packagefile unicode/utf8=c:\go\pkg\windows_amd64\unicode\utf8.a
packagefile internal/bytealg=c:\go\pkg\windows_amd64\internal\bytealg.a
packagefile internal/cpu=c:\go\pkg\windows_amd64\internal\cpu.a
packagefile runtime/internal/atomic=c:\go\pkg\windows_amd64\runtime\internal\atomic.a
packagefile runtime/internal/math=c:\go\pkg\windows_amd64\runtime\internal\math.a
packagefile runtime/internal/sys=c:\go\pkg\windows_amd64\runtime\internal\sys.a
packagefile internal/reflectlite=c:\go\pkg\windows_amd64\internal\reflectlite.a
packagefile sort=c:\go\pkg\windows_amd64\sort.a
packagefile sync/atomic=c:\go\pkg\windows_amd64\sync\atomic.a
packagefile math/bits=c:\go\pkg\windows_amd64\math\bits.a
packagefile internal/oserror=c:\go\pkg\windows_amd64\internal\oserror.a
packagefile internal/poll=c:\go\pkg\windows_amd64\internal\poll.a
packagefile internal/syscall/windows=c:\go\pkg\windows_amd64\internal\syscall\windows.a
packagefile internal/testlog=c:\go\pkg\windows_amd64\internal\testlog.a
packagefile syscall=c:\go\pkg\windows_amd64\syscall.a
packagefile time=c:\go\pkg\windows_amd64\time.a
packagefile unicode/utf16=c:\go\pkg\windows_amd64\unicode\utf16.a
packagefile unicode=c:\go\pkg\windows_amd64\unicode.a
packagefile internal/race=c:\go\pkg\windows_amd64\internal\race.a
packagefile internal/syscall/windows/sysdll=c:\go\pkg\windows_amd64\internal\syscall\windows\sysdll.a
packagefile internal/syscall/windows/registry=c:\go\pkg\windows_amd64\internal\syscall\windows\registry.a
EOF
mkdir -p $WORK\b001\exe\
cd .
"c:\\go\\pkg\\tool\\windows_amd64\\link.exe" -o "C:\\Users\\lidic\\AppData\\Local\\Temp\\go-build803697114\\b001\\exe\\hello.exe" -importcfg "C:\\Users\\lidic\\AppData\\Local\\Temp\\go-build803697114\\b001\\importcfg.link" -s -w -buildmode=exe -buildid=boKU76zxCBTD9TQHu-ws/-9UkAzfu17Kq0_a_DtT8/N9PQnONczWnG7Jk1PdaH/boKU76zxCBTD9TQHu-ws -extld=gcc "C:\\Users\\lidic\\AppData\\Local\\go-build\\a8\\a8c4809a5f80405952f1e99a0a10c35826595987039565f2c4eed84d206e5060-d"
$WORK\b001\exe\hello.exe
hello, world

The text was updated successfully, but these errors were encountered:

ghost · 2020-01-25T22:38:27Z

go run (timing)

It may be of interest to the Golang developers to check the timing of other available downloads.

go1.13.6.linux-386.tar.gz

bash-4.3$ uname -mp                             
i686 Intel(R) Atom(TM) CPU N270   @ 1.60GHz
bash-4.3$ free --giga | perl -lae 'print @F[0,1] if $. == 2'
Mem:1
bash-4.3$ export PATH=/usr/local/go/bin:${PATH}
bash-4.3$ mkdir -p ~/go/src/hello
bash-4.3$ cat << EOF | gofmt > ~/go/src/hello/hello.go
> package main
> 
> import "fmt"
> 
> func main() {
> fmt.Printf("hello, world\n")
> }
> EOF
bash-4.3$ /usr/bin/time -p go run ~/go/src/hello/hello.go
hello, world
real 2.33
user 1.59
sys 0.31
bash-4.3$ rm -r ~/.cache/go-build
bash-4.3$ $PLAN9/bin/time go run ~/go/src/hello/hello.go
hello, world
1.57u 0.35s 1.86r 	 go run /home/eric/go/src/hello/hello.go
bash-4.3$ file /usr/local/go/bin/go
/usr/local/go/bin/go: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, Go BuildID=bVwMwAdXBpYp4PBvIrqZ/vbfX15UF6cgq3THigMUx/QSxQRVWmK-8uFUlegS1A/TD0XQ0eQXwJkvrDVR_B1, not stripped
bash-4.3$ rm -r ~/.cache/go-build
bash-4.3$

Stripping the files from go1.13.6.linux-386.tar.gz the time is slightly different.

GNU 1.7
real 2.23
user 1.49
sys 0.36

Plan 9 from User Space
1.58u 0.31s 1.81r

ianlancetaylor · 2020-01-25T22:48:44Z

You didn't fill out the issue template. What is the output of go env?

Also, what is the output of go run -x hello.go?

Zyxon123 · 2020-01-25T23:33:58Z

You didn't fill out the issue template. What is the output of go env?

Also, what is the output of go run -x hello.go?

Ok I've updated my post

ianlancetaylor · 2020-01-26T01:23:01Z

Thanks. Unfortunately I still don't see why it is so slow.

What is the output of go build -ldflags=-v hello.go?

Since this is Windows, the problem may be due to a virus checker.

Zyxon123 · 2020-01-26T01:51:56Z

Thanks. Unfortunately I still don't see why it is so slow.

What is the output of go build -ldflags=-v hello.go?

Since this is Windows, the problem may be due to a virus checker.

Here's the output:

PS C:\go\src\hello> go build -ldflags=-v hello.go
# command-line-arguments
HEADER = -H11 -T0xffffffffffffffff -R0xffffffff
 0.00 deadcode
 0.01 symsize = 0
 0.08 pclntab=460347 bytes, funcdata total 109854 bytes
 0.09 dodata
 0.09 dwarf
 0.12 reloc
 0.12 asmb
 0.12 codeblk
 0.12 rodatblk
 0.13 datblk
 0.13 sym
 0.13 dwarf
 0.13 headr
 0.13 symsize = 0
 0.13 cpu time
53707 symbols
63284 liveness data

ianlancetaylor · 2020-01-26T02:54:07Z

Thanks. I was wondering if it were using external linking, but it's not.

I have no explanation other than possibly a virus checker. Sorry.

ghost · 2020-01-26T04:19:34Z

The go.exe env output lists GOPATH=C:\Users\lidic\go.

From your update it looks like C:\Go\src\hello is where your source file is.

I installed Golang only once on Windows 10 (64-bit).

I downloaded a go1.xx.x.windows-amd64.zip file, extracted it, opened an elevated command prompt and moved it to C:\.

Next I disabled via the context menu indexing for C:\go.

Does locating hello.go in C:\Users\lidic\go\src\hello make any difference?

Zyxon123 · 2020-01-26T07:29:36Z

The go.exe env output lists GOPATH=C:\Users\lidic\go.

From your update it looks like C:\Go\src\hello is where your source file is.

I installed Golang only once on Windows 10 (64-bit).

I downloaded a go1.xx.x.windows-amd64.zip file, extracted it, opened an elevated command prompt and moved it to C:\.

Next I disabled via the context menu indexing for C:\go.

Does locating hello.go in C:\Users\lidic\go\src\hello make any difference?

I moved hello.go to C:\Users\lidic\go\src\hello and it made no difference. I also changed my GOPATH to C:\Go and it made no difference too.

ghost · 2020-01-26T20:34:50Z

I received a reply to an e-mail I sent to Redmond.

Doesn't look like it's specific to AV/Defender.

The way you rule out (or in) Defender is to temporarily disable it, do the compile, and then compare on vs off times. Looks like they've done that, though.

ghost · 2020-01-26T20:37:56Z

#36768 (comment).

The root of the Go tree cannot be included as a value in the Go path.

proyb6 · 2020-01-27T11:54:03Z

I'm not using Windows, my assumption if did you have made any Windows performance tuning e.g. disable timestamp or etc?

Zyxon123 · 2020-01-28T01:02:38Z

I'm not using Windows, my assumption if did you have made any Windows performance tuning e.g. disable timestamp or etc?

No. I am able to reproduce this behaviour on another PC that is also windows 10 with i7.

0intro · 2020-01-28T19:42:02Z

@Zyxon123 asked me to provide the time measurements on Plan 9 for reference.

% go version
go version go1.13.6 plan9/386
% time go run hello.go
hello, world
0.04u 0.25s 0.77r 	 go run hello.go

ALTree · 2020-01-28T20:57:53Z

This is sometimes caused by a broken installation. In this Go1.13 install the first you did on your machine? Or did you update Go from an older version you had?

Also, when you write:

I’ve also tried reinstalling

do you mean reinstalling over the existing version? If yes, can you instead completely nuke every trace of Go from your machine, and try to follow the installation steps again, from scratch?

Zyxon123 · 2020-01-28T21:42:13Z

Go 1.13.6 is the first version I did on my machine.

This is sometimes caused by a broken installation. In this Go1.13 install the first you did on your machine? Or did you update Go from an older version you had?

Also, when you write:

I’ve also tried reinstalling

do you mean reinstalling over the existing version? If yes, can you instead completely nuke every trace of Go from your machine, and try to follow the installation steps again, from scratch?

I have just uninstalled Go again by deleting C:\Go, C:\Users\lidic\AppData\Local\go-build, C:\Users\lidic\AppData\Local\Temp\go-buildxxxxx, and removing it from PATH. I also clicked the "Uninstall" option after opening the installer. However, the issue still persists (with exclusion and real time protection off).

PS C:\Go\src\hello> Measure-Command {go run hello.go}


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 17
Milliseconds      : 229
Ticks             : 172295263
TotalDays         : 0.000199415813657407
TotalHours        : 0.00478597952777778
TotalMinutes      : 0.287158771666667
TotalSeconds      : 17.2295263
TotalMilliseconds : 17229.5263

ALTree · 2020-01-30T08:07:23Z

@Zyxon123 thanks for trying. If your installation is not broken and you're absolutely sure your antivirus is not disturbing the compilation process, I don't have any other idea to fix this.

These kind of slow-compilation-on-windows issues are most commonly caused by a broken installation or by a nosy antivirus. It's clear that there is something wrong with your machines and/or with the way you're installing Go, since many many people are able to compile a Go hello world in < 1s on Windows; but I don't know what that may be.

ghost · 2020-02-02T20:43:27Z

Could someone other than the OP add 9pm's time.exe output
for go1.13.6 running on Windows 10 64 bit.

go1.10.windows-386.zip
9pm051031.zip (S. Quinlan's 9pm)

% time.exe /Programmer/go/bin/go.exe run hello.go
hello, world
0.438u 0.359s 2.125r 	 /Programmer/go/bin/go.exe run hello.go
%

ctd1500 · 2020-02-03T23:46:59Z

@apparluk Windows 10 64-bit time output for reference

# go version
go version go1.13.6 windows/amd64

# ..\9pm\bin\time.exe go run hello.go
hello, world
0.031u 0.188s 1.022r     go run hello.go

proyb6 · 2020-02-04T05:37:36Z

I'm not using Windows, my assumption if did you have made any Windows performance tuning e.g. disable timestamp or etc?

No. I am able to reproduce this behaviour on another PC that is also windows 10 with i7.

Mind to share the model and brand of your PC, how much free memory does it has?

ghost · 2020-02-04T20:36:24Z

The program in which the command time.exe go.exe run hello.go is executed (on win10 64-bit) might be factored out by running that command (time.exe referring to Quinlan's 9pm) in 9term.exe, cmd.exe and powershell.exe.

Here's a result from Windows 10 (run in 9term.exe).

go1.13.7.windows-amd64.zip

% time.exe /go/bin/go.exe run hello.go
hello, world
0.391u 1.563s 6.071r      /go/bin/go.exe run hello.go
%

I didn't have the opportunity to obtain the specs on the hardware,
nor to clear out go-build and rerun in powershell.exe.

There was an earlier ticket regarding CRLF, and IIRC, PowerShell (by default) generates UTF-16 content, when running a go executable to output a string, and that is then redirected to a file.

ghost · 2020-02-10T03:52:47Z

Measurement on macOS for reference.

go1.13.7.darwin-amd64.tar.gz

$time go run hello.go
hello, world

real   0m1.410s
user   0m0.306s
sys    0m0.377s
$

MaxSem · 2020-02-10T09:19:40Z

@Zyxon123, does it still run slowly if you try it several times in a rapid succession? For me (with both workspace and Go installation whitelisted in Defender), it drops from

real    0m3.609s
user    0m0.015s
sys     0m0.000s

to

real    0m0.561s
user    0m0.000s
sys     0m0.000s

And that's on a M.2 drive with 3Gb/s read speed.

andreykaipov · 2020-08-11T01:14:31Z

I'm experiencing slow successive builds too. Tried both 1.14.2 and 1.15rc2.

I've made sure my GOCACHE is set and excluded it from Windows Defender, among other directories like the Go installation and project directory. I don't think hardware is an issue because building the project within my WSL distro is quick as expected. Hell - it's quicker iteration for me to cross-compile to Windows from the WSL distro! 😅

This gave me the idea to try both binaries from the same environment:

$ uname -a
Linux Andrey 4.4.0-18362-Microsoft #836-Microsoft Mon May 05 16:04:00 PST 2020 x86_64 x86_64 x86_64 GNU/Linux

$ alias gowindows=/mnt/c/Users/Andrey/sdk/go1.14.2/bin/go.exe

$ alias golinux=/home/andrey/bin/go

$ gowindows version
go version go1.14.2 windows/amd64

$ golinux version
go version go1.14.2 linux/amd64

$ time gowindows build hello.go

real    0m5.237s
user    0m0.000s
sys     0m0.031s

$ time golinux build hello.go

real    0m0.402s
user    0m0.031s
sys     0m0.422s

For completeness the file contents are:

package main

import "fmt"

func main() {
        fmt.Println("hello")
}

Both installations were installed similarly - just unzipping the folder and adding a/b/c/go/bin to the system path. I even tried using the MSI installer but no dice.

Strangely it's not just go build that's slow:

$ time gowindows env >/dev/null

real    0m3.101s
user    0m0.000s
sys     0m0.000s

$ time golinux env >/dev/null

real    0m0.146s
user    0m0.016s
sys     0m0.172s

Something is definitely wrong but I can't find what.

davecheney · 2020-08-11T01:22:57Z

Could the difference in build time be explained by the file system bridge that WSL uses?

andreykaipov · 2020-08-11T04:23:48Z

Could the difference in build time be explained by the file system bridge that WSL uses?

Are you saying that since I'm accessing go.exe from the WSL side, that might be the cause for the slowness? I've also tried downloading Go for Windows onto the WSL distro and accessing it from there directly, and experience similar behavior, so I'm not sure if that's part of it.

One interesting thing I've noticed is when building with -x, there's a significant delay between when the WORK directory is printed and when the rest of the compilation commands are printed:

C:\Users\Andrey\Projects\Test> go build -x -ldflags=-v hello.go
WORK=C:\Users\Andrey\AppData\Local\Temp\go-build674494450
██████ significant delay here ██████
mkdir -p $WORK\b001\
cat >$WORK\b001\importcfg.link << 'EOF' # internal
packagefile command-line-arguments=C:\Users\Andrey\.cache\go\f1\f174f809fb722f16637244c56012840d3d558a2a2b7a0dc8d5cdcfd33fd35af1-d
packagefile fmt=C:\Users\Andrey\sdk\go1.14.2\pkg\windows_amd64\fmt.a
packagefile runtime=C:\Users\Andrey\sdk\go1.14.2\pkg\windows_amd64\runtime.a
...

I think I've narrowed it down to this code range just by playing around with build flags to cause fatal errors that failed quickly. https://github.com/golang/go/blob/go1.14.2/src/cmd/go/internal/work/exec.go#L98-L190

I've got no clue what any of this code does, but I hope I'll find it within myself to keep going and build Go from source with debug print statements to narrow down the slowness further, but I'm considering just working off the MacBook at this point lol

davecheney · 2020-08-11T04:37:47Z

Are you saying that since I'm accessing go.exe from the WSL side, that might be the cause for the slowness?

That's my suggestion. I don't have a WSL host to verify, sorry.

qmuntal · 2023-07-21T15:28:42Z

I have ~~a counter~~ an additional proposal to solve the original issue: make exec.LookPath faster. The most important performance problem in exec.LookPath seems to be that it calls os.Stat on every file candidate to know if it exist. os.Stat will internally call syscall.GetFileAttributesEx and syscall.CreateFile for each non-existan fail, which is slow. We could replace os.Stat by a simpler syscall.GetFileAttributes, as we really only need to know if the file exists and that it is not a directory. There would be two behavior differences:

syscall.GetFileAttributes will fail with protected system files like C:\pagefile.sys, while os.Stat wouldn't. This is a no-issue because those files are not executable anyway.
os.Stat detects broken reparse points, while the new approach would not. Therefore os.LookPath could return a path that points to a broken link. Symbolic links are not common in Windows, broken symbolic links are even harder to see, I don't even know how to get one, so a nice speed bump could justify not covering this edge case.

The code could look like this:

func chkStat(file string) error {
	p, err := syscall.UTF16PtrFromString(file)
	if err != nil {
		return &os.PathError{Op: "Stat", Path: file, Err: err}
	}
	a, err := syscall.GetFileAttributes(p)
	if err != nil {
		return &os.PathError{Op: "GetFileAttributes", Path: file, Err: err}
	}
	if a&syscall.FILE_ATTRIBUTE_DIRECTORY != 0 {
		return fs.ErrPermission
	}
	return nil
}

This change makes exec.LookPath twice as fast in my computer when looking for a non-existing binary:

goos: windows
goarch: amd64
pkg: os/exec
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
            │   old.txt    │               new.txt                │
            │    sec/op    │    sec/op     vs base                │
LookPath-12   8.815m ± 18%   4.275m ± 12%  -51.50% (p=0.000 n=10)

            │   old.txt    │               new.txt               │
            │     B/op     │     B/op      vs base               │
LookPath-12   95.72Ki ± 0%   95.61Ki ± 0%  -0.11% (p=0.000 n=10)

            │  old.txt   │              new.txt              │
            │ allocs/op  │ allocs/op   vs base               │
LookPath-12   891.0 ± 0%   890.0 ± 0%  -0.11% (p=0.000 n=10)

This CL package exec.LookPath to internal/par. LookPath and adds cache. BenchmarkLookPath-4 26475454 47.84 ns/op 0 B/op 0 allocs/op Fixes golang#36768 Change-Id: I7b80d6d86d57c88da212ff46ec881ae0dc03efe3

Follow up on CL 511458, see https://go-review.googlesource.com/c/go/+/511458/2..4/src/cmd/go/main.go#b270 . Fixes golang#36768 Change-Id: Icc2a4dbb1219b1d69dd10a900478957b0e975847

gopherbot · 2023-07-22T02:46:09Z

Change https://go.dev/cl/512155 mentions this issue: os/exec: fix GOOS=windows,Cmd.Run always calls LookPath before returning

Follow up on CL 511458, see https://go-review.googlesource.com/c/go/+/511458/2..4/src/cmd/go/main.go#b270 . Fixes golang#36768 Change-Id: Icc2a4dbb1219b1d69dd10a900478957b0e975847

Follow up on CL 511458, see https://go-review.googlesource.com/c/go/+/511458/2..4/src/cmd/go/main.go#b270 . For golang#36768 Change-Id: Icc2a4dbb1219b1d69dd10a900478957b0e975847

Follow up on CL 511458, see https://go-review.googlesource.com/c/go/+/511458/2..4/src/cmd/go/main.go#b270 . For #36768. Change-Id: Icc2a4dbb1219b1d69dd10a900478957b0e975847 Change-Id: Icc2a4dbb1219b1d69dd10a900478957b0e975847 GitHub-Last-Rev: bac7e66 GitHub-Pull-Request: #61517 Reviewed-on: https://go-review.googlesource.com/c/go/+/512155 Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com>

qiulaidongfeng · 2023-08-12T10:28:44Z

@bcmills CL 511458 is ready for review, Can you remove Hold.

This CL package exec.LookPath to internal/cfg.LookPath and adds cache. BenchmarkLookPath-4 26475454 47.84 ns/op 0 B/op 0 allocs/op Fixes golang#36768 Change-Id: I199a780d1eab9bd5397bb3759bb42191fff716e9 Change-Id: I199a780d1eab9bd5397bb3759bb42191fff716e9 GitHub-Last-Rev: 04cb24b GitHub-Pull-Request: golang#61464

This CL package exec.LookPath to internal/cfg.LookPath and adds cache. BenchmarkLookPath-4 24149096 50.48 ns/op 0 B/op 0 allocs/op Fixes golang#36768 Change-Id: I199a780d1eab9bd5397bb3759bb42191fff716e9 Change-Id: I199a780d1eab9bd5397bb3759bb42191fff716e9 GitHub-Last-Rev: 04cb24b GitHub-Pull-Request: golang#61464

ianlancetaylor · 2023-09-12T00:40:01Z

I'm reverting CL 512155, which mentions this issue; see https://go.dev/cl/527337 for more information. As far as I can tell, that reversion does not affect this issue directly. Please let us know if that is not the case. Thanks.

qiulaidongfeng · 2023-09-12T05:00:31Z

I am the author of CL 512155, and I remember that without this CL, on Windows, exec.LookPath would be called once regardless. This means that on Windows, this issue has not been resolved because the actual cached result is not being used, which is approximately equivalent to no cache.

mvdan · 2023-09-12T09:38:39Z

Now discovering this thread - kudos to @omaskery and @jayconrod for the investigation above :)

@qmuntal I think your suggestion in #36768 (comment) sounds reasonable - given that this issue is now solved, perhaps raise a new issue so that it doesn't get lost. You could also raise a CL at the same time, although it's easier for a CL to get lost when there's no issue to track it with.

gopherbot · 2023-09-13T16:08:52Z

Change https://go.dev/cl/528038 mentions this issue: os/exec: avoid calling LookPath in cmd.Start for resolved paths

This reapplies CL 512155, which was previously reverted in CL 527337. The race that prompted the revert should be fixed by CL 527820, which will be submitted before this one. For #36768. Updates #62596. Change-Id: I3c3cd92470254072901b6ef91c0ac52c8071e0a2 Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-race,gotip-windows-amd64-race,gotip-windows-amd64-longtest Reviewed-on: https://go-review.googlesource.com/c/go/+/528038 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Bryan Mills <bcmills@google.com>

qiulaidongfeng · 2023-09-17T07:21:20Z

CL 528038 after ,Is there any reason not to close this issue?

bcmills · 2023-09-18T16:28:03Z

Agreed, this is now done.

gopherbot · 2024-03-29T16:04:28Z

Change https://go.dev/cl/575255 mentions this issue: os/exec: revert "avoid calling LookPath in cmd.Start for resolved paths"

ALTree added the OS-Windows label Jan 28, 2020

ALTree changed the title ~~Go compiles really slowly even for a hello world program~~ cmd/go: compiles really slowly even for a hello world program Jan 28, 2020

ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 28, 2020

ALTree added this to the Unplanned milestone Jan 28, 2020

bcmills added ToolSpeed GoCommand cmd/go and removed Performance labels Jul 20, 2023

qiulaidongfeng mentioned this issue Jul 22, 2023

os/exec: avoid calling LookPath in cmd.Start for resolved paths #61517

Closed

gopherbot closed this as completed in dfb2e42 Sep 11, 2023

dmitshur modified the milestones: Backlog, Go1.22 Sep 11, 2023

bcmills reopened this Sep 12, 2023

bcmills closed this as completed Sep 18, 2023

cmd/go: cache results of exec.LookPath #36768

cmd/go: cache results of exec.LookPath #36768

Comments

Zyxon123 commented Jan 25, 2020 • edited

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

go run -x output

ghost commented Jan 25, 2020 • edited by ghost

ianlancetaylor commented Jan 25, 2020

Zyxon123 commented Jan 25, 2020

ianlancetaylor commented Jan 26, 2020

Zyxon123 commented Jan 26, 2020

ianlancetaylor commented Jan 26, 2020

ghost commented Jan 26, 2020

Zyxon123 commented Jan 26, 2020

ghost commented Jan 26, 2020

ghost commented Jan 26, 2020

proyb6 commented Jan 27, 2020

Zyxon123 commented Jan 28, 2020

0intro commented Jan 28, 2020

ALTree commented Jan 28, 2020

Zyxon123 commented Jan 28, 2020

ALTree commented Jan 30, 2020 • edited

ghost commented Feb 2, 2020 • edited by ghost

ctd1500 commented Feb 3, 2020

proyb6 commented Feb 4, 2020

ghost commented Feb 4, 2020 • edited by ghost

ghost commented Feb 10, 2020

MaxSem commented Feb 10, 2020

andreykaipov commented Aug 11, 2020

davecheney commented Aug 11, 2020

andreykaipov commented Aug 11, 2020

davecheney commented Aug 11, 2020

qmuntal commented Jul 21, 2023 • edited

gopherbot commented Jul 22, 2023

qiulaidongfeng commented Aug 12, 2023

ianlancetaylor commented Sep 12, 2023

qiulaidongfeng commented Sep 12, 2023

mvdan commented Sep 12, 2023

gopherbot commented Sep 13, 2023

qiulaidongfeng commented Sep 17, 2023

bcmills commented Sep 18, 2023

gopherbot commented Mar 29, 2024

Zyxon123 commented Jan 25, 2020 •

edited

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

ghost commented Jan 25, 2020 •

edited by ghost

ALTree commented Jan 30, 2020 •

edited

ghost commented Feb 2, 2020 •

edited by ghost

ghost commented Feb 4, 2020 •

edited by ghost

qmuntal commented Jul 21, 2023 •

edited