Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/test: caching test results takes 50 seconds #35801

Closed
rillig opened this issue Nov 23, 2019 · 5 comments
Closed

cmd/test: caching test results takes 50 seconds #35801

rillig opened this issue Nov 23, 2019 · 5 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Backlog

Comments

@rillig
Copy link
Contributor

rillig commented Nov 23, 2019

What version of Go are you using (go version)?

go version go1.13.1 windows/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\rillig\AppData\Local\go-build
set GOENV=C:\Users\rillig\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\rillig\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\rillig\Program Files\cygwin\tmp\go-build091584506=/tmp/go-build -gno-record-gcc-switches
go: failed to remove work dir: GetFileInformationByHandle C:\Users\rillig\Program Files\cygwin\tmp\go-build091584506\NUL: Unzulässige Funktion.

The last line of the above output looks interesting in itself.

What did you do?

cd "$GOPATH/src"
go get github.com/rillig/pkglint
mv github.com/rillig/pkglint to netbsd.org/pkglint
cd netbsd.org/pkglint

go generate ./... && time go test ./...
# takes 20 seconds to run the test
# then, takes 50 seconds to cache the test results
go generate ./... && time go test -test.count 1 ./...
# takes 20 seconds

What did you expect to see?

Caching the test results needs less than a second.

What did you see instead?

Caching the test results takes much longer than running the test.
It doesn't matter whether I run the tests from within cmd.exe or Cygwin.
rillig added a commit to rillig/pkglint that referenced this issue Nov 23, 2019
@rillig
In qa-pkglint, disable cache for "go test"
233e377 
Before: 70 seconds, after: 20 seconds.

See golang/go#35801
@toothrot
Copy link
Contributor

That go env failure sounds similar to #30789, which @bcmills, @bradfitz and @alexbrainman have some experience with. I'm wondering if it's related to the comment added in golang.org/cl/172337.

@toothrot toothrot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 26, 2019
@toothrot toothrot added this to the Backlog milestone Nov 26, 2019
@bcmills
Copy link
Contributor

bcmills commented Nov 26, 2019

Both of those seem like symptoms of an unusually slow filesystem.

@rillig, what kind of disk is your GOCACHE hosted on, and do you have any utilities (such as antivirus scanners) running for files on that disk?

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Nov 26, 2019
@rillig
Copy link
Contributor Author

rillig commented Nov 26, 2019

@bcmills The GOCACHE is on an SSD, therefore I expect it to be pretty fast. The antivirus scanner sounds plausible, therefore I tried to reproduce it today, but it was reasonably fast. I'll try again tomorroy, to get another measuring.

@alexbrainman
Copy link
Member

@rillig I don't have Cygwin. I tried guessing your repro, but failed

c:\Users\Alex\dev\go\src>cd %GOPATH%

c:\Users\Alex\dev>cd src

c:\Users\Alex\dev\src>go get github.com/rillig/pkglint
package netbsd.org/pkglint/getopt: unrecognized import path "netbsd.org/pkglint/getopt": reading https://netbsd.org/pkglint/getopt?go-get=1: 404 Not Found
package netbsd.org/pkglint/histogram: unrecognized import path "netbsd.org/pkglint/histogram": reading https://netbsd.org/pkglint/histogram?go-get=1: 404 Not Found
package netbsd.org/pkglint/licenses: unrecognized import path "netbsd.org/pkglint/licenses": reading https://netbsd.org/pkglint/licenses?go-get=1: 404 Not Found
package netbsd.org/pkglint/pkgver: unrecognized import path "netbsd.org/pkglint/pkgver": reading https://netbsd.org/pkglint/pkgver?go-get=1: 404 Not Found
package netbsd.org/pkglint/regex: unrecognized import path "netbsd.org/pkglint/regex": reading https://netbsd.org/pkglint/regex?go-get=1: 404 Not Found
package netbsd.org/pkglint/textproc: unrecognized import path "netbsd.org/pkglint/textproc": reading https://netbsd.org/pkglint/textproc?go-get=1: 404 Not Found
package netbsd.org/pkglint/trace: unrecognized import path "netbsd.org/pkglint/trace": reading https://netbsd.org/pkglint/trace?go-get=1: 404 Not Found

c:\Users\Alex\dev\src>

Alex

@mvdan
Copy link
Member

mvdan commented Jun 15, 2021

Closing old issues that still have the WaitingForInfo label where enough details to investigate weren't provided. Feel free to leave a comment with more details and we can reopen.

@mvdan mvdan closed this as completed Jun 15, 2021
@golang golang locked and limited conversation to collaborators Jun 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
6 participants
@toothrot @rillig @mvdan @bcmills @gopherbot @alexbrainman