Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: cache tests with -race flag #61608

Closed
tarasmadan opened this issue Jul 27, 2023 · 3 comments
Closed

cmd/go: cache tests with -race flag #61608

tarasmadan opened this issue Jul 27, 2023 · 3 comments
Labels
GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.

Comments

@tarasmadan
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.20.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/syzkaller/.cache/go-build"
GOENV="/syzkaller/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/syzkaller/gopath/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/syzkaller/gopath:/gopath"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/syzkaller/gopath/src/github.com/google/syzkaller/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build64555202=/tmp/go-build -gno-record-gcc-switches"

What did you do?

go test -race ./...
go test -race ./...

What did you expect to see?

I expected test result to be cached.
Race detector is quite expensive and the results caching is a good thing to have.

What did you see instead?

$ go help test
.....
The rule for a match in the cache is that the run involves the same
test binary and the flags on the command line come entirely from a
restricted set of 'cacheable' test flags, defined as -benchtime, -cpu,
-list, -parallel, -run, -short, -timeout, -failfast, and -v.
If a run of go test has any test or non-test flags outside this set,
the result is not cached.
....
@tarasmadan
Copy link
Author

tarasmadan commented Jul 27, 2023 •
edited

People in #23565 are asking for the caching improvements since 2018.

tarasmadan added a commit to tarasmadan/syzkaller that referenced this issue Jul 27, 2023
@tarasmadan
ci.yml: cache build artefacts where it is needed
f922bf2 
If some job is not a build bottleneck - let's remove caching.
The main focus points then are "build" and "race" then.
To cache test results independently let's use 2 different cache prefixes.
See https://markphelps.me/posts/speed-up-your-go-builds-with-actions-cache/.

The idea is to always generate cash miss using the second precision key.
After the miss we download latest uploaded one using "restore-keys".

Potential improvements:
1. golang/go#23565 asks to cache -coverprofile results since 2018.
2. golang/go#61608 to cache -race results.
@seankhliao seankhliao changed the title affected/package: "go test -race" command cmd/go: cache tests with -race flag Jul 27, 2023
@seankhliao
Copy link
Member

please provide test code that reproduces the issue.
go test -race should use the cache when available, -race is a build flag, not a test flag.
see also #48484

@seankhliao seankhliao added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go labels Jul 27, 2023
github-merge-queue bot pushed a commit to google/syzkaller that referenced this issue Aug 23, 2023
@tarasmadan
ci.yml: cache build artefacts where it is needed
069a095 
If some job is not a build bottleneck - let's remove caching.
The main focus points then are "build" and "race" then.
To cache test results independently let's use 2 different cache prefixes.
See https://markphelps.me/posts/speed-up-your-go-builds-with-actions-cache/.

The idea is to always generate cash miss using the second precision key.
After the miss we download latest uploaded one using "restore-keys".

Potential improvements:
1. golang/go#23565 asks to cache -coverprofile results since 2018.
2. golang/go#61608 to cache -race results.
@gopherbot
Copy link

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

@gopherbot gopherbot closed this as not planned Won't fix, can't repro, duplicate, stale Aug 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@gopherbot @seankhliao @tarasmadan