Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: don't cache tests that use testing/quick.Check and friends #26276

Closed
ainar-g opened this issue Jul 8, 2018 · 2 comments
Closed

cmd/go: don't cache tests that use testing/quick.Check and friends #26276

ainar-g opened this issue Jul 8, 2018 · 2 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.11

Comments

@ainar-g
Copy link
Contributor

ainar-g commented Jul 8, 2018

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

go version devel +b001ffb Sun Jul 8 04:14:45 2018 +0000 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ainar/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/ainar/go"
GORACE=""
GOROOT="/home/ainar/go/go1.10"
GOTMPDIR=""
GOTOOLDIR="/home/ainar/go/go1.10/pkg/tool/linux_amd64"
GCCGO="/usr/bin/gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build505661023=/tmp/go-build -gno-record-gcc-switches"

What did you do?

https://play.golang.org/p/e6SHcVstHOW

$ go test foo_test.go
$ go test foo_test.go
$ go test foo_test.go

What did you expect to see?

ok  	command-line-arguments	0.002s
ok  	command-line-arguments	0.001s
ok  	command-line-arguments	0.003s

What did you see instead?

ok  	command-line-arguments	0.002s
ok  	command-line-arguments	(cached)
ok  	command-line-arguments	(cached)

My assumption here is that the people who use testing/quick want the tests to run as often as possible to maximise the potential of finding a bug. Caching the first successful result thus seems counterproductive. I know that the official way to disable caching for tests is -count 1, but it seems silly to provide a randomised testing package and than have it be almost useless by default.

If the consensus is that the people who use testing/quick should just also use -count 1, then I think it would be nice to reflect that in the package documentation.
@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 8, 2018
@ianlancetaylor ianlancetaylor added this to the Go1.11 milestone Jul 8, 2018
@josharian josharian mentioned this issue Jul 8, 2018
Closed
@rsc
Copy link
Contributor

rsc commented Aug 10, 2018

Duplicate of #23799, essentially, with the same likely answer: no.

@rsc rsc closed this as completed Aug 10, 2018
@rsc
Copy link
Contributor

rsc commented Aug 10, 2018

In particular see my response #23799 (comment).

@golang golang locked and limited conversation to collaborators Aug 10, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.11
Development

No branches or pull requests
4 participants
@rsc @ianlancetaylor @ainar-g @gopherbot