Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

proposal: testing: panic if benchmark function tampers with b.N? #14479

Closed
minux opened this issue Feb 23, 2016 · 3 comments
Closed

proposal: testing: panic if benchmark function tampers with b.N? #14479

minux opened this issue Feb 23, 2016 · 3 comments
Labels
FrozenDueToAge Proposal
Milestone
Unplanned

Comments

@minux
Copy link
Member

minux commented Feb 23, 2016

If the benchmark function changes b.N, the testing package will get very confused.

As an example, this (contrived) function:

func Benchmark(b *testing.B) {
    b.N = 1000000000
    time.Sleep(time.Second)
}

will lead to this:

Benchmark-4    1000000000               1.00 ns/op

I've seen this in the wild, for example:
https://groups.google.com/forum/#!topic/golang-nuts/DjKNjpSHi3M
@minux minux added the Proposal label Feb 23, 2016
@minux minux changed the title proposal: testing: detect if benchmark function tampers with b.N? proposal: testing: panic if benchmark function tampers with b.N? Feb 23, 2016
@cznic
Copy link
Contributor

cznic commented Feb 23, 2016

There are probably other places in the stdlib, as well as in 3rd party packages, where mutating a struct field/package level variable, intended only for reading, will create mess. That's why I think some //go:readonly or similar annotation considered by go vet might be of more general use.

@dsymonds
Copy link
Contributor

On the surface, a vet check is plausible. On the other hand, if you're new enough to Go to consider setting b.N then you are likely to not even know about vet. I think the best vet checks are those that will catch things that even experienced Go programmers can make.

@bradfitz bradfitz modified the milestone: Unplanned Apr 7, 2016
@adg
Copy link
Contributor

adg commented Aug 15, 2016

These seems of extremely marginal utility, and once we go down this road we will spend more time being defensive than being productive.

@adg adg closed this as completed Aug 15, 2016
@golang golang locked and limited conversation to collaborators Aug 15, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge Proposal
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
6 participants
@bradfitz @dsymonds @minux @cznic @adg @gopherbot