Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Running testing.Benchmark produces wrong number of iterations #40012

Closed
aslatter opened this issue Jul 3, 2020 · 1 comment
Closed

Running testing.Benchmark produces wrong number of iterations #40012

aslatter opened this issue Jul 3, 2020 · 1 comment
Labels
FrozenDueToAge

Comments

@aslatter
Copy link

aslatter commented Jul 3, 2020

What version of Go are you using (go version)?

$ go version
go version go1.13.5 linux/amd64

Does this issue reproduce with the latest release?

This issue reproduces on the Go playground (as of 2020-07-02).

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/antoine/.cache/go-build"
GOENV="/home/antoine/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/antoine/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/antoine/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/antoine/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/antoine/projects/testing/go-bench/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build279526372=/tmp/go-build -gno-record-gcc-switches"

What did you do?

https://play.golang.org/p/CYHoKOYMgTj

package main

import (
	"log"
	"sync/atomic"
	"testing"
	"time"
)

var actualBenchIters uint32 = 0

func main() {
	result := testing.Benchmark(myBench)

	log.Printf("Result Duration: %v", result.T)
	log.Printf("Result Iterations: %v", result.N)
	log.Printf("Actual Iterations: %v (no more than)", actualBenchIters)
	log.Printf("Summary: %v", result)

}

func myBench(b *testing.B) {
	atomic.AddUint32(&actualBenchIters, 1)
	time.Sleep(time.Microsecond * 3)
}

What did you expect to see?

Printed 'actual iteterations' should not be less-than the iterations reported by the return-value of testing.Benchmark.

What did you see instead?

The number of iterations recorded by my counter was '6'. The number of iterations reported by testing.Benchmark was '1000000000'.

Playground output:

2009/11/10 23:00:00 Result Duration: 3µs
2009/11/10 23:00:00 Result Iterations: 1000000000
2009/11/10 23:00:00 Actual Iterations: 6 (no more than)
2009/11/10 23:00:00 Summary: 1000000000	         0.000003 ns/op

This makes the 'ns/op' calculation incorrect.
@davecheney
Copy link
Contributor

Your benchmark does not iterate over b.N. I will not report correct values as the benchmark will appear to run faster the larger the value of b.N supplied.

@golang golang locked and limited conversation to collaborators Jul 3, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@davecheney @aslatter @gopherbot