Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

testing: benchmark performance misaligned for fast tests #30928

Open
kokes opened this issue Mar 19, 2019 · 2 comments
Open

testing: benchmark performance misaligned for fast tests #30928

kokes opened this issue Mar 19, 2019 · 2 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Testing An issue that has been verified to require only test changes, not just a test failure.
Milestone
Unplanned

Comments

@kokes
Copy link

kokes commented Mar 19, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ondrej.kokes/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/ondrej.kokes/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12.1/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/mk/dqryy0_947936yjsdw8jl6v00000gp/T/go-build420651873=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I ran some fast tests, where each iteration took less than a nanosecond. This led to the results table to be misaligned.

Here's a reproducible example:

package foo

import (
	"testing"
	"time"
)

func BenchmarkFoo(b *testing.B) {
	for j := 0; j < b.N; j++ {
		time.Sleep(time.Millisecond)
	}
}

func BenchmarkBar(b *testing.B) {
	for j := 0; j < b.N; j++ {

	}
}

What did you expect to see?

goos: darwin
goarch: amd64
BenchmarkFoo-4   	    1000	   1357941 ns/op
BenchmarkBar-4        2000000000	      0.32 ns/op

What did you see instead?

goos: darwin
goarch: amd64
BenchmarkFoo-4   	    1000	   1357941 ns/op
BenchmarkBar-4   	2000000000	         0.32 ns/op

Thoughts

I guess this has to do with performance numbers being printed as %8d, so anything larger than that will get shifted.
@katiehockman katiehockman added the Testing An issue that has been verified to require only test changes, not just a test failure. label Mar 22, 2019
@katiehockman katiehockman added this to the Unplanned milestone Mar 22, 2019
@katiehockman
Copy link
Contributor

/cc @mpvl
This seems like a small bug, I'll leave it up to you on whether or not this should/can be fixed easily.

@cespare
Copy link
Contributor

cespare commented Mar 22, 2019 •
edited

Looks like the ns/op are aligned in the example (by decimal point).

The iteration counts don't look great, though.

@cespare cespare mentioned this issue Mar 22, 2019
Closed
@julieqiu julieqiu added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label May 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Testing An issue that has been verified to require only test changes, not just a test failure.
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
4 participants
@cespare @julieqiu @kokes @katiehockman