Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: single benchmark starts multiple time #23398

Closed
ghost opened this issue Jan 10, 2018 · 3 comments
Closed

testing: single benchmark starts multiple time #23398

ghost opened this issue Jan 10, 2018 · 3 comments

Comments

@ghost
Copy link

ghost commented Jan 10, 2018

What version of Go are you using (go version)?

go version go1.10beta1 darwin/amd64

Does this issue reproduce with the latest release?

yes with go1.9.2

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jeffrey/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/jeffrey/GoSpace:/Users/jeffrey/GolangProjects"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/r9/zplfsltx4mx8yf05hrx0dnwm0000gp/T/go-build516212567=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I was trying to make a benchmark with my mqtt client lib (libmqtt), using go test as

go test -v -bench . -benchmem -count=1

And here is the code

func BenchmarkLibmqttClient(b *testing.B) {
	b.N = 1
	b.ReportAllocs()
	var client lib.Client
	var err error

	b.Log("init client")
	if client, err = lib.NewClient(
		lib.WithServer("localhost:1883"),
		lib.WithKeepalive(3600, 1.2),
		lib.WithCleanSession(true)); err != nil {
		b.Error(err)
	}

	client.HandleUnSub(func(topic []string, err error) {
		if err != nil {
			b.Error(err)
		}
                // destroy client after  unsub success
		client.Destroy(true)
	})

	client.Connect(func(server string, code lib.ConnAckCode, err error) {
		if err != nil {
			b.Error(err)
		} else if code != lib.ConnAccepted {
			b.Error(code)
		}

		b.ResetTimer()
		for i := 0; i < b.N; i++ {
			client.Publish(&lib.PublishPacket{
				TopicName: testTopic,
				Qos:       testQos,
				Payload:   testTopicMsg,
			})
		}
		client.UnSubscribe(testTopic)
	})
	client.Wait()
}

Once added the client.Destroy(true) call (which just cancel the context of the client, so all workers can return and client.Wait() will return) will result in multiple restart

What did you expect to see?

benchmark starts and finishes both only one time

What did you see instead?

benchmark starts multiple times and finishes only one time

Here is the output for the benchmark

go test -v -bench . -benchmem -count=1
--- BENCH: BenchmarkLibmqttClient-4
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        ... [output truncated]
PASS
ok      github.com/goiiot/libmqtt/benchmark     0.173s
@davecheney
Copy link
Contributor

This is expected, the testing package runs your benchmark multiple times with increasing values of b.N.

http://talks.godoc.org/github.com/davecheney/high-performance-go-workshop/high-performance-go-workshop.slide#21

Closing as there is no bug to be found here.

@ghost
Copy link
Author

ghost commented Jan 10, 2018

@davecheney thanks for your nice slides.

@davecheney
Copy link
Contributor

davecheney commented Jan 10, 2018 via email

@mikioh mikioh changed the title single benchmark starts multiple time testing: single benchmark starts multiple time Jan 14, 2018
@golang golang locked and limited conversation to collaborators Jan 14, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants