Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

testing: Creating sub-benchmark with b.Run means that parent will not show anything from b.ReportMetric #66353

Open
HaraldNordgren opened this issue Mar 16, 2024 · 2 comments
Open

testing: Creating sub-benchmark with b.Run means that parent will not show anything from b.ReportMetric #66353

HaraldNordgren opened this issue Mar 16, 2024 · 2 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@HaraldNordgren
Copy link
Member

HaraldNordgren commented Mar 16, 2024 •
edited

Go version

go version go1.21.6 darwin/arm64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/Harald/Library/Caches/go-build'
GOENV='/Users/Harald/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/Harald/go/pkg/mod'
GONOPROXY='github.com/dietdoctor/*'
GONOSUMDB='github.com/dietdoctor/*'
GOOS='darwin'
GOPATH='/Users/Harald/go'
GOPRIVATE='github.com/dietdoctor/*'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/homebrew/Cellar/go/1.21.6/libexec'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/homebrew/Cellar/go/1.21.6/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.21.6'
GCCGO='gccgo'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/Users/Harald/dd/hive/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/j2/0w9hqz1x01g3j4_yz0w0sw780000gp/T/go-build4150210059=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

func BenchmarkTest(b *testing.B) {
	for _, name := range []string{"Test"} {
		b.Run(name, func(b *testing.B) {
			for _, subName := range []string{"Subtest"} {
				b.Run(subName, func(bb *testing.B) {
					bb.ReportMetric(100, "sub_test_metric")
				})
			}
			b.ReportMetric(200, "some_metric")
		})
	}
}

What did you see happen?

BenchmarkTest
BenchmarkTest/Test
BenchmarkTest/Test/Subtest
BenchmarkTest/Test/Subtest-8 	1000000000	         0.0000002 ns/op	       100.0 sub_test_metric
PASS

What did you expect to see?

I want to see some_metric be reported on the BenchmarkTest/Test benchmark.
@dr2chase
Copy link
Contributor

CC @neild @adonovan

@dr2chase dr2chase added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 18, 2024
@adonovan
Copy link
Member

adonovan commented Mar 18, 2024 •
edited

@aclements Was this effect an unexpected consequence of ResetTimer's clearing of the extra mapping, or was it intended, perhaps on the groups that b.Run has all kinds of performance effects that render the outer benchmark invalid as anything other than a container for subbenchmarks?

Should the extra mapping be inherited by subbenchmarks?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dr2chase @adonovan @HaraldNordgren