Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/tools/benchmark: support parsing custom metrics #34475

Open
jsternberg opened this issue Sep 23, 2019 · 2 comments
Open

x/tools/benchmark: support parsing custom metrics #34475

jsternberg opened this issue Sep 23, 2019 · 2 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone
Unreleased

Comments

@jsternberg
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.13 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN="/Users/jsternberg/go/pkg/bin/github.com/influxdata/flux"
GOCACHE="/Users/jsternberg/Library/Caches/go-build"
GOENV="/Users/jsternberg/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/jsternberg/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/jsternberg/go/src/github.com/influxdata/flux/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/w5/25fg3zv56y7cgd20q4fx_24m0000gn/T/go-build317574403=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I used ReportMetric when running a benchmark and attempted to use benchcmp to compare the benchmarks.

What did you expect to see?

I expected benchcmp to add an additional section showing the change in numbers for the custom benchmarks.

I then took a look at golang.org/x/tools/benchmark/parse to see if that functionality had been added or if the library had been updated so I could write the tool myself, but I see that the code there has not been updated to either read or expose the new custom metrics.

It would be very helpful to update this tool to support these custom metrics. We are beginning to use them to track other metrics and we would like to see how these other metrics change between two changesets.

What did you see instead?

Nothing.
@gopherbot gopherbot added this to the Unreleased milestone Sep 23, 2019
@gopherbot gopherbot added the Tools This label describes issues relating to any tools in the x/tools repository. label Sep 23, 2019
@bcmills
Copy link
Contributor

bcmills commented Sep 23, 2019

CC @ianthehat

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 23, 2019
@jsternberg
Copy link
Author

If there's interest, I wrote some code to be able to accommodate my own personal need and it's in a fork at https://github.com/jsternberg/tools. I can clean it up and make it into a changeset if there is interest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
3 participants
@jsternberg @bcmills @gopherbot