Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/perf/cmd/benchstat: no output #58576

Closed
phenpessoa opened this issue Feb 17, 2023 · 7 comments
Closed

x/perf/cmd/benchstat: no output #58576

phenpessoa opened this issue Feb 17, 2023 · 7 comments
Labels
FrozenDueToAge
Milestone
Unreleased

Comments

@phenpessoa
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.20.1 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\pedro\AppData\Local\go-build
set GOENV=C:\Users\pedro\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\pedro\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\pedro\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.20.1
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\test\go.mod
set GOWORK=
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=C:\Users\pedro\AppData\Local\Temp\go-build281116068=/tmp/go-build -gno-record-gcc-switches

What did you do?

go install golang.org/x/perf/cmd/...@latest
benchstat old.txt new.txt

What did you expect to see?

The benchstat output

What did you see instead?

No output at all

If I run benchstat only with no arguments I get:

Usage: benchstat [flags] inputs...

benchstat computes statistical summaries and A/B comparisons of Go
benchmarks. It shows benchmark medians in a table with a row for each
benchmark and a column for each input file. If there is more than one
input file, it also shows A/B comparisons between the files. If a
difference is likely to be noise, it shows "~".

For details, see https://pkg.go.dev/golang.org/x/perf/cmd/benchstat.
  -alpha α
        consider change significant if p < α (default 0.05)
  -col projection
        split results into columns by distinct values of projection (default ".file")
  -confidence level
        confidence level for ranges (default 0.95)
  -filter query
        use only benchmarks matching benchfilter query (default "*")
  -format format
        print results in format:
          text - plain text
          csv  - comma-separated values (warnings will be written to stderr)
         (default "text")
  -ignore keys
        ignore variations in keys
  -row projection
        split results into rows by distinct values of projection (default ".fullname")
  -table projection
        split results into tables by distinct values of projection (default ".config")

Meaning benchstat is properly installed

image

I also tried:

benchstat old.txt new.txt > output.txt

Even though the output.txt file was created, it was empty.

Files used

old.txt Content
goos: windows
goarch: amd64
pkg: pptests/invites
cpu: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
BenchmarkInvitesInString-16             	  742873	      1669 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString-16             	  703620	      1736 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString-16             	  691200	      1710 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString-16             	  732019	      1716 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString-16             	  686360	      1718 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString-16             	  726889	      1723 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString-16             	  690158	      1722 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString-16             	  779018	      1708 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString-16             	  778911	      1721 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString-16             	  631538	      1731 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString_Parallel-16    	  493700	      2362 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString_Parallel-16    	  525888	      2312 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString_Parallel-16    	  514244	      2337 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString_Parallel-16    	  514758	      2319 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString_Parallel-16    	  514744	      2336 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString_Parallel-16    	  514729	      2331 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString_Parallel-16    	  525934	      2330 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString_Parallel-16    	  503484	      2348 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString_Parallel-16    	  503995	      2328 ns/op	     360 B/op	       6 allocs/op
BenchmarkInvitesInString_Parallel-16    	  345625	      3357 ns/op	     360 B/op	       6 allocs/op
PASS
ok  	pptests/invites	29.052s
new.txt Content
goos: windows
goarch: amd64
pkg: pptests/invites
cpu: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
BenchmarkInvitesInString-16             	 3589579	       328.7 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString-16             	 3412801	       337.5 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString-16             	 3455905	       337.1 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString-16             	 3458821	       347.1 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString-16             	 3470880	       344.7 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString-16             	 3436590	       348.3 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString-16             	 3444130	       346.9 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString-16             	 3448120	       345.1 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString-16             	 3458286	       347.2 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString-16             	 3359350	       350.2 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString_Parallel-16    	 2404923	       490.9 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString_Parallel-16    	 2478717	       488.5 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString_Parallel-16    	 2458682	       483.2 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString_Parallel-16    	 2473642	       486.4 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString_Parallel-16    	 2451098	       485.3 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString_Parallel-16    	 2453709	       492.0 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString_Parallel-16    	 2491608	       480.1 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString_Parallel-16    	 2451085	       491.1 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString_Parallel-16    	 2494173	       492.6 ns/op	     320 B/op	       1 allocs/op
BenchmarkInvitesInString_Parallel-16    	 2448741	       499.1 ns/op	     320 B/op	       1 allocs/op
PASS
ok  	pptests/invites	35.760s
@gopherbot gopherbot added this to the Unreleased milestone Feb 17, 2023
@seankhliao
Copy link
Member

can you provide zipped up versions of your input?

@phenpessoa
Copy link
Author

@seankhliao sure

inputs.zip

@seankhliao
Copy link
Member

That's a utf16le encoded text file, the tools should only work on utf8

@phenpessoa
Copy link
Author

I noticed that if I change this line (https://github.com/golang/perf/blob/801c7ef9e5c5eb8fb97c176023267056756fc52d/benchfmt/reader.go#L151) to:

line := bytes.ReplaceAll(r.s.Bytes(), null, nil)

Where null == []byte{0}

It works properly

@phenpessoa
Copy link
Author

That's a utf16le encoded text file, the tools should only work on utf8

I'm generating the files using go test -bench . -count 10 > old.txt and go test -bench . -count 10 > new.txt. So I'm not sure why it is generating a utf16le file?

@seankhliao
Copy link
Member

that would be a powershell issue https://stackoverflow.com/questions/40098771/changing-powershells-default-output-encoding-to-utf-8

@phenpessoa
Copy link
Author

Thanks, that answers it!

@phenpessoa phenpessoa mentioned this issue Feb 17, 2023
Open
@seankhliao seankhliao mentioned this issue Jul 9, 2023
Closed
@golang golang locked and limited conversation to collaborators Feb 17, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
3 participants
@phenpessoa @gopherbot @seankhliao