Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/build: regression of performance for go-benchmark(build) #31533

Closed
jiebinn opened this issue Apr 18, 2019 · 15 comments
Closed

go/build: regression of performance for go-benchmark(build) #31533

jiebinn opened this issue Apr 18, 2019 · 15 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@jiebinn
Copy link

jiebinn commented Apr 18, 2019

The go-benchmark(build) of ubuntu18.10 has a same level of regression when I upgrade from go-1.11.2 to go-1.12.2.

What version of Go are you using (go version)?

$ go version
from go-1.11.2 to  go-1.12.2

Does this issue reproduce with the latest release?

I have the benchmark run on ubuntu18.10, not on Ubuntu 19.04 yet.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
    "Audio": "Realtek ALC891", 
    "BIOS Version": "1.5.11", 
    "Cache Size": "12288 KB", 
    "Chipset": "Intel Cannon Lake PCH Shared SRAM", 
    "Compiler": "GCC 8.3.1 20190411 + Clang 8.0.0 + LLVM 8.0.0", 
    "Core Count": "6", 
    "DISK": "Samsung SSD 970 EVO 250GB", 
    "Desktop": "GNOME Shell 3.32.0", 
    "Disk Scheduler": "MQ-DEADLINE", 
    "Display Driver": "modesetting 1.20.4", 
    "Display Server": "X Server 1.20.4", 
    "Extensions": "SSE 4.2 + AVX2 + AVX + RDRAND + FSGSBASE", 
    "File-System": "ext4", 
    "Frequency": "1200MHz", 
    "GRAPHICS": "Intel UHD 630 3GB", 
    "Kernel": "5.0.7-729.native (x86_64)", 
    "MEMORY": "8192MB", 
    "MOTHERBOARD": "Dell 0HVPDY", 
    "Microcode": "0x96", 
    "Monitor": "U32J59x", 
    "Mount Options": "relatime rw stripe=256", 
    "Network": "Realtek RTL8111/8168/8411 + Qualcomm Atheros QCA9565 / AR9565", 
    "OpenCL": "OpenCL 2.1", 
    "OpenGL": "4.5 Mesa 19.1.0-devel", 
    "PROCESSOR": "Intel Core i7-8700 @ 4.60GHz", 
    "Scaling Driver": "intel_pstate performance", 
    "Screen": "3840x2160", 
    "Security": "KPTI + __user pointer sanitization + Full generic retpoline IBPB", 
    "Thread Count": "12", 
    "Vulkan": "1.1.102"

What did you do?

I have the run the benchmark of go-benchmark(build) on i7 machine. There is a regression when the package of go is upgraded from 11.2 to 12.2.

What did you expect to see?

No regression of performance when I upgraded from go-11.2 to go-12.2.

What did you see instead?

The performance of go-benchmark(build) for go-12.2 is 10% worse than go-11.2.

@davecheney
Copy link
Contributor

@Jiebin thank you for your report. Before we can investigate you need to compete your error report by providing us with detailed information about what you did, what you saw, and what you expected to see. These steps need to be reproducible by someone else for us to investigate.

Thank you

@davecheney davecheney added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Apr 19, 2019
@jiebinn
Copy link
Author

jiebinn commented Apr 19, 2019

@davecheney Thanks. And I have provided the info above. Please tell me if you need other info.

@randall77
Copy link
Contributor

What is go-benchmark(build)? How would I get it and run it?

@jiebinn
Copy link
Author

jiebinn commented Apr 23, 2019

What is go-benchmark(build)? How would I get it and run it?

Go-benchmark is a benchmark. "Build" is an indicator for that benchmark. You can get it from the official website or run it from Phoronix.com.

@davecheney
Copy link
Contributor

@jiebinn please understand that we are not in a position to reproduce the phoronix benchmark setup. You're going to have to meet us half way by writing down in this PR the steps required to reproduce the issue.

@jiebinn
Copy link
Author

jiebinn commented Apr 23, 2019

@randall77 Here is the steps. You can replace the go-package and then run it on Ubuntu or Clear Linux.

  1. cd ~
    wget https://phoronix-test-suite.com/releases/phoronix-test-suite-8.6.1.tar.gz
  2. tar zxf ~/phoronix-test-suite-8.4.1.tar.gz
  3. cd ~/phoronix-test-suite
    sudo ~/phoronix-test-suite/install-sh
  4. phoronix-test-suite
    y
    n
  5. phoronix-test-suite run go-benchmark
    1
    n
  6. It will show the result of go-benchmark-"build".

@davecheney
Copy link
Contributor

Is it possible to extract the code that the phoronix benchmark is running? it will be building some source code at some revision. If you can identify which programs it builds, then we can reduce the scope of the problem to time go build

@jiebinn
Copy link
Author

jiebinn commented Apr 23, 2019

@davecheney Thanks. I will looking into how to run go-benchmark from the source code.

@davecheney
Copy link
Contributor

@jiebinn thank you, that will be the best way to resolve this. Once we've confirmed if there is a regression in the build benchmarks between Go 1.11 and Go 1.12 without phoronix-test-suite, then that can be addressed first. If there is no regression then the issue may be with how phoronix-test-suite invokes go build and that will probably have to be escalated to someone on the phoronix-test-suite side.

@jiebinn
Copy link
Author

jiebinn commented Apr 23, 2019

@davecheney I agree. I have found the regression on Ubuntu(11%) and Clear Linux(10%) by running it with phoronix-test-suite. Need to make sure if that's a problem of phoronix-test-suite.

@jiebinn
Copy link
Author

jiebinn commented Apr 25, 2019

@davecheney Here is the steps for go-benchmark(build).

  1. wget http://www.phoronix-test-suite.com/benchmark-files/golang-benchmarks-121017.tar.gz
  2. tar zxvf golang-benchmarks-121017.tar.gz
    go-benchmark-04122017/x/benchmarks/build
  3. go build "build.go" and run it.

@randall77
Copy link
Contributor

I don't see any regression from 1.11.2 to 1.12.2. In fact, 1.12.2 is a tad faster.

name   old time/op                 new time/op                 delta
Build                  31.9s ± 2%                  31.6s ± 1%  -1.16%  (p=0.023 n=10+10)

This is on linux/amd64.

In any case, lots of things have changed about the build system from 1.11 to 1.12, e.g. more aggressive inlining. I wouldn't expect build times to be stable. We really care about regressions in generated code speed, but it probably isn't worth chasing down a few percent in build speed.

@randall77
Copy link
Contributor

Sorry, I lied. Just building build.go with different Go versions is not enough. You also have to change the version of the Go tool that the benchmark shells out to.
Now I get (from 1.11.2 to 1.12.2):

name   old time/op                 new time/op                 delta
Build                  21.5s ± 1%                  24.9s ± 1%  +16.19%  (p=0.008 n=5+5)
name   old binary-size             new binary-size             delta
Build                  13.0M ± 0%                  14.6M ± 0%  +12.15%  (p=0.008 n=5+5)

The larger binary size makes me think inlining is to blame for the extra time.

@andybons andybons changed the title regression of performance for go-benchmark(build) go/build: regression of performance for go-benchmark(build) May 15, 2019
@andybons
Copy link
Member

@randall77 are we still waiting for info? Add back the label if need be. Seems like this needs investigation.

@andybons andybons added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels May 15, 2019
@andybons andybons added this to the Go1.13 milestone May 15, 2019
@randall77
Copy link
Contributor

I'm going to close this issue. It falls under the general heading of making compiling faster (see all the ToolSpeed tagged issues) and/or making binaries smaller (#6853).
It may be that inlining is too aggressive here also, that falls under #17566.

@golang golang locked and limited conversation to collaborators May 14, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants