Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: slow compile/build compared to Go 1.4 #13646

Closed
kokizzu opened this issue Dec 17, 2015 · 16 comments
Closed

cmd/compile: slow compile/build compared to Go 1.4 #13646

kokizzu opened this issue Dec 17, 2015 · 16 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. ToolSpeed
Milestone

Comments

@kokizzu
Copy link

kokizzu commented Dec 17, 2015

It's been a while, we still use 1.4 because of this issue, it affect to much to our build-test-debug cycle.

$ uname -a
Linux f9988 4.1.14-1-lts #1 SMP Thu Dec 10 21:34:50 CET 2015 x86_64 GNU/Linux

# go 1.1 and go 1.2 failed to compile github.com/yosuke-furukawa/json5/encoding/json5

$ rm -rf $GOPATH/pkg/* 2>1 > /dev/null; go version ; time go build
go version go1.3.3 linux/amd64

CPU: 5.35s      Real: 3.34s     RAM: 287316KB
CPU: 5.63s      Real: 3.40s     RAM: 287316KB
CPU: 5.87s      Real: 3.41s     RAM: 287272KB

$ rm -rf $GOPATH/pkg/* 2>1 > /dev/null; go version ; time go build
go version go1.4.2 linux/amd64

CPU: 5.83s      Real: 3.34s     RAM: 290024KB
CPU: 5.71s      Real: 3.45s     RAM: 290024KB
CPU: 5.96s      Real: 3.46s     RAM: 289948KB

$ rm -rf $GOPATH/pkg/* 2>1 > /dev/null; go version ; time go build
go version go1.5.2 linux/amd64

CPU: 21.93s     Real: 7.92s     RAM: 310148KB
CPU: 21.07s     Real: 7.46s     RAM: 318660KB
CPU: 21.45s     Real: 7.58s     RAM: 310160KB

packages from: http://seblu.net/a/archive/packages/g/go/

for comparison, it's now as slow as gcc-go:

# for comparison:
$ rm -rf $GOPATH/pkg/* 2>1 > /dev/null; go version ; time go build
go version go1.4.2 gccgo (GCC) 5.3.0 linux/amd64

CPU: 20.76s     Real: 9.65s     RAM: 203212KB
CPU: 21.33s     Real: 9.83s     RAM: 203340KB
CPU: 21.09s     Real: 9.85s     RAM: 203336KB
@cespare
Copy link
Contributor

cespare commented Dec 17, 2015

What about after you've installed dependencies? (go build -i or go install)

@kokizzu
Copy link
Author

kokizzu commented Dec 17, 2015

it's significantly faster if not removing the pkg and doing go build -i at the first time..

1.5.2
CPU: 3.03s      Real: 1.99s     RAM: 243676KB
CPU: 3.04s      Real: 1.96s     RAM: 238416KB
CPU: 3.04s      Real: 1.96s     RAM: 238348KB

but still far slower than

1.4.2
CPU: 0.88s      Real: 0.99s     RAM: 207080KB
CPU: 0.93s      Real: 0.99s     RAM: 207000KB
CPU: 0.87s      Real: 1.00s     RAM: 207084KB

@ianlancetaylor ianlancetaylor changed the title Slow Compile/Build on 64-bit Linux for go 1.5 cmd/compile: slow compile/build on 64-bit Linux for go 1.5 Dec 17, 2015
@ianlancetaylor ianlancetaylor added this to the Go1.7 milestone Dec 17, 2015
@ALTree
Copy link
Member

ALTree commented Dec 17, 2015

Isn't a 2x slowdown between 1.4 and 1.5 expected? See https://golang.org/doc/go1.5#performance

Builds in Go 1.5 will be slower by a factor of about two

@kokizzu
Copy link
Author

kokizzu commented Jan 23, 2016

it got lot worse on windows 8,1 64-bit, with slower processor but SSD harddisk

1.4.3

$ time go build

real 0m7.670s
user 0m0.015s
sys 0m0.000s

real 0m6.886s
user 0m0.000s
sys 0m0.000s

real 0m7.280s
user 0m0.000s
sys 0m0.000s

1.5.3

real 0m15.825s
user 0m0.000s
sys 0m0.000s

real 0m17.916s
user 0m0.000s
sys 0m0.000s

real 0m16.444s
user 0m0.000s
sys 0m0.000s

@adg adg changed the title cmd/compile: slow compile/build on 64-bit Linux for go 1.5 cmd/compile: slow compile/build compared to Go 1.4 Feb 1, 2016
gopherbot pushed a commit that referenced this issue Feb 24, 2016
Reuseable cache of Prog entries.

Improves compiler speed by ~10%.

Update #13646

Change-Id: I01bd8606540d989ea8b8ba5131d1275ba380d976
Reviewed-on: https://go-review.googlesource.com/19868
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@kokizzu
Copy link
Author

kokizzu commented May 9, 2016

i got even worse result on Mac OSX 64-bit mid-2012 (i7-3xxx, 16GB, SSD)

$ rm -rf $GOPATH/pkg/* 2>1 > /dev/null; go version ; time go build
go version go1.4.3 darwin/amd64

real    0m43.886s
user    0m45.049s
sys 0m2.789s

@bradfitz bradfitz modified the milestones: Go1.8, Go1.7 May 9, 2016
@bradfitz
Copy link
Contributor

bradfitz commented May 9, 2016

Much has been done on this in Go 1.7. There will be more in 1.8.

@quentinmit quentinmit added the NeedsFix The path to resolution is known, but the work has not been done. label Oct 11, 2016
@rsc
Copy link
Contributor

rsc commented Oct 21, 2016

Much has been done on this in Go 1.8. There will be more in 1.9.

@rsc rsc modified the milestones: Go1.9, Go1.8 Oct 21, 2016
@McGiver-
Copy link

McGiver- commented Nov 6, 2016

I dont understand why my friend who is compiling the same code on his
i5 6200 cpu on his laptop compiles at around 1.4 seconds and my i7 6700k cpu
on my desktop compiles at 6 seconds. go 1.7.3

@kokizzu
Copy link
Author

kokizzu commented Nov 6, 2016

maybe ssd

On Nov 6, 2016 11:36 AM, "McGyver-" notifications@github.com wrote:

I dont understand why my friend who is compiling the same code on his
i5 6200 cpu on his laptop compiles at around 1.4 seconds and my i7 6700k
cpu
on my desktop compiles at 6 seconds. go 1.7.3


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#13646 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABAy6qBzsCBeGVbcIm7WooS2vPVsGUGMks5q7VlAgaJpZM4G3C23
.

@McGiver-
Copy link

McGiver- commented Nov 6, 2016

@kokizzu Nope my Desktop has no hard disk only ssd. 16gb ram.
Im wondering if there is something wrong with config but i dont think i have anything different
especially since im the one who set up go on my friends laptop im pretty sure
i performed the same steps.

@McGiver-
Copy link

McGiver- commented Nov 6, 2016

here is my go env
set GOARCH=amd64
set GOBIN=C:\Users\Something\Documents\CodeProjects\Go\bin
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\Something\Documents\CodeProjects\Go
set GORACE=
set GOROOT=C:\Go
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set CC=gcc
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0
set CXX=g++
set CGO_ENABLED=1

@bradfitz
Copy link
Contributor

bradfitz commented Nov 6, 2016

Let's move this conversation to a mailing list. This is just a tracking bug.

@kokizzu
Copy link
Author

kokizzu commented Dec 1, 2016

Performance check :3 rm -rf $GOPATH/pkg/* 2>1 > /dev/null; go version ; time go build

the exact same project as the first one, 1.5 or below not tested because the code no longer compatible.

go version go1.6.3 linux/amd64

CPU: 93.47s     Real: 36.97s    RAM: 470020KB
CPU: 91.22s     Real: 36.26s    RAM: 463844KB

go version go1.7.3 linux/amd64

CPU: 64.09s     Real: 31.05s    RAM: 353952KB
CPU: 64.95s     Real: 31.23s    RAM: 352440KB

go version go1.8beta1 linux/amd64

CPU: 56.83s     Real: 29.00s    RAM: 352572KB
CPU: 59.78s     Real: 30.26s    RAM: 354760KB

@josharian
Copy link
Contributor

The link http://seblu.net/a/archive/packages/g/go/ leads to a 404. Do you have an updated place to go find the code which is slow to compile? Thanks.

@kokizzu
Copy link
Author

kokizzu commented Mar 16, 2017

@josharian seblu.net was a place to look archlinux package archive (now i'm using agetpkg-git), not code that slow to compile. The code itself is owned by my previous company i've been working to, so i can't publish it here. it's about 122K LOC (4.6 MB of source code).

current benchmark

alias time='/usr/bin/time -f "\nCPU: %Us\tReal: %es\tRAM: %MKB"'
rm -rf $GOPATH/pkg/ ; go version ; time go build

go version go1.6.3 linux/amd64
CPU: 93.02s     Real: 37.26s    RAM: 488024KB

go version go1.7.5 linux/amd64
CPU: 66.35s     Real: 31.31s    RAM: 353620KB

go version go1.8 linux/amd64
CPU: 58.85s     Real: 29.63s    RAM: 353604KB

go 1.5 and 1.4 no longer able to compile (break with go-sqlcipher and mime/quotedprintable)

If you're looking for large go source code to compile, you can try juju or docker.

@josharian
Copy link
Contributor

It is well known that compilation times are slower from 1.5 on and that they are getting slowly better (as your current benchmarks show).

There are many different things that may make some code particularly slow to compile. Without actual code to work with, there's nothing to be done here. I'm going to close this. If/when there's code to look at, feel free to leave a comment and I'll reopen.

@golang golang locked and limited conversation to collaborators Mar 16, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. ToolSpeed
Projects
None yet
Development

No branches or pull requests

10 participants