Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: make rebuild logic content-based instead of time-based #21477

Closed
rsc opened this issue Aug 16, 2017 · 3 comments
Closed

cmd/go: make rebuild logic content-based instead of time-based #21477

rsc opened this issue Aug 16, 2017 · 3 comments
Milestone

Comments

@rsc
Copy link
Contributor

rsc commented Aug 16, 2017

This is a tracking issue for work toward making cmd/go rebuild logic use a content hash of all inputs to the build instead of modification times.

@rsc rsc added this to the Go1.10 milestone Aug 16, 2017
@rsc rsc self-assigned this Aug 16, 2017
@josharian
Copy link
Contributor

Possibly related: #15752

@gopherbot
Copy link

Change https://golang.org/cl/73212 mentions this issue: cmd/go: switch to entirely content-based staleness determination

@ALTree
Copy link
Member

ALTree commented Oct 31, 2017

I haven't investigated this yet, but just FYI after CL 73212 (cmd/go: switch to entirely content-based staleness determination) I see a 10x slowdown in the compiler-fuzzing driver I use to fuzz gc.

The driver is a go program that generates random go programs using gosmith and then uses exec.Cmd to invoke go build on them. This is the typical output when using 8 workers on a 8-cores machine:

$ go run driver.go -p 8 -checkers amd64
2017/10/31 15:05:51 testing with 8 workers
2017/10/31 15:05:51 0 tests, 0 known, 0 build, 0 ssadump, 0 gofmt, 0 exec
2017/10/31 15:05:54 39 tests, 0 known, 0 build, 0 ssadump, 0 gofmt, 0 exec
2017/10/31 15:05:57 82 tests, 0 known, 0 build, 0 ssadump, 0 gofmt, 0 exec

Notice how in the 6 seconds before I stopped it, 82 multi-package go programs were generated and compiled successfully. After git revision 7dea509 (the CL 73212 I mentioned above), the same command gives this output

$ go run driver.go -p 8 -checkers amd64
2017/10/31 15:07:38 testing with 8 workers
2017/10/31 15:07:38 0 tests, 0 known, 0 build, 0 ssadump, 0 gofmt, 0 exec
2017/10/31 15:07:41 0 tests, 0 known, 0 build, 0 ssadump, 0 gofmt, 0 exec
2017/10/31 15:07:44 8 tests, 0 known, 0 build, 0 ssadump, 0 gofmt, 0 exec

The same 6 seconds of fuzzing, and a ~10x reduction in the number of compiled programs.

@golang golang locked and limited conversation to collaborators Oct 31, 2018
@rsc rsc removed their assignment Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants