Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: record and collect compiler benchmarks #17167

Open
mdempsky opened this issue Sep 20, 2016 · 10 comments
Open

x/build: record and collect compiler benchmarks #17167

mdempsky opened this issue Sep 20, 2016 · 10 comments
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@mdempsky
Copy link
Member

Recently we added a -bench flag to cmd/compile to benchmark compiler phases. It might be interesting if the builders used this flag and then persisted the output file so we could monitor compiler performance over time.

There's a risk that measuring the standard library compile time is confounded with changes to the standard library, but I think only a couple packages (runtime, net, net/http) still seem under active development. So it's probably still a decent signal.

/cc @bradfitz @griesemer

@mdempsky mdempsky added the Builders x/build issues (builders, bots, dashboards) label Sep 20, 2016
@mdempsky mdempsky added this to the Unreleased milestone Sep 20, 2016
@mdempsky
Copy link
Member Author

/cc @davecheney too, since he's been benchmarking build times against other code bases too.

@bradfitz
Copy link
Contributor

I've been logging more and more to Datastore & BigQuery, and it's on my short-term list to add more timing info there, as part of optimizing build+test speeds for #17104

So, this isn't much more work.

/cc @quentinmit

@quentinmit
Copy link
Contributor

Yep, and I'm working on collecting benchmark results. This sounds great. Where does -bench put its output?

Also, which compile command would you instrument? There are a bunch involved in all.bash.

@mdempsky
Copy link
Member Author

You can use -bench=foo.txt and the compiler will append to foo.txt, so the entire build can be accumulated into a single file. The individual measurements indicate the package being compiled.

I would probably tweak all.bash so that the "go install std cmd" step adds -gcflags="-bench=/tmp/bench.txt" on the builders, and then save /tmp/bench.txt somewhere we can easily collect and analyze.

I think benchmarking the Go1 benchmarks' compile times would be nice too, since they're a fixed target.

@bradfitz
Copy link
Contributor

We don't use all.bash on the builders. And make.bash already respects the env var $GO_GCFLAGS

@mdempsky
Copy link
Member Author

I meant "all.bash" generically to refer to whatever build process was used by the builders. GO_GCFLAGS sounds good to me if that means we can do it non-intrusively.

@davecheney
Copy link
Contributor

SGTM!

On Wed, Sep 21, 2016 at 4:19 AM, Matthew Dempsky notifications@github.com
wrote:

/cc @davecheney https://github.com/davecheney too, since he's been
benchmarking build times against other code bases too.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#17167 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAAcA26_YEbWWITcnZNrkOVI6n0kZvA3ks5qsCO2gaJpZM4KB8V8
.

@gopherbot
Copy link

CL https://golang.org/cl/51192 mentions this issue.

gopherbot pushed a commit to golang/build that referenced this issue Jul 26, 2017
…o child

Updates golang/go#21153
Updates golang/go#17167

Change-Id: Ibe575d295468235a16c01f3420a9597373ab3891
Reviewed-on: https://go-review.googlesource.com/51192
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@dmitshur
Copy link
Contributor

dmitshur commented Jun 17, 2022

@mdempsky This issue might be a candidate for closing as a duplicate given we now have #48803 and #49207. What do you think? (CC @prattmic.)

@dmitshur dmitshur added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Jun 17, 2022
@prattmic
Copy link
Member

Yes, we have benchmark tracking now. I think the action item here would be to add some compiler phase benchmarks to x/benchmarks. Perhaps alongside our existing overall build benchmarks in https://cs.opensource.google/go/x/benchmarks/+/master:sweet/harnesses/go-build.go?

@dmitshur dmitshur removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jun 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

7 participants