Skip to content

x/build/builders: subrepos shouldn't cause full rebuild of Go #8646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bradfitz opened this issue Sep 4, 2014 · 10 comments
Closed

x/build/builders: subrepos shouldn't cause full rebuild of Go #8646

bradfitz opened this issue Sep 4, 2014 · 10 comments
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge
Milestone

Comments

@bradfitz
Copy link
Contributor

bradfitz commented Sep 4, 2014

Currently if I submit to go.tools or some subrepo, all builders need to rebuild the tip
Go commit just to rebuild the subrepo.

We should save the result of the build (tar it up to GCS or something) after hg tip is
build, to avoid that extra work.

This is related to sharding: we only need to "make.bash" in order to run the
subrepo builds & tests.
@dvyukov
Copy link
Member

dvyukov commented Sep 4, 2014

Comment 1:

If we parallelize bootstrap, then there won't be need in caching on good machines. go
install -a std takes 6.5s on my machine, make.bash must not take significantly more.
Probably 10 seconds or so.

@dvyukov
Copy link
Member

dvyukov commented Sep 4, 2014

Comment 2:

Also you can take a look at how I cache build for benchmarks (grep for lastWorkdir or
something like this).

@adg
Copy link
Contributor

adg commented Sep 4, 2014

Comment 3:

Do we really want to parallelize bootstrap, though?
Seems like adding a lot of complexity to a fundamental part of our system, in exchange
for a minor gain that can be achieved by other means.

@dvyukov
Copy link
Member

dvyukov commented Sep 4, 2014

Comment 4:

Sometimes I run bootstrap like 100 per day, so that has value beyond builders.
Also parallel bootstrap is local complexity in exchange to system complexity.

@bradfitz
Copy link
Contributor Author

bradfitz commented Sep 4, 2014

Comment 5:

Labels changed: added builder.

@bradfitz
Copy link
Contributor Author

bradfitz commented Jan 3, 2015

@adg, I can't figure out the interface to the dashboard here.

The openbsd-amd64-gce56 builder is only building Go commits and not packages (you can see the blank cells at the bottom of https://build.golang.org), but these URLs don't give me any info:

http://build.golang.org/todo?builder=openbsd-amd64-gce56&kind=build-go-commit
http://build.golang.org/todo?builder=openbsd-amd64-gce56&kind=build-package

What is the URL to ask the daskboard what subrepo packages need to be built?

@bradfitz
Copy link
Contributor Author

bradfitz commented Jan 4, 2015

And now those URLs are returning data! I'm even more confused. You didn't push a new version, did you?

@adg
Copy link
Contributor

adg commented Jan 4, 2015

I didn't change anything.

@bradfitz
Copy link
Contributor Author

bradfitz commented Jan 5, 2015

Per discussion with @adg we're going to not modify the /todo handler, so we don't break the old-style builders.

The coordinator instead will learn which subrepos it needs to build based on the data from http://build.golang.org/?mode=json (which is to be done in #9493)

@rsc rsc added this to the Unplanned milestone Apr 10, 2015
@rsc rsc changed the title builders: subrepos shouldn't cause full rebuild of Go x/build/builders: subrepos shouldn't cause full rebuild of Go Apr 14, 2015
@rsc rsc modified the milestones: Unreleased, Unplanned Apr 14, 2015
@rsc rsc added the Builders x/build issues (builders, bots, dashboards) label Jun 11, 2015
@bradfitz
Copy link
Contributor Author

This is now fixed with the recent builder work.

@golang golang locked and limited conversation to collaborators Jun 25, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge
Projects
None yet
Development

No branches or pull requests

5 participants