Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

builders: run builds in VMs in addition to containers #9492

Closed
bradfitz opened this issue Jan 2, 2015 · 5 comments
Closed

builders: run builds in VMs in addition to containers #9492

bradfitz opened this issue Jan 2, 2015 · 5 comments
Labels
FrozenDueToAge
Milestone
Unplanned

Comments

@bradfitz
Copy link
Contributor

bradfitz commented Jan 2, 2015

This bug is about the non-Linux (but still x86) part of http://golang.org/s/builderplan

The current build coordinator (x/tools/dashboard/coordinator) only runs builds in Docker containers. These are what https://github.com/golang/go/wiki/DashboardBuilders calls the "New-style" builders.

They're only Linux configs, though.

To run non-Linux OSes, we want to run VMs as well. This bug is about adding support to run builds in VMs on GCE (so x86 only)

The first part was recently done in https://go-review.googlesource.com/#/c/2180/ which added the buildlet. The buildlet runs on each VM (Windows, Plan 9, Open BSD, etc) and provides remote command execution and file extraction over HTTP. This is the interface that the coordinator will then use instead of docker for VM-based builds.

Specific bugs have been filed for each operating system moving to this new scheme:

FreeBSD is #8639
OpenBSD is #8642 (furthest along)
Windows is #8640
Plan 9 is #9491
@minux
Copy link
Member

minux commented Jan 3, 2015

What about NetBSD?

For ARM builders, I suggest we still use real hardware, because qemu system
can't
do perfect emulation yet. And if we care that much about speed, we can have
multiple
arm builder, each running parts of the tests. And we can even cross compile
the
toolchain for ARM on a beefy x86 host.
(The go code are building pretty fast even on ARM, but the C part are not.)

@bradfitz
Copy link
Contributor Author

bradfitz commented Jan 3, 2015

Yes, NetBSD, Solaris, etc. I just didn't list them all.

For ARM, let's move discussion to #8647

@bradfitz
Copy link
Contributor Author

bradfitz commented Jan 3, 2015

It's starting to work ...

Coordinator showing it's watching a buildlet on a VM it started:
coord

And the VM for just that one build:
alive

@bradfitz bradfitz mentioned this issue Jan 3, 2015
Closed
@bradfitz
Copy link
Contributor Author

bradfitz commented Jan 3, 2015

The openbsd-amd64-gce56 builder is now running.

It doesn't do subrepos yet, though. I will do that as part of fixing #8646 (builders: subrepos shouldn't cause full rebuild of Go) with @adg

@bradfitz bradfitz mentioned this issue Jan 3, 2015
Open
bradfitz added a commit to golang/tools that referenced this issue Jan 5, 2015
@bradfitz
dashboard/coordinator: use VMs to run non-Linux builders
58c8b8a 
Currently this is running an OpenBSD amd64 builder, but other operating
systems are now easy.

Updates golang/go#8642
Updates golang/go#9492

Change-Id: I4e1026c49ee4d253dfcf78c7606767294784f02e
Reviewed-on: https://go-review.googlesource.com/2264
Reviewed-by: Andrew Gerrand <adg@golang.org>
bradfitz added a commit to golang/tools that referenced this issue Jan 5, 2015
@bradfitz
dashboard/coordinator: use new dashboard JSON interface to find work
ac848a9 
This uses the new JSON interface to the build dashboard (from
golang.org/cl/2290) to find all work, and re-enables the OpenBSD
builder[*], and can do multiple things at a time. Andrew and I just
watched it fire up 8 OpenBSD VMs at once to catch up.

[*] The OpenBSD builder was disabled because it would only report
results for the main repo, not subrepos, and the old
build.golang.org/todo interface didn't understand that was
possible. Now the steps are considered separate.

Update golang/go#8642 (OpenBSD)
Update golang/go#9492 (builds in VMs)

Change-Id: Ic6c2f73ee3da218dd54ef1a33f3afc97046ea3cc
Reviewed-on: https://go-review.googlesource.com/2282
Reviewed-by: Andrew Gerrand <adg@golang.org>
@bradfitz
Copy link
Contributor Author

bradfitz commented Jan 9, 2015

NetBSD CL at https://go-review.googlesource.com/2612

bradfitz added a commit to golang/build that referenced this issue Jan 21, 2015
@bradfitz
dashboard/coordinator: use VMs to run non-Linux builders
8831f36 
Currently this is running an OpenBSD amd64 builder, but other operating
systems are now easy.

Updates golang/go#8642
Updates golang/go#9492

Change-Id: I4e1026c49ee4d253dfcf78c7606767294784f02e
Reviewed-on: https://go-review.googlesource.com/2264
Reviewed-by: Andrew Gerrand <adg@golang.org>
bradfitz added a commit to golang/build that referenced this issue Jan 21, 2015
@bradfitz
dashboard/coordinator: use new dashboard JSON interface to find work
f72e62c 
This uses the new JSON interface to the build dashboard (from
golang.org/cl/2290) to find all work, and re-enables the OpenBSD
builder[*], and can do multiple things at a time. Andrew and I just
watched it fire up 8 OpenBSD VMs at once to catch up.

[*] The OpenBSD builder was disabled because it would only report
results for the main repo, not subrepos, and the old
build.golang.org/todo interface didn't understand that was
possible. Now the steps are considered separate.

Update golang/go#8642 (OpenBSD)
Update golang/go#9492 (builds in VMs)

Change-Id: Ic6c2f73ee3da218dd54ef1a33f3afc97046ea3cc
Reviewed-on: https://go-review.googlesource.com/2282
Reviewed-by: Andrew Gerrand <adg@golang.org>
@rsc rsc added this to the Unplanned milestone Apr 10, 2015
@golang golang locked and limited conversation to collaborators Sep 22, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
4 participants
@bradfitz @rsc @minux @gopherbot