You can also use the VIX bindings I wrote just in case vmrun falls short for any reason: https://github.com/hooklift/govix

@c4milo, any chance you want to own this whole bug? :)

hahaha, @bradfitz let me look into it in detail before making any commitments.

@bradfitz alright, how were you envisioning the little API server you mentioned above?

I'd need API calls to:

• start a pre-configured OS X VM, and return me an ID for it (so I can delete it later), and IP:port I can use to hit its port 80.
• an API call to destroy that VM

That's it. The VM would need to run a couple line shell script on boot.

Working on it.

That's it. The VM would need to run a couple line shell script on boot.

As root?

@c4milo, yeah, root works. I can drop down to a less-privileged user if I need to. But some of our net tests require root, or at least more capabilities.

The script you'll want to run as root is just:

#!/bin/bash
set -e
curl --fail -o /usr/local/bin/buildlet https://storage.googleapis.com/go-builder-data/buildlet.darwin-amd64
chmod +x /usr/local/bin/buildlet
exec /usr/local/bin/buildlet

That should work already. Be careful with that server: it is a remote code execution daemon. Don't run it on your real machine if it's exposed to the internet.

But you should be able to hit it in a VM and see it reply "I am a buildlet" or whatever at its root path.

(and don't bake in the buildlet to the VM image... only that script. That lets us update the buildlet over time easier than updating the whole VM image)

Got it.

Also, tell me what to buy. :)

VMWare Fusion 7 Pro? http://store.vmware.com/store/vmware/en_US/DisplayProductDetailsPage/ThemeID.2485600/productID.304322800

I have a iMac 5K running Yosemite and a Macbook Pro running Mavericks.

Ideally we'll want to test the Go builders with OS X 10.8, 10.9, and 10.10. Maybe 10.7 too. So can I just buy those OS X discs from Apple somehow if I can no longer find my original CDs? Or do I need "Server CDs"? I know you're not a lawyer, and I'm not asking for legal advice. We're definitely not looking to avoid paying for things. I'm happy giving Apple plenty of (more) money here... I just want to know technically what's required, once we pay for all the parts the Apple-sanctioned ways.

I'm using VMWare Fusion 7 Pro. I was also thinking in using OSX Server as the guest OS, since it seems to boot faster, but I might need to double check that once I start testing the API.

On 6 January 2015 at 05:19, Brad Fitzpatrick notifications@github.com
wrote:

@c4milo https://github.com/c4milo, yeah, root works. I can drop down to
a less-privileged user if I need to. But some of our net tests require
root, or at least more capabilities.

Really? None of the builders I have configured have executed the tests as
root, and I don't think we should be doing that.

They skip the tests when they're not root.

I don't think it's a big deal, and let's not get sidetracked on this bug.

I think at least for 10.6, you are not allowed to run regular OS X inside
VMWare Fusion.
You must use OS X Server.

It's probably also true for 10.7.

PS: we do need a builder for 10.6, because 10.6 is the earliest version
that we still
support.

We can do an old-style builder for 10.6, for as long as we still support it, which probably won't be much longer. It's possible we even drop 10.6 support in Go 1.5.

Removing OS X 10.6 support in 1.5 SGTM.
It's causing too much trouble in support, and its support is crippled by the
fact that external linking is not working anyway.

Moving from email thread, I confirmed I can boot into Yosemite into a desktop running Terminal and the buildlet in under 1 minute. It's about 57 seconds on my iMac 5K i7 and VMWare Fusion Pro.

The buildlet is accessible on port 8080 from the host, if you know its IP:

And you can create a start-up item of a +x file named start-buildlet.command (the *.command extension is key) which launches a shell script in Terminal:

We can then snapshot that and create VMs which boot like that, in ~57 seconds. The host API server on the outside should probably cache the ~6MB buildlet too, to reduce network I/O. We can do that later, though.

Here is the first draft of the API: https://github.com/c4milo/osx-builder.

@bradfitz, keep in mind that Login Items will only run upon the user logging in. It might be more convenient for the API if they run on boot as there won't be need to log into the guest OS in order to make the script run. As I mentioned in our email thread, this can be done through Launchd. There is more information about Launchd here: https://developer.apple.com/library/mac/documentation/MacOSX/Conceptual/BPSystemStartup/BPSystemStartup.pdf

I configured the user to automatically log in, so the Login Items works great.

oh, yeah that works too. nice!

@bradfitz what is left here?

@c4milo we're going to set up a small cluster of OS X machines to use as builders (@crawshaw is doing that). Once they're set up, we'll know what else needs to be done, if anything.

@c4milo in addition to what Andrew said, I also haven't had found time yet to use what you've made. I created a VM in VMWare that boots right into a buildlet and I could hit it from the outside, which was enough to convince me that the concept would work, but I haven't tested it end-to-end.

We'll also need to add support for the daemon that runs on the host to (your code) to connect to the build coordinator and setup a reverse tunnel back to it, for our farm of Macs running in an isolated network.

CL https://golang.org/cl/28218 mentions this issue.

CL https://golang.org/cl/28572 mentions this issue.

CL https://golang.org/cl/28570 mentions this issue.

CL https://golang.org/cl/28584 mentions this issue.

@bradfitz Can we closed this bug now? farmer thinks there are a bunch of running darwin buildlets, which I think are in virtualization.

The coordinator doesn't yet directly manage the creation of VMs.

The current setup is a transitional hack to get us going where they're VMs, but they're created by a tool which creates them statically once, by hand, and then they just act like normal static machines in reverse mode. But if they die, they never come back to life, and we can't react to load or demand. Right now we only create one OS X 10.8 VM, 3 10.8 VMs, and the rest 10.11. That's pretty arbitrary. If four people wanted OS X 10.8 VMs via gomote, we can't create that right now.

So I want to keep this open until things are better integrated.

CL https://golang.org/cl/37457 mentions this issue.

Change https://golang.org/cl/82355 mentions this issue: cmd/buildlet: always halt Mac VMs after builds, regardless of macOS version