-
Notifications
You must be signed in to change notification settings - Fork 17.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
x/build: cross-compile ARM for speed? #17105
Comments
SGTM |
SGTM. It should be fine as there are already many tests that do invoke the compiler/linker/etc. |
SGTM.
|
Good point. I didn't consider that. So maybe the parallel |
I think running |
Yeah, it makes me feel more comfortable too. And it's easy enough and still basically within my time goal. |
CL https://golang.org/cl/29670 mentions this issue. |
This is a new builder in prep for the change to the "linux-arm" builder where the GOARCH=arm make.bash will be cross-compiled from a Kubernetes container on fast hardware. Updates golang/go#17105 (cross-compile ARM builders' make.bash) Updates golang/go#17104 (5 minute trybots) Change-Id: Icfd2644d77639f731151abe54839322960418254 Reviewed-on: https://go-review.googlesource.com/29670 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
CL https://golang.org/cl/29677 mentions this issue. |
With @jfrazelle's help, we've almost got this working. But we just saw a crash when running the cross-built GOROOT files on the normal Scaleway.com ARM machine:
Not sure what to make of that. |
But on the same machine, it seems to work by hand:
|
It's weird that those tests failed with a segmentation fault but that there was no output. One thing that can cause that is if the signal handler itself gets a signal, but neither of those tests is expected to get a signal. Both of those tests involve a non-Go thread calling a Go function, so my guess is that it has something to do with that. I can't think of anything else useful. |
And it failed on the staging builder again in the same way. Doesn't seem to be a flake. |
@ianlancetaylor, the only difference I can see between how I'm running it "by hand" vs by the builders is that when I run it by hand and it works, it's running under bash. The builders run it under the Go buildlet binary. Is there some environment difference I'm not considering? |
For these tests the test itself will run |
That is, what is |
@ianlancetaylor, ah hah! I bet that's the issue. I can totally believe the CC_FOR_TARGET or CGO_ENABLED isn't being set in the tests. Thanks! |
Er, on second thought: we're not cross-compiling when running the tests, so we're using the system default:
In the Kubernetes container where we cross-compile make.bash, we use https://packages.debian.org/stretch/gcc-arm-linux-gnueabihf ...
Are you saying that mixing those is the problem? Maybe we need an older arm-linux-gnueabihf-gcc version? |
The Scaleway machines are running Debian GNU/Linux 8.1 (jessie). |
The far superior linux distro of champions. Updates golang/go#17105 Change-Id: I5ea0cd2361753f61bb74bf3d4dea6c181f1427fa Reviewed-on: https://go-review.googlesource.com/29687 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We switched to Jessie in the cross-compiling Kubernetes container, but no luck. It still fails, and I see this on Scaleway builder just before it fails while running the test:
And on that same Scaleway machine:
|
And in the Kubernetes container:
So at least now we seem to be running the same compiler? (albeit one on an |
@jfrazelle and I are stumped. Going to take a break from this for now. Clues welcome. |
What system libraries are available on the cross-compiler host and on the real host? Is there any for you to snag a copy of the testproccgo program that is failing? |
Can you run one of the failing binaries under gdb? |
I'll work on both those things. I just finally now got it to reproduce by hand in a shell. I made the buildlet log the environment it runs the test with: In a browser, watching the hacked-up coordinator:
And then I was able to make it do it by hand: In ssh:
So now I can actually modify things easily and see what's happening I hope. |
Mystery/clue: running
(Almost all that time is TestCollisions, it turns out... #17217) |
Okay, got a binary.
Wrong GOARM= level? |
From func xgetgoarm() string {
//...
if gohostarch != "arm" || goos != gohostos {
// Conservative default for cross-compilation.
return "5"
} That's my best guess at the moment. |
On Scaleway,
So I should probably make the Kubernetes cross-compiler set |
Yup! That was it. Thanks @jfrazelle, @ianlancetaylor, and @crawshaw! |
Omg yay!!! On Friday, September 23, 2016, Brad Fitzpatrick notifications@github.com
Jessie Frazelle |
That default cross compilation setting gets you every time. We should On Sat, Sep 24, 2016 at 12:32 PM, Brad Fitzpatrick <notifications@github.com
|
As part of #17104 to improve trybot speed and get Trybots down to 5 minutes, I now see that
linux-arm
is the slowest builder, even sharded 8 machines wide.The problem is that just
make.bash
onlinux-arm
takes 5 minutes itself, even without running any tests, so sharding 8 machines wide doesn't help much.What do people think about cross-compiling the
linux-arm
make.bash
onlinux-amd64
(on Kubernetes) first (which takes about 33 seconds), and then pushing that out to 7 real ARM machines for tests? (instead of pushing out the same everything-built tarball from the ARM machine)In parallel, we could run a real ARM
make.bash
(for 5 minutes) to verify it works, but never use its output for testing.Thoughts?
/cc @ianlancetaylor @quentinmit @davecheney @minux @cherrymui
The text was updated successfully, but these errors were encountered: