Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: linux-mips64 builder not reporting success or failure #19953

Closed
bradfitz opened this issue Apr 12, 2017 · 3 comments
Closed

x/build: linux-mips64 builder not reporting success or failure #19953

bradfitz opened this issue Apr 12, 2017 · 3 comments
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge
Milestone

Comments

@bradfitz
Copy link
Contributor

(Continued from #19930 (comment))

The linux-mips64 builder keeps working, but no success or failure shows up on build.golang.org, which means the builder is disappearing or something.

I see make.bash finishes in 13 minutes, and now I'm watching it do the snapshot copy to Google Cloud Storage. There's a comment on that code that says:

func (st *buildStatus) writeSnapshot(bc *buildlet.Client) (err error) {
        sp := st.createSpan("write_snapshot_to_gcs")
        defer func() { sp.done(err) }()
        // This should happen in 15 seconds or so, but I saw timeouts                                                                                                          
        // a couple times at 1 minute. Some buildlets might be far                                                                                                             
        // away on the network, so be more lenient. The timeout mostly                                                                                                         
        // is here to prevent infinite hangs.                                                                                                                                  
        ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
        defer cancel()

There's a 5 minute timeout. It's possible those arm5 machines can't gzip quickly, and/or their network is slow.

Yup, confirmed:

Error: writeSnapshot: net/http: request canceled

Maybe we don't need to snapshot those builds to Google Cloud Storage. That means we can't shard tests, but we're not doing that currently anyway.

/cc @vstefanovic @adams-sarah @cybrcodr

@bradfitz bradfitz added the Builders x/build issues (builders, bots, dashboards) label Apr 12, 2017
@bradfitz bradfitz added this to the Unreleased milestone Apr 12, 2017
@bradfitz bradfitz self-assigned this Apr 12, 2017
@gopherbot
Copy link
Contributor

CL https://golang.org/cl/40505 mentions this issue.

@bradfitz
Copy link
Contributor Author

mips64 has an "ok" on the dashboard now.

@vstefanovic
Copy link
Member

@bradfitz, thanks!

(mips and mips64 builders are slow partly because they run on the same machine, they are slowing down each other.)

@golang golang locked and limited conversation to collaborators Apr 13, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge
Projects
None yet
Development

No branches or pull requests

3 participants