Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/build/cmd/gopherbot: race condition in auto-closure of scratch CLs causing gopherbot to stall 30s #28286

Open
andybons opened this issue Oct 19, 2018 · 3 comments
Open

x/build/cmd/gopherbot: race condition in auto-closure of scratch CLs causing gopherbot to stall 30s #28286

andybons opened this issue Oct 19, 2018 · 3 comments
Labels
Builders x/build issues (builders, bots, dashboards) NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Unreleased

Comments

@andybons
Copy link
Member

When closing scratch CLs, the maintner corpus doesn't get updated fast enough and it attempts to close the same CLs again, resulting in a 409 conflict:

2018/10/19 20:19:32 closing scratch CL https://golang.org/cl/140997 ...
2018/10/19 20:19:32 abandon scratch reviews: HTTP status 409 Conflict; change is abandoned
2018/10/19 20:19:32 HTTP status 409 Conflict; change is abandoned
2018/10/19 20:19:32 gopherbot ran in 1.745149115s
2018/10/19 20:19:32 sleeping 30s after previous error.

This error causes gopherbot to sleep for 30s as a result.

This is related to #28226 /cc @dmitshur
@andybons andybons added the NeedsFix The path to resolution is known, but the work has not been done. label Oct 19, 2018
@andybons andybons added this to the Unreleased milestone Oct 19, 2018
@dmitshur
Copy link
Contributor

dmitshur commented Oct 20, 2018 •
edited

I think I've already seen code somewhere that uses Gerrit API in addition to maintner to confirm the "latest" state before taking some action. @bradfitz will probably remember where that was.

I suspect the fix for this can use a similar approach.

Note that this is different from #28226 in that the reported state isn't inconsistent, it's just not up to date.

@dmitshur dmitshur changed the title cmd/gopherbot: race condition in auto-closure of scratch CLs causing gopherbot to stall 30s x/build/cmd/gopherbot: race condition in auto-closure of scratch CLs causing gopherbot to stall 30s Oct 20, 2018
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label Oct 20, 2018
@andybons
Copy link
Member Author

@dmitshur auto-assignment of reviews and gerritbot both use the Gerrit API so that’s likely where you saw it.

@bradfitz
Copy link
Contributor

@dmitshur, can you take this one?

@dmitshur dmitshur assigned dmitshur and unassigned bradfitz Oct 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Builders x/build issues (builders, bots, dashboards) NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
4 participants
@bradfitz @andybons @dmitshur @gopherbot