-
Notifications
You must be signed in to change notification settings - Fork 17.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
x/build: one linux/arm builder (scaleway-prod-50) is missing and not coming back #32801
Comments
The scaleway deployment had 2 restarts:
(Not sure what happened in the restart prior to that one, but likely a similar timeout issue.) Since then,
It was hopelessly wedged. I manually logged in to Scaleway UI and moved it out of the way.
https://farmer.golang.org/status/scaleway says "ok": This issue is resolved now. (If failures such as this one become very common, we can do more work to automate their resolution.) |
Note, I'm getting an "Error: server should be running" error from Scaleway UI when trying to forcibly shut it down or delete it, so this kind of issue is unlikely to be possible to automate. The wedged instance is moved out of the way so it's not a problem; I'll try again in a few days and file a ticket with them if it's still un-deletable by then. |
From https://farmer.golang.org/#health:
Only 1 machine out of 50 is missing (98% of
linux/arm
builder capacity is still there), so this isn't a serious or urgent issue.But need to investigate and see why it's not coming back. The
x/build/cmd/scaleway
daemon is supposed to monitor and restart wedged instances.The text was updated successfully, but these errors were encountered: