Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/build: dragonfly builders acting up #44584

Closed
tuxillo opened this issue Feb 24, 2021 · 3 comments
Closed

x/build: dragonfly builders acting up #44584

tuxillo opened this issue Feb 24, 2021 · 3 comments
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Unreleased

Comments

@tuxillo
Copy link
Contributor

tuxillo commented Feb 24, 2021

According to the farmer page, the dragonfly builders are stacking the builds up without (probably) finishing them.

host-dragonfly-amd64-5_8: 1234 waiting (oldest 42h1m51s, newest 1m52s, progress 2m53s)
host-dragonfly-amd64-master: 1259 waiting (oldest 42h1m26s, newest 1m52s, progress 5m51s)

Not sure what's going on or how to troubleshoot. Can you please give us a hint?

Thanks.
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label Feb 24, 2021
@gopherbot gopherbot added this to the Unreleased milestone Feb 24, 2021
@dmitshur
Copy link
Contributor

dmitshur commented Mar 3, 2021

Looking at it now, I see both builders are in use:

host-dragonfly-amd64-5_8: 1/1
host-dragonfly-amd64-master: 1/1

dragonfly-amd64 rev 3e524ee6 (sub-repo review rev 2465c8e7); running; http://dragonflybsd-amd64 reverse peer dragonflybsd-amd64/199.233.90.83:51429 for host type host-dragonfly-amd64-master, 2h37m45s ago
...
  2021-03-03T17:38:41Z finish_write_go_src_tar after 6.99s
  2021-03-03T17:38:41Z make_and_test 
  2021-03-03T17:38:41Z make src/make.bash
 +149.3s (now)

dragonfly-amd64-5_8 rev 3e524ee6 (sub-repo website rev 02ef8fa5); running; http://dragonflybsd-amd64 reverse peer dragonflybsd-amd64/199.233.90.83:50368 for host type host-dragonfly-amd64-5_8, 2h37m37s ago
...
  2021-03-03T17:38:19Z finish_write_go_src_tar after 5.82s
  2021-03-03T17:38:19Z make_and_test 
  2021-03-03T17:38:19Z make src/make.bash
 +172.2s (now)

I think the large waiting number just means there's a lot of old commits that are available to be built. As long as it's actively working on recent commits, I don't think this is a problem.

CC @golang/release.

@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 3, 2021
@tuxillo
Copy link
Contributor Author

tuxillo commented Mar 5, 2021

Amazing, the backlog is not decreasing at all:

host-dragonfly-amd64-5_8: 2021 waiting (oldest 256h43m15s, newest 11m48s)
host-dragonfly-amd64-master: 2472 waiting (oldest 256h42m50s, newest 11m48s, progress 25m21s)

@bcmills bcmills mentioned this issue Mar 24, 2021
Closed
@tklauser tklauser mentioned this issue Mar 31, 2021
Closed
@dmitshur
Copy link
Contributor

It has come down by this time:

host-dragonfly-amd64-5_8: 518 waiting (oldest 24h38m57s, newest 2h30m48s, progress 2s)
host-dragonfly-amd64-master: 552 waiting (oldest 24h38m57s, newest 2h30m36s, progress 2m9s)

Even though it's a large number, it is stable, and it's consistent with how coordinator has behaved for a while.

I don't think there's a problem or something to do here at this time, but we can reopen or revisit this when there's more actionable information available.

@golang golang locked and limited conversation to collaborators Apr 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
3 participants
@tuxillo @dmitshur @gopherbot