Skip to content

x/build/cmd/watchflakes: opens duplicate issues #67393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
dmitshur opened this issue May 15, 2024 · 3 comments
Closed

x/build/cmd/watchflakes: opens duplicate issues #67393

dmitshur opened this issue May 15, 2024 · 3 comments
Assignees
@dmitshur
Labels
Builders x/build issues (builders, bots, dashboards) NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Unreleased

Comments

@dmitshur
Copy link
Contributor

dmitshur commented May 15, 2024 •
edited
Loading

watchflakes is opening many duplicate issues recently, like "runtime:cpu4: TestUsingVDSO failures" is #67392 and #67391 and others.

I suspect the problem is that they're not added to the Test Flakes project successfully, so watchflakes doesn't realize it already filed an existing issue.

CC @golang/release.
@dmitshur dmitshur added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Soon This needs action soon. (recent regressions, service outages, unusual time-sensitive situations) labels May 15, 2024
@dmitshur dmitshur added this to the Unreleased milestone May 15, 2024
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label May 15, 2024
@dmitshur dmitshur self-assigned this May 15, 2024
@dmitshur dmitshur moved this to In Progress in Go Release May 15, 2024
@dmitshur
Copy link
Contributor Author

Logs show it's running into a limit of items that can be added to a GitHub project:

# new issue: runtime:cpu4: TestUsingVDSO failures
2024-05-15 13:52 gotip-linux-ppc64_power8 go@6ccd8e4c runtime:cpu4.TestUsingVDSO
https://ci.chromium.org/b/8747866791326770913
pkg == "runtime:cpu4" && test == "TestUsingVDSO"

=== RUN   TestUsingVDSO
=== PAUSE TestUsingVDSO
=== CONT  TestUsingVDSO
    vdso_test.go:51: GO_WANT_HELPER_PROCESS=1 /bin/strace -f -e clock_gettime /home/swarming/.swarming/w/ir/x/t/go-build3885972143/b001/runtime.test -test.run=^TestUsingVDSO$
    vdso_test.go:57: /bin/strace: Process 140051 attached
        /bin/strace: Process 140052 attached
        /bin/strace: Process 140050 attached
        /bin/strace: Process 140053 attached
        [pid 140050] ???( <unfinished ...>
        [pid 140049] --- stopped by SIGURG ---
        /bin/strace: ptrace(PTRACE_LISTEN,pid:140049,sig:0): Input/output error
        /bin/strace: Process 140049 detached
        /bin/strace: Process 140051 detached
        /bin/strace: Process 140052 detached
        /bin/strace: Process 140050 detached
        /bin/strace: Process 140053 detached
    vdso_test.go:70: exit status 1
--- FAIL: TestUsingVDSO (0.14s)

watchflakes: 2024/05/15 16:10:33 1
watchflakes: 2024/05/15 16:10:33 2		  mutation($Project: ID!, $Issue: ID!) {
watchflakes: 2024/05/15 16:10:33 3		    addProjectV2ItemById(input: {projectId: $Project, contentId: $Issue}) {
watchflakes: 2024/05/15 16:10:33 4		      clientMutationId
watchflakes: 2024/05/15 16:10:33 5		    }
watchflakes: 2024/05/15 16:10:33 6		  }
watchflakes: 2024/05/15 16:10:33 7		
watchflakes: 2024/05/15 16:10:33 graphql error: Projects cannot have more than 1200 items. To add more, please archive or delete existing items.

@dmitshur
Copy link
Contributor Author

I've stopped it from running as a short-term workaround. Removing Soon label.

We've applied to https://github.blog/changelog/2024-02-12-github-issues-projects-projects-without-limits-private-beta/, which should help.

In the meantime, I'll work on a change to make it handle this limit more gracefully (i.e., don't keep opening new issues without end).

@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed Soon This needs action soon. (recent regressions, service outages, unusual time-sensitive situations) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels May 15, 2024
@dmitshur
Copy link
Contributor Author

We've made two changes to help with this:

  • there's now an auto-archive workflow in the Test Flakes project
    • it archives closed issues after 4 weeks of inactivity; we can adjust the exact time to strike the optimal balance between it reopening existing issue vs opening a new one
  • we've started taking advantage of GitHub's early support for removing limits on project issue count

That should prevent this from happening again. After archiving closed issues that hadn't been updated in the last 4 weeks, there's now 519 items in the Test Flakes project (down from 1200).

I've resumed watchflakes and it's working okay now. Closing this as resolved.

@github-project-automation github-project-automation bot moved this from In Progress to Done in Go Release May 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dmitshur dmitshur

Labels
Builders x/build issues (builders, bots, dashboards) NeedsFix The path to resolution is known, but the work has not been done.
Projects
Go Release
Archived in project
Milestone
Unreleased
Development

No branches or pull requests
2 participants
@dmitshur @gopherbot