Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build/cmd/gopherbot: keeps repeating some actions without effect #28320

Open
1 of 3 tasks
dmitshur opened this issue Oct 22, 2018 · 11 comments
Open
1 of 3 tasks

x/build/cmd/gopherbot: keeps repeating some actions without effect #28320

dmitshur opened this issue Oct 22, 2018 · 11 comments
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. umbrella
Milestone

Comments

@dmitshur
Copy link
Contributor

dmitshur commented Oct 22, 2018

I noticed this in the gopherbot logs recently, and it's possible to reproduce locally in dry-run mode. Gopherbot keeps on repeating some actions every iteration. These actions don't have an effect, and so gopherbot never stops repeating them.

2018/10/22 07:44:15 got corpus update after 252.395017ms
2018/10/22 07:44:15 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135536
2018/10/22 07:44:15 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135536...
2018/10/22 07:44:15 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135537
2018/10/22 07:44:15 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135537...
2018/10/22 07:44:16 closing scratch CL https://golang.org/cl/71730 ...
2018/10/22 07:44:16 closing scratch CL https://golang.org/cl/71850 ...
2018/10/22 07:44:16 closing scratch CL https://golang.org/cl/72090 ...
2018/10/22 07:44:16 closing scratch CL https://golang.org/cl/72091 ...
2018/10/22 07:44:16 closing scratch CL https://golang.org/cl/72110 ...
2018/10/22 07:44:16 closing scratch CL https://golang.org/cl/72131 ...
2018/10/22 07:44:16 gopherbot ran in 1.335854405s
Reproduce Steps
$ go get -u golang.org/x/build/cmd/gopherbot
$ go run golang.org/x/build/cmd/gopherbot -dry-run -daemon
2018/10/22 14:27:47 Loading data from log *maintner.netMutSource ...
2018/10/22 14:27:47 Downloading 39043 bytes of https://maintner.golang.org/logs/41 ...
2018/10/22 14:27:47 wrote /Users/dmitshur/Library/Caches/golang-maintner/0041.growing.mutlog
2018/10/22 14:27:55 Reloaded data from log *maintner.netMutSource.
2018/10/22 14:27:56 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135536
2018/10/22 14:27:56 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135536...
2018/10/22 14:27:56 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135537
2018/10/22 14:27:56 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135537...
2018/10/22 14:27:56 [dry-run] would've closed scratch CL https://golang.org/cl/71730 ...
2018/10/22 14:27:56 [dry-run] would've closed scratch CL https://golang.org/cl/71850 ...
2018/10/22 14:27:56 [dry-run] would've closed scratch CL https://golang.org/cl/72090 ...
2018/10/22 14:27:56 [dry-run] would've closed scratch CL https://golang.org/cl/72091 ...
2018/10/22 14:27:56 [dry-run] would've closed scratch CL https://golang.org/cl/72110 ...
2018/10/22 14:27:56 [dry-run] would've closed scratch CL https://golang.org/cl/72131 ...
2018/10/22 14:27:56 gopherbot ran in 1.434313646s
2018/10/22 14:27:56 Updating data from log *maintner.netMutSource ...
2018/10/22 14:29:22 Downloading 1242 bytes of https://maintner.golang.org/logs/41 ...
2018/10/22 14:29:22 wrote /Users/dmitshur/Library/Caches/golang-maintner/0041.growing.mutlog
2018/10/22 14:29:22 Reloaded data from log *maintner.netMutSource.
2018/10/22 14:29:22 got corpus update after 1m25.647922592s
2018/10/22 14:29:23 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135536
2018/10/22 14:29:23 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135536...
2018/10/22 14:29:23 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135537
2018/10/22 14:29:23 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135537...
2018/10/22 14:29:23 [dry-run] would've closed scratch CL https://golang.org/cl/71730 ...
2018/10/22 14:29:23 [dry-run] would've closed scratch CL https://golang.org/cl/71850 ...
2018/10/22 14:29:23 [dry-run] would've closed scratch CL https://golang.org/cl/72090 ...
2018/10/22 14:29:23 [dry-run] would've closed scratch CL https://golang.org/cl/72091 ...
2018/10/22 14:29:23 [dry-run] would've closed scratch CL https://golang.org/cl/72110 ...
2018/10/22 14:29:23 [dry-run] would've closed scratch CL https://golang.org/cl/72131 ...
2018/10/22 14:29:23 gopherbot ran in 860.347827ms
2018/10/22 14:29:23 Updating data from log *maintner.netMutSource ...
2018/10/22 14:29:23 Downloading 1317 bytes of https://maintner.golang.org/logs/41 ...
2018/10/22 14:29:23 wrote /Users/dmitshur/Library/Caches/golang-maintner/0041.growing.mutlog
2018/10/22 14:29:23 gerrit code.googlesource.com/gocloud: Ref {CLNumber:34491 Version:0} => 4702bd58135b78127c759a94658b7606e5445e46
2018/10/22 14:29:23 Reloaded data from log *maintner.netMutSource.
2018/10/22 14:29:23 got corpus update after 195.853575ms
2018/10/22 14:29:24 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135536
2018/10/22 14:29:24 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135536...
2018/10/22 14:29:24 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135537
2018/10/22 14:29:24 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135537...
2018/10/22 14:29:24 [dry-run] would've closed scratch CL https://golang.org/cl/71730 ...
2018/10/22 14:29:24 [dry-run] would've closed scratch CL https://golang.org/cl/71850 ...
2018/10/22 14:29:24 [dry-run] would've closed scratch CL https://golang.org/cl/72090 ...
2018/10/22 14:29:24 [dry-run] would've closed scratch CL https://golang.org/cl/72091 ...
2018/10/22 14:29:24 [dry-run] would've closed scratch CL https://golang.org/cl/72110 ...
2018/10/22 14:29:24 [dry-run] would've closed scratch CL https://golang.org/cl/72131 ...
2018/10/22 14:29:24 gopherbot ran in 726.996791ms
2018/10/22 14:29:24 Updating data from log *maintner.netMutSource ...
2018/10/22 14:29:41 Downloading 37 bytes of https://maintner.golang.org/logs/41 ...
2018/10/22 14:29:41 wrote /Users/dmitshur/Library/Caches/golang-maintner/0041.growing.mutlog
2018/10/22 14:29:41 Reloaded data from log *maintner.netMutSource.
2018/10/22 14:29:41 got corpus update after 17.501948429s
2018/10/22 14:29:42 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135536
2018/10/22 14:29:42 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135536...
2018/10/22 14:29:42 No reviewers or cc: https://go-review.googlesource.com/c/gddo/+/135537
2018/10/22 14:29:42 Adding no-owners tag to change https://go-review.googlesource.com/c/gddo/+/135537...
2018/10/22 14:29:42 [dry-run] would've closed scratch CL https://golang.org/cl/71730 ...
2018/10/22 14:29:42 [dry-run] would've closed scratch CL https://golang.org/cl/71850 ...
2018/10/22 14:29:42 [dry-run] would've closed scratch CL https://golang.org/cl/72090 ...
2018/10/22 14:29:42 [dry-run] would've closed scratch CL https://golang.org/cl/72091 ...
2018/10/22 14:29:42 [dry-run] would've closed scratch CL https://golang.org/cl/72110 ...
2018/10/22 14:29:42 [dry-run] would've closed scratch CL https://golang.org/cl/72131 ...
2018/10/22 14:29:42 gopherbot ran in 723.348371ms
2018/10/22 14:29:42 Updating data from log *maintner.netMutSource ...

It's pretty harmless, but would be nice to fix.

The underlying issues causing this are:

Fixing those issues should resolve this issue.

/cc @bradfitz @andybons

@dmitshur dmitshur added the Builders x/build issues (builders, bots, dashboards) label Oct 22, 2018
@gopherbot gopherbot added this to the Unreleased milestone Oct 22, 2018
@gopherbot
Copy link

Change https://golang.org/cl/152779 mentions this issue: maintner: fix GerritMeta.Hashtags to look at earlier meta parents for answer

gopherbot pushed a commit to golang/build that referenced this issue Dec 7, 2018
… answer

The Gerrit meta commit graph is a linear history. The most recent meta
with a "Hashtags: " footer line has the complete set. We just have to
go back and look for it.

Fixes golang/go#28318
Updates golang/go#28510 (fixes after gopherbot re-deployed)
Updates golang/go#28320 (fixes after gopherbot re-deployed)

Change-Id: I43705075800ae3d353c1c8f60ab7685883ea5602
Reviewed-on: https://go-review.googlesource.com/c/152779
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
@dmitshur
Copy link
Contributor Author

There's a new addition, as of #30182:

2019/02/12 01:45:58 got corpus update after 24.897577657s
freeze
	https://golang.org/issue/23772  x/build/cmd/gerritbot: <fill this in>
2019/02/12 01:45:59 closing scratch CL https://golang.org/cl/71730 ...
2019/02/12 01:45:59 closing scratch CL https://golang.org/cl/71850 ...
2019/02/12 01:45:59 closing scratch CL https://golang.org/cl/72090 ...
2019/02/12 01:45:59 closing scratch CL https://golang.org/cl/72091 ...
2019/02/12 01:45:59 closing scratch CL https://golang.org/cl/72110 ...
2019/02/12 01:45:59 closing scratch CL https://golang.org/cl/72131 ...
2019/02/12 01:45:59 gopherbot ran in 1.894136882s

To fix the endless freezing of an issue that's 404, we'll need to update maintner so that when an issue goes 404, its NotExist field gets set to true. I've opened #30184 for tracking that.

@dmitshur
Copy link
Contributor Author

The current state as of September 2019:


2019/09/12 13:33:28 Downloading 68 bytes of https://maintner.golang.org/logs/56 ...
2019/09/12 13:33:28 wrote /cache/golang-maintner/0056.growing.mutlog
2019/09/12 13:33:28 Reloaded data from log *maintner.netMutSource.
2019/09/12 13:33:28 got corpus update after 2m24.51314801s
freeze
	https://golang.org/issue/23772  x/build/cmd/gerritbot: <fill this in>
	https://golang.org/issue/27223  x/mobile: 
2019/09/12 13:33:30 Issue 32620 already has label "Go2"; no need to send request to add it
2019/09/12 13:33:30 Issue 32620 already has label "LanguageChange"; no need to send request to add it
2019/09/12 13:33:30 Issue 33171 already has label "Go2"; no need to send request to add it
2019/09/12 13:33:31 No reviewers or cc: https://go-review.googlesource.com/c/website/+/192418
2019/09/12 13:33:31 Setting review on https://go-review.googlesource.com/c/website/+/192418: {Message: Labels:map[] Comments:map[] Reviewers:[{Reviewer:dmitshur@golang.org State:} {Reviewer:channing@golang.org State:CC} {Reviewer:365204+andybons@users.noreply.github.com State:CC}]}
2019/09/12 13:33:31 Could not set review for change "website~192418": HTTP status 400 Bad Request; )]}'
{
  "reviewers": {
    "365204+andybons@users.noreply.github.com": {
      "input": "365204+andybons@users.noreply.github.com",
      "error": "Account \u0027365204+andybons@users.noreply.github.com\u0027 not found\n365204+andybons@users.noreply.github.com does not identify a registered user or group"
    },
    "dmitshur@golang.org": {
      "input": "dmitshur@golang.org"
    },
    "channing@golang.org": {
      "input": "channing@golang.org"
    }
  },
  "error": "error adding reviewer"
}
2019/09/12 13:33:32 No reviewers or cc: https://go-review.googlesource.com/c/sys/+/178400
2019/09/12 13:33:32 Setting review on https://go-review.googlesource.com/c/sys/+/178400: {Message: Labels:map[] Comments:map[] Reviewers:[{Reviewer:iant@golang.org State:} {Reviewer:bradfitz@golang.org State:} {Reviewer:tklauser@distanz.ch State:}]}
2019/09/12 13:33:33 Could not set review for change "sys~178400": HTTP status 400 Bad Request; )]}'
{
  "reviewers": {
    "tklauser@distanz.ch": {
      "input": "tklauser@distanz.ch",
      "error": "Account \u0027tklauser@distanz.ch\u0027 not found\ntklauser@distanz.ch does not identify a registered user or group"
    },
    "iant@golang.org": {
      "input": "iant@golang.org"
    },
    "bradfitz@golang.org": {
      "input": "bradfitz@golang.org"
    }
  },
  "error": "error adding reviewer"
}
2019/09/12 13:33:33 closing scratch CL https://golang.org/cl/71730 ...
2019/09/12 13:33:33 closing scratch CL https://golang.org/cl/71850 ...
2019/09/12 13:33:33 closing scratch CL https://golang.org/cl/72090 ...
2019/09/12 13:33:33 closing scratch CL https://golang.org/cl/72091 ...
2019/09/12 13:33:34 closing scratch CL https://golang.org/cl/72110 ...
2019/09/12 13:33:35 closing scratch CL https://golang.org/cl/72131 ...
2019/09/12 13:33:36 gopherbot ran in 7.460417149s

@gopherbot
Copy link

Change https://golang.org/cl/195062 mentions this issue: internal/gophers: restore valid Gerrit emails (again)

@gopherbot
Copy link

Change https://golang.org/cl/176038 mentions this issue: cmd/gopherbot: update map of deleted CLs to include some from scratch

gopherbot pushed a commit to golang/build that referenced this issue Sep 13, 2019
Updates golang/go#22635
Updates golang/go#28320

Change-Id: I8769dc83187ad1b1b2c1d751818964565f34e6d3
Reviewed-on: https://go-review.googlesource.com/c/build/+/176038
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Sep 18, 2019
This is a cherry-pick of CL 135456 that restores Gerrit emails
for some people that were incorrectly changed in CL 165639, with
manual no-op addPerson line merges to address code review comments.

The cherry-pick applied very cleanly with just two minor merge
conflicts: one due to a Filippo's email already being fixed in
CL 176037, and another due to a close new entry.

Add tests to catch this from happening again, until the source
of the problem is resolved in issue golang/go#34259.

Updates golang/go#34259
Updates golang/go#28320
Updates golang/go#31919
Updates golang/go#27517
Updates golang/go#27631

Change-Id: Ia03a2b94403334d3f571ac5623e12d3bfd6f1e4f
Reviewed-on: https://go-review.googlesource.com/c/build/+/195062
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
codebien pushed a commit to codebien/build that referenced this issue Nov 13, 2019
Updates golang/go#22635
Updates golang/go#28320

Change-Id: I8769dc83187ad1b1b2c1d751818964565f34e6d3
Reviewed-on: https://go-review.googlesource.com/c/build/+/176038
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
codebien pushed a commit to codebien/build that referenced this issue Nov 13, 2019
This is a cherry-pick of CL 135456 that restores Gerrit emails
for some people that were incorrectly changed in CL 165639, with
manual no-op addPerson line merges to address code review comments.

The cherry-pick applied very cleanly with just two minor merge
conflicts: one due to a Filippo's email already being fixed in
CL 176037, and another due to a close new entry.

Add tests to catch this from happening again, until the source
of the problem is resolved in issue golang/go#34259.

Updates golang/go#34259
Updates golang/go#28320
Updates golang/go#31919
Updates golang/go#27517
Updates golang/go#27631

Change-Id: Ia03a2b94403334d3f571ac5623e12d3bfd6f1e4f
Reviewed-on: https://go-review.googlesource.com/c/build/+/195062
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label May 14, 2020
@gopherbot
Copy link

Change https://golang.org/cl/308790 mentions this issue: cmd/gopherbot: add more deleted issues to deletedIssues map

gopherbot pushed a commit to golang/build that referenced this issue Apr 9, 2021
A good amount of time has passed since the deletedIssues map was last
updated, and the "freeze old issues" task was needlessly making 34 API
calls to freeze issues that are gone. After this change, that task is
making 0 API calls (whenever there aren't existing issues to freeze).

Some gardening tasks were converted to be more general and run on more
issue trackers in CL 233377, so update the deletedIssues map to track
the repo ID in addition to the issue number.

For golang/go#28320.
Updates golang/go#22635.
Updates golang/go#44406.
Updates golang/go#39008.

Change-Id: I3b477bf717f7d97676e9ef950214a3598ec3abd2
Reviewed-on: https://go-review.googlesource.com/c/build/+/308790
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
@gopherbot
Copy link

Change https://go.dev/cl/386754 mentions this issue: cmd/gopherbot: drop non-Gerrit owners for review assignment

gopherbot pushed a commit to golang/build that referenced this issue Feb 18, 2022
Owners may include users that do not have an associated Gerrit account,
notably GitHub Teams. When determining reviewers to add a CL, drop all
these users from the potential reviewer set. Otherwise Gerrit chokes on
attempting to set reviewer to "".

Dropping these users from owners may result in an empty primary reviewer
set (e.g., on packages where a GitHub team is the only primary owner).
In this case, upgrade the secondary owners to primary so they end up as
reviewers rather than just CC'd. If both lists are empty, a 'no-owners'
tag is added to the CL by assignReviewersToCLs.

For golang/go#50723.
For golang/go#28320.

Change-Id: I99ac5426e080bf32c46817c09debeac507282e01
Reviewed-on: https://go-review.googlesource.com/c/build/+/386754
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
@gopherbot
Copy link

Change https://go.dev/cl/397634 mentions this issue: cmd/gopherbot: remember which issues are discovered to be gone

@gopherbot
Copy link

Change https://go.dev/cl/397635 mentions this issue: cmd/gopherbot: skip issues that are gone more consistently

gopherbot pushed a commit to golang/build that referenced this issue Apr 8, 2022
Issues that are gone don't come back. When gopherbot encounters such
issues, it might as well remember it for the next iteration.

This change makes it so that repeating actions without effect
happen once per gopherbot startup, rather than on each iteration.
This is to reduce needless GitHub API quota consumption.

Also update the initial deletedIssues value with newly gone issues.

For golang/go#28320.
Updates golang/go#40640.

Change-Id: I36f3cb780c96ac407482d6c40b4c3e935286e10b
Reviewed-on: https://go-review.googlesource.com/c/build/+/397634
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Apr 8, 2022
The maintner Go corpus makes all issues available to its caller,
including those that are gone, are Pull Requests, are closed,
and so on. Most of gopherbot tasks operate on open non-PR issues
that aren't gone, and need to filter out non-actionable issues.

Create a wrapper that adapts maintner's ForeachIssue API for the
needs of gopherbot, so the code checking an issue's NotExist field or
presence in deletedIssues map doesn't need to be copied in each task.

Tested by running gopherbot in dry-run mode.
It didn't try to take any unexpected actions.

For golang/go#28320.
Updates golang/go#40640.

Change-Id: Icf235d046df5d371217ea14f86911ed7a39e6a3f
Reviewed-on: https://go-review.googlesource.com/c/build/+/397635
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
@gopherbot
Copy link

Change https://go.dev/cl/442156 mentions this issue: cmd/gopherbot: skip more deleted issues in addGitHubComment

gopherbot pushed a commit to golang/build that referenced this issue Oct 14, 2022
The deleted issue 55403 wasn't being detected as deleted and causing
errors. Add a little logic to handle this category of deleted issues.

For golang/go#28320.
Updates golang/go#30184.

Change-Id: I37f06a1330b6993e5526ea196074102c96fe98f9
Reviewed-on: https://go-review.googlesource.com/c/build/+/442156
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
@gopherbot
Copy link

Change https://go.dev/cl/502855 mentions this issue: internal/gophers: update Kevin Burke's Gerrit email

gopherbot pushed a commit to golang/build that referenced this issue Jun 22, 2023
Gopherbot fails to add the previous email as a reviewer because it's
not associated with any Gerrit account anymore. Update it to the one
reported by 'curl https://go-review.googlesource.com/accounts/13437'.
Also merge the two addPerson calls into one while here.

For golang/go#28320.

Change-Id: If2ee53299d9f54eb1d1d3e8609e7c92ff705f91e
Reviewed-on: https://go-review.googlesource.com/c/build/+/502855
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. umbrella
Projects
None yet
Development

No branches or pull requests

2 participants