Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build/cmd/gopherbot: doesn't skip issues that were transferred #45461

Closed
dmitshur opened this issue Apr 9, 2021 · 3 comments
Closed

x/build/cmd/gopherbot: doesn't skip issues that were transferred #45461

dmitshur opened this issue Apr 9, 2021 · 3 comments
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@dmitshur
Copy link
Contributor

dmitshur commented Apr 9, 2021

GopherBot doesn't skip issues that were transferred from one GitHub repository to another when processing them. The "set vscode-go milestones" task recently added in CL 308350 is highly affected by this, because there are many issues in golang/vscode-go that were transferred to the main Go issue tracker.

This can be reproduced locally with gopherbot in -dry-run mode:

$ go run golang.org/x/build/cmd/gopherbot -dry-run -only-run="set vscode-go milestones"
2021/04/08 19:02:23 Loading data from log *maintner.netMutSource ...
[...]
milestone-Untriaged [dry-run]
	https://github.com/golang/vscode-go/issues/144  support post-fix completion snippets
	https://github.com/golang/vscode-go/issues/151  Off by one error in name function name change
	https://github.com/golang/vscode-go/issues/205  Add location aware snippets for append and other utility methods
	https://github.com/golang/vscode-go/issues/211  Enabling language server breaks local modules
	https://github.com/golang/vscode-go/issues/241  x/tools/gopls: bad completion with syntax errors
	https://github.com/golang/vscode-go/issues/242  x/tools/gopls: offer completions for package names
	https://github.com/golang/vscode-go/issues/298  x/tools/gopls: "Inconsistent vendoring detected" message pops up with every keypress
	https://github.com/golang/vscode-go/issues/322  x/tools/gopls: provide foldingRange for the case sections within a select
	https://github.com/golang/vscode-go/issues/341  x/tools/gopls: crash in import organization
	https://github.com/golang/vscode-go/issues/348  Gopls very high CPU/memory/latency
	https://github.com/golang/vscode-go/issues/349  gopls: automated issue report (crash) -- SegFault (SIGSEGV: segmentation violation)
	https://github.com/golang/vscode-go/issues/353  x/tools/gopls: crash during SignatureHelp
	https://github.com/golang/vscode-go/issues/367  gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/369  gopls: high cpu and bad performance on large project with non-golang code located in adjacent directory and workspace including all code
	https://github.com/golang/vscode-go/issues/370  Link in popover goes to wrong section of package doc
	https://github.com/golang/vscode-go/issues/393  On save autoformat corrupts file based on number of lines removed
	https://github.com/golang/vscode-go/issues/421  gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/429  gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/431  gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/460  x/tools/gopls: improve workspace/symbol ranking
	https://github.com/golang/vscode-go/issues/470  The code in the workspace failed to compile (see the error message below).
	https://github.com/golang/vscode-go/issues/493  Show loading in statusbar before LSP is ready like Typescript does
	https://github.com/golang/vscode-go/issues/498  %s instead of name of the library to be added
	https://github.com/golang/vscode-go/issues/524  BUG: Saving file adds unknown text
	https://github.com/golang/vscode-go/issues/530  "go list" leaks in CPU usage
	https://github.com/golang/vscode-go/issues/533  gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/534  vscode-go hangs - mac  hard reboot mac is the only option
	https://github.com/golang/vscode-go/issues/535  x/tools/gopls: consider codelens for functions, variables references
	https://github.com/golang/vscode-go/issues/536  go.mod && replace directive
	https://github.com/golang/vscode-go/issues/540  x/tools/gopls: consider moving methods inside its types in outline view (DocumentSymbolRequest)
	https://github.com/golang/vscode-go/issues/560  x/tools/gopls: automated issue report (crash) - unhandled basic type
	https://github.com/golang/vscode-go/issues/562  x/tools/gopls: diagnostics crash with empty go.mod
	https://github.com/golang/vscode-go/issues/575  go/types: improve error messages by using compact package names
	https://github.com/golang/vscode-go/issues/596  gopls: automated issue report (initialization)
	https://github.com/golang/vscode-go/issues/617  x/tools/gopls: no folding ranges for invalid code
	https://github.com/golang/vscode-go/issues/676  go generate not working
	https://github.com/golang/vscode-go/issues/693  Blank identifiers ("_") show up in Outline view
	https://github.com/golang/vscode-go/issues/700  gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/741  Auto import does not choose latest version
	https://github.com/golang/vscode-go/issues/746  x/tools/gopls: option to autocomplete function call without parentheses
	https://github.com/golang/vscode-go/issues/791  Setting Up Workspace : Loading packages indefinetly 
	https://github.com/golang/vscode-go/issues/820  x/tools/gopls: Getting code actions from Go takes forever
	https://github.com/golang/vscode-go/issues/830  x/tools/gopls: SIGSEGV during renaming
	https://github.com/golang/vscode-go/issues/842  x/tools/gopls: crash in internal/lsp.(*Server).handleFatalErrors (v0.5.1)
	https://github.com/golang/vscode-go/issues/854  Go extension stuck on: "Setting up workspace: Loading packages..."
	https://github.com/golang/vscode-go/issues/876  x/tools/gopls: SIGSEGV in cache.missingModuleError
	https://github.com/golang/vscode-go/issues/908  x/tools/gopls: report errors for open files inside of vendor directories
	https://github.com/golang/vscode-go/issues/909  VSCode alters/corrupts source file on save if code contains a block comment
	https://github.com/golang/vscode-go/issues/915  x/tools/gopls: When accepting auto-completion, the suggestion is replaced over the current word (including the part after the cursor)
	https://github.com/golang/vscode-go/issues/946  x/tools/gopls: Import autocomplete appends suggestion instead of replacing
	https://github.com/golang/vscode-go/issues/955  Saving getting code actions from GO very long
	https://github.com/golang/vscode-go/issues/959  Frequent import error for standard library packages
	https://github.com/golang/vscode-go/issues/975  x/tools/gopls: out of memory (crash)
	https://github.com/golang/vscode-go/issues/977  gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/980  gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/995  x/tools/gopls: gopls stopped showing dependencies updates for go.mod after upgrading to v0.5.4
	https://github.com/golang/vscode-go/issues/1023  x/tools/gopls: extreme slowdown when using VSCode Go + gopls
	https://github.com/golang/vscode-go/issues/1028  2 minute stall on "Getting code actions from "Go"" every 20th save even though actions disabled
	https://github.com/golang/vscode-go/issues/1039  "Error loading workspace" after upgrading gopls to 0.60
	https://github.com/golang/vscode-go/issues/1062  Suppress signature help inside string literal
	https://github.com/golang/vscode-go/issues/1074  gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/1080  x/tools/gopls: .go files starting with "_" can cause error
	https://github.com/golang/vscode-go/issues/1083  x/tools/gopls: panic in internal/lsp/cache/imports.go
	https://github.com/golang/vscode-go/issues/1096  x/tools/gopls: rearranging struct fields deletes comments
	https://github.com/golang/vscode-go/issues/1100  gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/1126  x/tools/gopls/internal/lsp/cache: fatal error: concurrent map iteration and map write
	https://github.com/golang/vscode-go/issues/1139  x/tools/gopls: SQL code highlighted as website URLs
	https://github.com/golang/vscode-go/issues/1172  x/tools/gopls: incorrectly added type conversion
	https://github.com/golang/vscode-go/issues/1208  `unusedparams` should ignore unused params in functions satisfying the `http.HandlerFunc` type
	https://github.com/golang/vscode-go/issues/1217  x/tools/gopls: invalid memory address or nil pointer dereference in internal/lsp/source.HoverIdentifier
	https://github.com/golang/vscode-go/issues/1232  gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/1235  gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/1240  gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/1256  x/tools/gopls: Format constant time.Duration in hover tooltip
	https://github.com/golang/vscode-go/issues/1272  x/tools/gopls: do not suggest names from unimportable internal packages
	https://github.com/golang/vscode-go/issues/1279  errors in build in library
	https://github.com/golang/vscode-go/issues/1290  gopls: packages.Load error dependency issue
	https://github.com/golang/vscode-go/issues/1336  x/tools/gopls: crash in go/types.(*Checker).recordTypeAndValue
	https://github.com/golang/vscode-go/issues/1351  Set prompt properly when go package import alias is .
	https://github.com/golang/vscode-go/issues/1366  "error loading workspace" on apparently valid package
	https://github.com/golang/vscode-go/issues/1369  x/tools/gopls: automated issue report (crash)
	https://github.com/golang/vscode-go/issues/1372  cmd/go: clarify the implication when module name is 'main'
	https://github.com/golang/vscode-go/issues/1402  gopls: automated issue report (crash)
2021/04/08 19:02:46 gopherbot ran in 1.372617ms

All of those issues were transferred and so they should be skipped in the task, but aren't, and GopherBot performs a GitHub API call to set the milestone in a loop without ever succeeding. This contributes to a significant increase of GitHub API rate limit quota usage (see related issues #28320 and #44406).

Since GopherBot uses maintner data to iterate over issues, issue #37370 is relevant.

To avoid depleting the GitHub quota way too quickly, we'll unfortunately need to disable the new "set vscode-go milestones" task until this is resolved.

CC @golang/release, @stamblerre, @hyangah.

@dmitshur dmitshur added help wanted Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Apr 9, 2021
@dmitshur dmitshur added this to the Unreleased milestone Apr 9, 2021
@gopherbot
Copy link

Change https://golang.org/cl/308789 mentions this issue: cmd/gopherbot: disable "set vscode-go milestones" task

gopherbot pushed a commit to golang/build that referenced this issue Apr 9, 2021
The maintner data model doesn't expose when an issue gets transferred
from one repo to another (that's golang.org/issue/37370), so GopherBot
currently doesn't skip transferred issues when processing them.

There turns out to be many transferred issues in the vscode-go repo,
causing gopherbot to take too many repetitive actions as reported in
golang.org/issue/45461. Skip the task by default until this is handled
better to avoid depleting the GitHub quota way too quickly.

For golang/go#45461.

Change-Id: I404c94b342f9a870f159d3cadef6cfa2a9979d2d
Reviewed-on: https://go-review.googlesource.com/c/build/+/308789
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
@dmitshur
Copy link
Contributor Author

dmitshur commented Apr 9, 2021

Closing as duplicate of #40640.

@gopherbot
Copy link

Change https://go.dev/cl/397234 mentions this issue: maintner: mark an issue as NotExist when its comments are gone

gopherbot pushed a commit to golang/build that referenced this issue Apr 1, 2022
When synchronizing comments of an issue, getting a 404 (or 410) from
GitHub REST API's list comments endpoint should happen if and only if
the issue no longer exists. Use that as a safe opportunity to append
a mutation to the log that updates the issue's NotExist field to true.

Maintner doesn't yet have complete support for transferred issues,
a feature that GitHub added after its initial design. It's been
difficult to add it retroactively as described in go.dev/issue/37603.

This change aims to make a small incremental change that we can feel
confident about applying and watching closely, and doesn't try to
implement a complete fix. It should help get maintner unstuck when
trying to sync comments in an issue tracker that has a transferred
issue, and incurs a minimal risk of writing bad mutations.

This CL is based on past work in CL 161521 and CL 176638,
but is smaller in scope intentionally.

For golang/go#37370.
For golang/go#40640.
Updates golang/go#45461.
Fixes golang/go#52017.

Change-Id: I3702d3c7b2e412cf75809fdb722c0ee693c8600e
Reviewed-on: https://go-review.googlesource.com/c/build/+/397234
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
@golang golang locked and limited conversation to collaborators Apr 1, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

2 participants