Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: excessive idle CPU usage #61049

Closed
adonovan opened this issue Jun 28, 2023 · 3 comments
Closed

x/tools/gopls: excessive idle CPU usage #61049

adonovan opened this issue Jun 28, 2023 · 3 comments
Assignees
Labels
gopls/performance Issues related to gopls performance (CPU, memory, etc). gopls Issues related to the Go language server, gopls. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone

Comments

@adonovan
Copy link
Member

adonovan commented Jun 28, 2023

gopls v0.12 uses too much CPU while idle: approximately 15% of a core on average, according to both top(1) and pprof (although the latter is unreliable due to a macOS kernel bug in setitimer). Based on the profile, it appears to be because the filecache gc calls filepath.Walk every minute. On my machine, with a cache of 20,000 directories and 355,000 files, it takes /usr/bin/find 1.7s of CPU to produce 44MB of filenames, and filepath.Walk probably costs some multiple of this.

We should reduce the cost of gc by some combination of
(a) doing it less often, especially since there's no longer any need to try to keep with the high rate of garbage production in tests.
(b) doing it at variable intervals, using some kind of backoff scheme if little was collected.
(c) doing it more efficiently, either by allocating less memory and fewer stat objects than filepath.Walk.
(d) doing it more intelligently, by building a stateful representation of the directory tree and only doing the I/O necessary to update it incrementally.
(e) doing it in proportion to the rate of garbage creation (e.g. using a leaky bucket fed by calls to Set).

@bcmills

@adonovan adonovan self-assigned this Jun 28, 2023
@gopherbot gopherbot added Tools This label describes issues relating to any tools in the x/tools repository. gopls Issues related to the Go language server, gopls. labels Jun 28, 2023
@gopherbot gopherbot added this to the Unreleased milestone Jun 28, 2023
@bcmills bcmills added ToolSpeed gopls/performance Issues related to gopls performance (CPU, memory, etc). and removed ToolSpeed labels Jun 28, 2023
@bcmills
Copy link
Contributor

bcmills commented Jun 28, 2023

doing it at variable intervals

FWIW, I still think it would be nice to gate the rate of cache cleaning based on the rate of cache creation.

(For example, the cleaner goroutine could do one pass at startup and then block until some operation creates a new file in the cache, at which point it could then unblock for another pass.)

@gopherbot
Copy link

Change https://go.dev/cl/506815 mentions this issue: gopls/internal/lsp/filecache: reduce GC frequency

gopherbot pushed a commit to golang/tools that referenced this issue Jun 28, 2023
This is a mitigation for observed high idle CPU usage.

Also, eliminate the obsolete special case for builders,
now that each CI run gets a fresh GOPLSCACHE.

Updates golang/go#61049

Change-Id: I8689ab9ebf4d9ef75d9e1a0f4bc7bc9d109bef71
Reviewed-on: https://go-review.googlesource.com/c/tools/+/506815
TryBot-Result: Gopher Robot <gobot@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Alan Donovan <adonovan@google.com>
@gopherbot
Copy link

Change https://go.dev/cl/506816 mentions this issue: gopls/internal/lsp/filecache: increase idle GC period to 6 hours

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gopls/performance Issues related to gopls performance (CPU, memory, etc). gopls Issues related to the Go language server, gopls. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Development

No branches or pull requests

5 participants