x/tools/gopls: bad performance with multiple workspace folders #40295

bensussman · 2020-07-17T19:48:31Z

Describe what you observed.

GPLS uses 200% of my CPU (both cores) indefinitely whenever I rebuild my golang in my terminal. The only way to make my computer responsive again is to force-quit gopls. Importantly, the usability of gopls (go-to-definition in VSCode for example) is broken until I force-quit the process enough times such that the CPU goes down. Only then does it actually work.

Please attach the stack trace from the crash.
A window with the error message should have popped up in the lower half of your screen.
Please copy the stack trace from that window and paste it in this issue.

[Info  - 3:45:30 PM] 2020/07/17 15:45:30 go/packages.Load
	snapshot=0
	directory=/Users/ben/spell/src/spell
	query=[./... builtin]
	packages=191

[Info  - 3:45:36 PM] 2020/07/17 15:45:36 go/packages.Load
	snapshot=6
	directory=/Users/ben/spell
	query=[./]
	packages=1

[Info  - 3:45:37 PM] 2020/07/17 15:45:37 go/packages.Load
	snapshot=6
	directory=/Users/ben/spell
	query=[file=/Users/ben/spell/docker/runs/save/main.go file=/Users/ben/spell/src/spell/constants/paths/paths.go file=/Users/ben/spell/src/spell/db/run.go file=/Users/ben/spell/src/spell/machine/remote_operations.go file=/Users/ben/spell/src/spell/runorchestrator/worker.go file=/Users/ben/spell/src/spell/servers/api/resources/ls.go]
	packages=1

[Info  - 3:45:37 PM] 2020/07/17 15:45:37 go/packages.Load
	snapshot=6
	package_path="command-line-arguments"
	files=[/Users/ben/spell/docker/runs/save/main.go]

[Error - 3:45:37 PM] 2020/07/17 15:45:37 no dep handle: no metadata for gocloud.dev/blob
	package="gocloud.dev/blob"

[Error - 3:45:37 PM] 2020/07/17 15:45:37 no dep handle: no metadata for gocloud.dev/blob/s3blob
	package="gocloud.dev/blob/s3blob"

[Error - 3:45:37 PM] 2020/07/17 15:45:37 no dep handle: no metadata for gocloud.dev/blob/gcsblob
	package="gocloud.dev/blob/gcsblob"

[Error - 3:45:37 PM] 2020/07/17 15:45:37 no dep handle: no metadata for github.com/fluent/fluent-logger-golang/fluent
	package="github.com/fluent/fluent-logger-golang/fluent"

[Error - 3:45:37 PM] 2020/07/17 15:45:37 no dep handle: no metadata for github.com/aws/aws-sdk-go/aws/awserr
	package="github.com/aws/aws-sdk-go/aws/awserr"

[Error - 3:45:40 PM] Connection to server got closed. Server will not be restarted.

The text was updated successfully, but these errors were encountered:

stamblerre · 2020-07-17T21:35:00Z

It looks like you may opening a directory that gopls doesn't recognize as being inside of a module or your GOPATH. If you're using modules, make sure to open VS Code at the root of your module (instead of a parent directory) - this is a current limitation that we're working on addressing.

More verbose logs would be helpful here - details on how to capture them here.

bensussman · 2020-07-17T22:13:50Z

Thanks @stamblerre

feel free to close this, i just clicked the "report" button and let it do its thing.

For the record, i have a large code base that looks kinda like this:

/code
  /python
  /js
  /golang
    /github.com
    /golang.org
    /myapp
       <my golang app code lives in here>

I have a single VSCode project open to the root of /code AND i did "Add Folder To Workspace" of /code/golang/myapp such that in my sidebar i have code and myapp as two "folders in the workspace". This was necessary to get gopls to work (i believe for the exact reason you described, such that the GOPATH is the root of the "workspace folder") but it feels like it barely works: it's constantly crashing, going super slowly, reporting errors. I frequently kill gopls in task manager, and reboot VSCode in order to get performance back (temporarily).

I wonder if there is a superior way to have my multi-language project open in a single VS code window (so i can easily cmd+p find a file or cmd+shift+f search through all my files) while using tools like gopls that need the root of the project to be the base of the golang project (and not the root of all my code).

stamblerre · 2020-07-17T22:23:30Z

Does the myapp directory contain a go.mod file or is /code/golang/myapp your GOPATH? If it's your GOPATH, you may have better luck adding /code/golang/myapp/src as your workspace folder - we avoid loading all of the packages under a GOPATH to avoid using too much CPU/memory.

If you can share the more details logs (https://github.com/golang/tools/blob/master/gopls/doc/troubleshooting.md#capturing-logs), I'll probably be able to identify the issue a bit quicker.

bensussman · 2020-07-17T22:47:38Z

yes the myapp directory contains go.mod and go.sum. I don't think i have a GOPATH set at all (as i was under the impression this was a legacy concept?)

The list i posted above are not the actual names, i renamed them for ease of conversation, but it seems like src is a standard name? Here are the actual names:

~/spell
  /python
  /js
  /src
    /github.com
    /golang.org
    /spell
       go.mod
       go.sum
       ....
       <more golang app code lives in here>

Here are the gopls (server) logs:

[Info  - 6:16:09 PM] 2020/07/17 18:16:09 Build info
----------
golang.org/x/tools/gopls 0.4.3
    golang.org/x/tools/gopls@v0.4.3 h1:irz7Q+XdHNECamFKbNWKvMV2Ak6zBbwdwbZndG4545I=
    github.com/BurntSushi/toml@v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
    github.com/sergi/go-diff@v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
    golang.org/x/mod@v0.2.0 h1:KU7oHjnv3XNWfa5COkzUifxZmxp1TyI7ImMXqFxLwvQ=
    golang.org/x/sync@v0.0.0-20190911185100-cd5d95a43a6e h1:vcxGaoTs7kV8m5Np9uUNQin4BrLOthgV7252N8V+FwY=
    golang.org/x/tools@v0.0.0-20200708181441-6004c8539734 h1:Vc0Vx98oU/O3+qPQ36fnTT5UduS55KLh2uSGbL7mqEo=
    golang.org/x/xerrors@v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
    honnef.co/go/tools@v0.0.1-2020.1.4 h1:UoveltGrhghAA7ePc+e+QYDHXrBps2PqFZiHkGR/xK8=
    mvdan.cc/xurls/v2@v2.2.0 h1:NSZPykBXJFCetGZykLAxaL6SIpvbVy/UFEniIfHAa8A=

Go info
-------
go version go1.13.6 darwin/amd64



[Info  - 6:16:09 PM] 2020/07/17 18:16:09 go/packages.Load
	snapshot=0
	directory=/Users/ben/spell
	query=[./ builtin]
	packages=2

[Info  - 6:16:09 PM] 2020/07/17 18:16:09 go env for /Users/ben/spell
(valid build configuration = false)
(build flags: [])
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ben/Library/Caches/go-build"
GOENV="/Users/ben/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/ben/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.6/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.6/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/68/c2qg0pm111l1m549k_dntx080000gn/T/go-build312012540=/tmp/go-build -gno-record-gcc-switches -fno-common"


[Info  - 6:16:09 PM] 2020/07/17 18:16:09 go/packages.Load
	snapshot=0
	directory=/Users/ben/spell
	query=[./]
	packages=1

[Info  - 6:16:10 PM] 2020/07/17 18:16:10 go env for /Users/ben/spell/src/spell
(valid build configuration = true)
(build flags: [])
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ben/Library/Caches/go-build"
GOENV="/Users/ben/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/ben/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.6/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.6/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/ben/spell/src/spell/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/68/c2qg0pm111l1m549k_dntx080000gn/T/go-build272018563=/tmp/go-build -gno-record-gcc-switches -fno-common"


[Info  - 6:16:16 PM] 2020/07/17 18:16:15 go/packages.Load
	snapshot=0
	directory=/Users/ben/spell/src/spell
	query=[./... builtin]
	packages=191

[Info  - 6:16:52 PM] 2020/07/17 18:16:52 background imports cache refresh starting

[Info  - 6:16:59 PM] 2020/07/17 18:16:59 background refresh finished after 6.639360654s

[Error - 6:42:11 PM] Request textDocument/foldingRange failed.
  Message: invalid pos
  Code: 0 
[Error - 6:42:11 PM] Request textDocument/codeAction failed.
  Message: computing fix edits: /Users/ben/spell/src/spell/runorchestrator/worker.go:943:121: missing ',' before newline in argument list (and 1265 more errors)
  Code: 0 
[Error - 6:42:52 PM] Request textDocument/foldingRange failed.
  Message: no parsed file for file:///Users/ben/spell/src/spell/runorchestrator/worker.go
  Code: 0

EDIT: looks like i do have a GOPATH at ~/go/ TBH i'd like to delete that, as ideally i have all my golang code for this project be colocated in the ~/spell/src directory (thats what i thought was happening with the ~/spell/src/github.com dir for example). Unclear to me if i can just remove it, or if it's helping / hurting.

stamblerre · 2020-07-19T17:25:00Z

EDIT: looks like i do have a GOPATH at ~/go/ TBH i'd like to delete that, as ideally i have all my golang code for this project be colocated in the ~/spell/src directory (thats what i thought was happening with the ~/spell/src/github.com dir for example). Unclear to me if i can just remove it, or if it's helping / hurting.

Don't worry about that - it's the default value, and it's having no effect if you're using modules.

I wonder if the high CPU usage is caused by the fact that you have a multiple workspaces. Does gopls work well if you only open the directory containing your module?

bensussman · 2020-07-20T16:04:26Z

Yes @stamblerre it does seem like closing out my multi-workspace-folder VSCode instance, and instead opening just the directory that contains my golang source code (in my case ~/spell/src/spell, where my go.mod lives) has significantly improved performance and CPU utilization.

This is not a great solution for me, because it means that I have to have at least two windows open:

with Golang code dir
with everything else (and this has to be a multi-directory workspace because python and js are siblings).

I think an ideal fix would be to explicitly enable or disable golang/gopls within each directory. That way i could have my one-giant-workspace, but gopls would only be enabled for the one golang directory and not the parent.

stamblerre · 2020-07-21T01:57:51Z

Glad to hear it's working better. I agree it's not an ideal solution, and our plan is to address this shortcoming of gopls for the v1.0 release (#32394). Our plan is to make sure that gopls works without needing to add your module as a workspace folder, so I think that will resolve this issue. Unfortunately, you may need to use this workaround for the time being.

stamblerre · 2020-07-21T23:29:06Z

Closing this, as it will be addressed with the more fundamental shifts in v1.0.

bensussman changed the title ~~gopls: automated issue report (crash)~~ gopls: high cpu and bad performance on large project with non-golang code located in adjacent directory and workspace including all code Jul 17, 2020

stamblerre transferred this issue from golang/vscode-go Jul 19, 2020

stamblerre changed the title ~~gopls: high cpu and bad performance on large project with non-golang code located in adjacent directory and workspace including all code~~ x/tools/gopls: bad performance with multiple workspace folders Jul 19, 2020

gopherbot added gopls Issues related to the Go language server, gopls. Tools This label describes issues relating to any tools in the x/tools repository. labels Jul 19, 2020

gopherbot added this to the Unreleased milestone Jul 19, 2020

stamblerre modified the milestones: Unreleased, gopls/v0.5.0 Jul 20, 2020

stamblerre closed this as completed Jul 21, 2020

stamblerre modified the milestones: gopls/v0.5.0, gopls/v0.4.4 Jul 22, 2020

golang locked and limited conversation to collaborators Jul 22, 2021

gopherbot added the FrozenDueToAge label Jul 22, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

x/tools/gopls: bad performance with multiple workspace folders #40295

x/tools/gopls: bad performance with multiple workspace folders #40295

bensussman commented Jul 17, 2020

stamblerre commented Jul 17, 2020

bensussman commented Jul 17, 2020

stamblerre commented Jul 17, 2020

bensussman commented Jul 17, 2020 •

edited

stamblerre commented Jul 19, 2020

bensussman commented Jul 20, 2020 •

edited

stamblerre commented Jul 21, 2020

stamblerre commented Jul 21, 2020

x/tools/gopls: bad performance with multiple workspace folders #40295

x/tools/gopls: bad performance with multiple workspace folders #40295

Comments

bensussman commented Jul 17, 2020

stamblerre commented Jul 17, 2020

bensussman commented Jul 17, 2020

stamblerre commented Jul 17, 2020

bensussman commented Jul 17, 2020 • edited

stamblerre commented Jul 19, 2020

bensussman commented Jul 20, 2020 • edited

stamblerre commented Jul 21, 2020

stamblerre commented Jul 21, 2020

bensussman commented Jul 17, 2020 •

edited

bensussman commented Jul 20, 2020 •

edited