Nothing in the entire Go codebase uses dbus. You have some exotic setup involving a wrapper for the go command, an LD_PRELOAD, or a gopackagesdriver.

You have some exotic setup involving a wrapper for the go command, an LD_PRELOAD, or a gopackagesdriver.

What makes you think this? I don't actually have any of the above. Per the log file, I've simply opened Vim in my home directory.

FYI I bisected this to 78fed78

Sorry, I assumed that this was an error from the Go command. We probably need better error messages in this area. I assume you've enabled ExperimentalWorkspaceModule?

@stamblerre, I guess this is a question of whether we should be ignoring errors while walking directories looking for go.mod files.

I assume you've enabled ExperimentalWorkspaceModule?

No I haven't

Change https://golang.org/cl/256578 mentions this issue: internal/lsp: ignore errors when finding workspace modules

Ah, I specifically changed it so that we would always look for workspace modules even without ExperimentalWorkspaceModule so that we could catch any bad behavior in that code :) It sounds like we should be ignoring errors--mailed a CL.

I don't think that's good practice -- this code is known to be at least somewhat unstable, and if it's not guarded behind a flag we run the risk of breaking users the next time we release. We got lucky that Paul noticed this first.

I don't think that's good practice

I'd tend to agree.

Is the plan with this new workspace module, when enabled, to recursively scan the directory in which the editor/gopls is opened?

Previous discussion of a related topic/issue: #35818

I don't think that's good practice

I'd tend to agree.

While it's true that users shouldn't see issues caused by experimental features while working the default mode, I'm not too concerned about this particular case. This seems like something we wouldn't have tested for, so it's really a matter of catching it now versus catching it before we enable multi-module workspaces by default. I do agree that we're lucky Paul noticed -- thank you for reporting!

Is the plan with this new workspace module, when enabled, to recursively scan the directory in which the editor/gopls is opened?

Yep--we are now walking the workspace root looking for go.mod files.

Yep--we are now walking the workspace root looking for go.mod files.

Is this approach going to scale when, as I did, someone opens an editor in their home directory?

I guess I'm not totally clear on the benefits of such a scan, vs adding modules when files from a module are opened in gopls:

• on my machine a find $HOME -name go.mod takes ~2 mins from cold
• newly created modules will not be automatically discovered unless we rescan or manually add them

I'm sorry, the issue got closed when I merged the CL. I'll reopen it and link it to the multi-module workspace umbrella issue (#32394), since this sounds more like a fundamental discussion about the multi-module workspace design.

To answer your question--the idea behind the scan is to support features like diagnostics and find-references, etc. across multiple or nested modules automatically. It may be a good idea to consider factoring in "depth" however--you're right that someone who opens their home directory would probably not want an enormous multi-module workspace. We could potentially incrementally add modules to the workspace as files are opened, but that does open up the possibility of non-deterministic results for features like find references. And monorepos might have many modules nested below a top-level directory, so distinguishing these cases would be difficult. As of right now, opening your $HOME directory in the multi-module workspace world is the equivalent of opening your entire $GOPATH--use at your own risk.

What do you think, @heschik?

I don't think we've spent enough time designing the UX for multi-module mode, so I don't think I have enough groundwork to give a good answer. I don't think a hardcoded limit either on depth or module count is the answer though. My first thought is that we should ask the user what they want at startup, but that could be annoying.

I believe that the previous behavior would have been for the folder to be detected as a misconfigured workspace, and then we'd do a Load on . rather than ./.... Why do we do that rather than failing to initialize? What use case does that serve, that this would presumably break?

I believe that the previous behavior would have been for the folder to be detected as a misconfigured workspace, and then we'd do a Load on . rather than ./.... Why do we do that rather than failing to initialize? What use case does that serve, that this would presumably break?

I think the goal of that was to spare the user the cost of an initial workspace load on every module beneath their $HOME directory. Asking the user what they want would be reasonable if we could could cache their answer on disk, but even then, a user might change their mind later on, so we'd need to have the UX for them to switch. Maybe if we discover some very large number of modules (10+?), we could warn the user that we're about to load them before we do so?

Change https://golang.org/cl/257137 mentions this issue: internal/lsp: don't search for workspace modules by default

My first thought is that we should ask the user what they want at startup

This feels like a sensible starting point.

Maybe if we discover some very large number of modules (10+?), we could warn the user that we're about to load them before we do so?

Isn't the issue that with such an approach you will load a totally arbitrary selection of the modules beneath the working directory that may or may not (likely not) coincide with what the user wants?

Asking the user what they want would be reasonable if we could could cache their answer on disk, but even then, a user might change their mind later on, so we'd need to have the UX for them to switch

To my mind, this goes all the way back to the conversation we had a GopherCon last year, the UI/UX around workspace configuration in an editor. Because simply having a module open does not imply (to my understanding) that the user does want it to be part of the multi module magic, hence I guess you reference to "have the UX for them to switch"?

Change https://golang.org/cl/268877 mentions this issue: internal/lsp: nest the workspace root if there is only one module

After enabling ExperimentalWorkspaceModule last week, I've noticed gopls got considerably slower, even when working on a project/workspace/directory tree with only 3 go.mod files.

It takes several seconds to update type info on files I'm working on, let alone provide code completion, and that's on a modern powerful laptop. The editor feedback is so slow that I'm going to disable the ExperimentalWorkspaceModule flag for now.

Would be happy to share debugging data it that helps.

Sorry to hear that, @rhcarvalho. If the projects you are working on are public, or if you can share a log, we can take a look and try to understand the issue.

@stamblerre thanks and sorry for I haven't gotten back to you earlier. I've just upgraded to v0.5.4 and decided to re-enable multi-module to give it another try. The initial 5min impression is things got better, thank you!

The latest version of gopls (db4c57d) now shows the following error message when I open govim in my home directory, opening a text file outside of the context of a Go module:

Error loading workspace folders (expected 1, got 0)    
failed to load view for file:///home/myitcv: exhausted

@stamblerre - is the latest thinking #41558 (comment)?

As I said above, I don't think we should be doing a scan in the first place when there is no trigger to do so. Such triggers to my mind would be opening a .go or go.{mod,sum} files, or when the file opened is within a Go module (although this case is somewhat debatable)

Nor do I think we should be suggesting the user not open that directory again or recommend they create a gopls.mod. At least in my case, both are incorrect suggestions.

As I said above, I don't think we should be doing a scan in the first place when there is no trigger to do so. Such triggers to my mind would be opening a .go or go.{mod,sum} files, or when the file opened is within a Go module (although this case is somewhat debatable)

I think this is a result of the fundamental difference between govim and plugins like VS Code Go. VS Code Go only starts gopls when there are Go files in the workspace, so gopls initializes, assuming that it should be ready to start handling requests. I know that govim initializes gopls no matter what the workspace is, and it's also fairly uncommon for users to open VS Code to their home directory.

Maybe we can consider some alternatives, like waiting to initialize if there are no open Go files and the Go code is only in subdirectories, but that would be tricky to get right. Right now, gopls is set up to just initialize whenever it's started, which makes sense given that if it was started, it is probably going to be in use. Could govim not start gopls until a Go file is opened?

Could govim not start gopls until a Go file is opened?

We could yes, but...

Maybe we can consider some alternatives, like waiting to initialize if there are no open Go files and the Go code is only in subdirectories, but that would be tricky to get right.

My point is that if it's difficult for gopls to get right, it's also difficult for govim to get right. And therefore that logic would be better placed in gopls, because that will have the side effect of then being consistent and correct (as we can make it) for all clients.

From our side we'll look at what we can do in any case.

As an example of an edge case that either govim or gopls would need to solve, I very often open Vim in a module context (i.e. main module is defined for the working directory) but with no file and immediately reach for workspace symbol search before any files are open. This would require logic to determine that we are being loaded in a module context. Again, not impossible, but I don't think this logic belongs in the LSP client.

> Error loading workspace folders (expected 1, got 0)    
> failed to load view for file:///home/myitcv: exhausted

Hi, I met the same error message with VSCode/gopls. gopls is not working correctly anymore.

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/thomaspeugeot/Library/Caches/go-build"
GOENV="/Users/thomaspeugeot/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/thomaspeugeot/go/pkg/mod"
GONOPROXY="github.com/thomaspeugeot"
GONOSUMDB="github.com/thomaspeugeot"
GOOS="darwin"
GOPATH="/Users/thomaspeugeot/go"
GOPRIVATE="github.com/thomaspeugeot"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/thomaspeugeot/go/src/github.com/thomaspeugeot/metabaron/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/_r/bxjvdzc17mb6p56qc7nms1r80000gn/T/go-build546449124=/tmp/go-build -gno-record-gcc-switches -fno-common"

go version go1.15.7 darwin/amd64

Any idea how to proceed in order to have gopls working again ? Is there any limit/budget to rise for gopls to work ?

thks in advance

@thomaspeugeot: Can you please file a new issue with your gopls logs attached? Information on how to provide this information can be found here.

Another datapoint in this investigation is that the package load does not appear to be interruptible. So for example if I open govim in my home directory, but then quickly exit Vim, the Vim exit is blocked whilst a call to Shutdown is blocked on the "loading packages" step to complete/exhaust itself.

@findleyr kindly shared some thoughts on the old question of "when should govim start gopls?"

The current model is that govim always starts gopls. This is the source of a number of issues, because gopls eagerly finds and then loads workspaces (per above). My thinking here has been that gopls could (should?) instead do things lazily, only reacting to Go-related file opens, or specific LSP method calls. But I can very much see there is merit in doing things eagerly. And a key point Rob made is that we need/want the Vim user to be in control of when gopls is started regardless.

The conclusion to our conversation: it makes sense to switch to a model of govim only loading gopls when it encounters a Go-related file, with the option of govim loading gopls eagerly if Vim is started with a working directory that is within a Go module context (i.e. go env GOMOD is not empty and not equal to /dev/null).

Thank you to @stamblerre, @heschi, @findleyr and others for your patience as I have slowly reached this position 😄

Hi, I'm just going through old issues, and I'm not sure what to do with this one.

The problem of configuring a workspace has moved to go.work, and is covered by (in-progress) gopls support for go.work files with 1.18.

The problem of general scalability is covered by (many) other issues.

This issue seems to be about finding a workspace that gopls can handle without blowing up. With 1.18, the way to do that will be to configure go.work accordingly. In the future, we hope that gopls can handle much larger workspaces than it currently handles.

So, I'm closing this issue as having run its course. Please reopen if I missed something.