Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/cmd/gopls: Return Newly Added Modules in KnownPackages #48545

Open
marwan-at-work opened this issue Sep 22, 2021 · 4 comments
Open

x/tools/cmd/gopls: Return Newly Added Modules in KnownPackages #48545

marwan-at-work opened this issue Sep 22, 2021 · 4 comments
Labels
gopls Issues related to the Go language server, gopls. Tools This label describes issues relating to any tools in the x/tools repository.

Comments

@marwan-at-work
Copy link
Contributor

KnownPackages is a gopls command that returns a list of all packages a workspace file can add to its "import" statement.

It is currently used by VSCode's "Add Import" functionality.

However, if you had just ran a "go get" command on a new module, that new module's packages will not show up. This is because it is a "newly added module", meaning it is recorded in your go.mod file but none of your Go files import a package within this module just yet. This is usually the typical time you'd want to list the available packages to add one to your import statement.

The reason for KnownPackages not returning newly added module packages is that the implementation of KnownPackages does two things:

  1. It considers all import paths that are cached in its session (which obviously wouldn't yet have the newly added module)
  2. It randomly searches all of your GOMODCACHE through the GetAllCandidates function but only limits itself to 80 milliseconds and returns the partial results. This is because it would take too long to search all of your GOMODCACHE directory. Therefore it is a hit or miss and often returns completely irrelevant results.

Proposal

I propose that we change the above implementation to something more guaranteed and would also fix the "newly added module" issue above:

When a client asks for KnownPackages relative to a file, we simply get the go.mod that belongs to that file by calling snapshot.GoModForFile then parse the file to get its dependencies. From there, we can simple loop over the "require" statements and find their location and return the valid packages within each module@version.

This achieves a few things:

  1. It only take into consideration your newly added module and show its packages.
  2. It should be quite fast because we don't run any "go list" or search all of the GOMODCACHE, but only what is in your go.mod file. Also worth noting that "go list" analyzes your import paths (IIRC) which means it would skip any newly added modules.
  3. With Go1.17 in your go.mod file, we know all of your dependencies whether direct or indirect are recorded there, so we'll always return exactly every single package that is truly importable within your module. For versions less than that, we can either keep the old behavior or apply the new behavior as arguably you don't usually import indirect packages that you may not even know is in your transitive dependencies.

Alternatives:

  1. Running "go list" from the root go.mod directory (will not pick up newly added modules)

Edge cases:

  1. This of course would not work in GOPATH mode, but in that case we can just default to the old behavior or just show the std lib.
  2. In neither GOPATH/Module modes, we can just show the standard library packages.
@gopherbot gopherbot added Tools This label describes issues relating to any tools in the x/tools repository. gopls Issues related to the Go language server, gopls. labels Sep 22, 2021
@gopherbot gopherbot added this to the Unreleased milestone Sep 22, 2021
@suzmue suzmue modified the milestones: Unreleased, gopls/unplanned Sep 22, 2021
@gopherbot
Copy link

Change https://golang.org/cl/351650 mentions this issue: internal/lsp/source: Deduce KnownPackages from the go.mod file

marwan-at-work added a commit to marwan-at-work/tools that referenced this issue Oct 8, 2021
This CL refactors the KnownPackages command to use the go.mod file
as the source of truth of all the packages we can offer the user to
add to their import path. The reason for this is that newly added modules
are still not part of the cache and are not visible to "go list" because
they are not imported in any Go files in the user's project yet.

Fixes golang/go#48545

Change-Id: Icbaed0a3677d8af3b2d88a531f03c8e083d0ee2b
@pavlelee
Copy link

pavlelee commented Mar 17, 2022

I use gopls 0.8.1, list_known_packages return incomplete package list. I modify 80 milliseconds to 30s will return complete package list

@pavlelee
Copy link

any response? Add imports is a very useful action

@pavlelee
Copy link

go list -f "{{.Name}};{{.ImportPath}}" all is incomplete in go mod, nearly two-thirds less.

go list -f "{{.Name}};{{.ImportPath}}" all
gopkgs -format '{{.Name}};{{.ImportPath}};{{.Dir}}' -workDir ./

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gopls Issues related to the Go language server, gopls. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Development

No branches or pull requests

4 participants