Background

Gopls is an incredible tool. Go programmers would truly not be as productive without it.

Gopls fully relies on go/packages to load and parse Go files. However, go/packages is known to be quite slow.

Therefore, the biggest benefit of gopls is arguably its caching of parsed Go files and their type system results that came from go/packages.

Once you have a bird's eye view of an Editor's workspace, the amount of features you can implement to make Go programmers productive are quite limitless. Gopls has a lot of features from code navigation, to code generation, and more.

However, it is practically impossible to fit all the features a programmer wants into gopls. The more features gopls adds, the more unmaintainable it becomes, the slower it gets, and the larger its codebase and resulting binary grow.

Therefore, there is a strong argument for gopls to support the minimal feature for it to be useful out of the box.

But that leaves huge swaths of incredible features that people can want from gopls that are still not worthy of putting into the Gopls codebase.

Below, I've listed a number of such use cases that I've personally run into. I imagine there are a lot more.

Problem

There are already many popular libraries in the community that leverage go/packages to do all sorts of work. To quickly do a reverse-lookup of those libraries, https://pkg.go.dev shows that go/packages is imported-by over 1000 packages in the wild not counting closed source Go code.

However, all of those packages suffer from the same problem that gopls has already solved: go/packages is too slow.

Projects that use multiple code generation libraries no longer enjoy the same speed typical Go programmers enjoy due to Go's fast compiler.

Anecdotally speaking, one company I interviewed with told me that their build system takes at least 10 seconds to generate all of their code and compile it.

Therefore, there is a strong need for such libraries to leverage the gopls cache without having to implement their features directly in the gopls codebase.

Proposal

I think the need for Gopls to be extensible is vital to the Go community.

Most tools out there write their code analysis tools in terms of go/packages but I'd love for many of these tools to take advantage of the already cached modules in their gopls session.

There are several ways to accomplish this task (I listed some ideas below) but I am indifferent with the implementation details. Because as a Go programmer, I have run into numerous scenarios where I wish gopls was extensible in any sort of way.

Use Cases

Code Generation

1. Ent: the Ent library uses Go code to define a SQL database schema and generates a full ORM with query and insert features that has become quite popular. The library uses go/packages to analyze your schema code and creates a number of packages that expose Go functions that can talk to a SQL server. As your schema grows, the time it takes to generate your code grows more.
2. gqlgen: gqlgen uses a GraphQL schema coupled with a config file to generate an entire Go GraphQL Server. The config file allows the code generator to know when to create a Go struct that maps to a GraphQL Object, or when to re-use an existing struct. Gqlgen heavily uses go/packages to analyze and generate code and can take many seconds.
3. entgql: interestingly, there is a library that combines both of the libraries above and you can imagine how slow that becomes.
4. Protocol Buffers: there are protobuf plugins out there like protoc-gen-twirpql that uses go/packages and could definitely leverage the gopls cache to generate its code. It's also worth noting that this is a more interesting case than the above because protobuf plugins are usually invoked from the protoc compiler so a plugin would have to somehow know how to latch on to a gopls session and not the other way around.
5. Google's Wire: just another popular example out there that uses go/packages for AST manipulation and code generation where it could be made a lot faster if it leveraged the gopls cache.

AutoComplete

Error Handling

Many projects has repeatable patterns for how they decorate and return errors in their functions. For example, Athens uses the Upspin Error Handling Pattern, CLIs just use fmt.Errorf("funcName: %w", err), pkgsite uses defers, and much more.

Gopls already has a great auto-complete snippet suggestion that just returns the error such as:

if err != nil {
  return <zeroValues...>, err
}

But an extension could be smarter and go beond just returning the error, such as:

if err != nil {
  return <zeroValues...>, errors.E(op, err) // Athens pattern
  // or
  return <zeroValues...>, fmt.Errorf("<enclosingFunc>: %w", err) // Go1.13 error wrapping pattern.
  // or
  return <zeroValues...>, errors.Wrap(err, "<enclosingFunc") // github.com/pkg/errors pattern.
}

Or in the case of http handlers, users can implement their own way of handling an error inside of a HandlerFunc. See this comment where it explicitly says:

We have been fairly hesitant about adding snippets, since it might seem like we're prescribing a "correct" or "Go team approved" way of doing things.

This is a perfect use case for extension so that gopls doesn't implement a single correct way, but projects can have their own snippet completions.

Snippet Completions

Besides error snippet completions, many companies and teams can have templates as starter points for their servers.

For example, Uber has fx, while the New York Times has Gizmo and they follow specific patterns. A company can publish an extension that could create snippets for starting their own framework as well as other LSP features.

Other Use Cases

1. Many companies have internal libraries and gopls could extend its Go: Add Import feature where a company can distribute an extension that lists such private modules as options to add to your go.mod or Go file.
2. Code Analyzers: Gopls already has a number of analyzers some of which are maintained by open source contributors. Some programmers don't use all of the analyzers. Therefore, it might make sense to pull some of them out into extensions that the open source maintainers can contribute to them directly.
3. Experimental features: many experimental features can be shipped as an extension first while they await proper review. Users can then give feedback and report errors so that it is more robust by the time it lands into upstream. A good example would be "the stub method" feature, but I'm sure there are many more.

Possible Implementations

There are several ways that can make gopls extensible:

1. Serialize the gopls cache so that other tools can quickly deserialize and load it.
2. Dynamically load extended features using the plugin library (which I have heard a few times that it is unstable and a bad idea)
3. Statically load extended features and compile them with gopls together. This makes installing gopls with extensions a bit more complex but avoids the need for serialization/deserialization or the use of the plugin library.

Difficulty

I completely understand that this task is nowhere near trivial.

The first thing that strikes me is the fact that the entirety of gopls is under an internal package path which makes it impossible for other codebases to re-use any of its type definitions and functions outside of the tools repository. Therefore, the first thing that gopls will need to do is export a lot of its types and functions so that extensiosn can use them. Perhaps just renaming internal to experimental is good enough as a start so that it doesn't block the progress of this task, until the gopls team is happy with the results.

Another thing that requires a lot of thinking is how gopls recognizes extensions and communicates with them to share its memory objects (Snapshots, Views, etc). I suggested some ideas above and I hope there are more ideas out there. But it is also not a trivial task to land at the perfect solution. I hope this issue can be a place to discuss those ideas and their tradeoffs.