Thanks for reporting! Can you share the full output of gopls -rpc.trace -v check path/to/file.go?

I see two suspect call sites:

https://github.com/golang/tools/blob/abb57c682a60b603ade8571659cde791b971ccf0/internal/lsp/cache/mod.go#L239
https://github.com/golang/tools/blob/cf670267be10c1b9044a9299196b3c9f2b9db125/go/packages/golist_overlay.go#L285

CC @matloob

Thanks for reporting! Can you share the full output of gopls -rpc.trace -v check path/to/file.go?

I'm not able to reproduce it reliably - I'm not quite sure what triggers it, but I will try to get a trace.

Based on the -json and the fact that the misconfiguration error comes up after we try to load the workspace, I would guess that the issue is the go list call in go/packages.

@bcmills: How should go/packages work around this - should we set the -mod=mod flag?

@stamblerre, as far as I can tell that use in go/packages is only used in the implementation of determineRootDirs, and determineRootDirs itself is only used in getPkgPath.

So I would recommend raising the abstraction level up a bit: go list accepts a directory argument, so (barring any remaining bugs) you can invoke go list $DIR to resolve the package path corresponding to filesystem directory $DIR:

~/src/golang.org/x/tools$ go1.13.8 list -f '{{.Dir}}' golang.org/x/xerrors
/usr/local/google/home/bcmills/pkg/mod/golang.org/x/xerrors@v0.0.0-20191011141410-1b5146add898

~/src/golang.org/x/tools$ go1.13.8 list /usr/local/google/home/bcmills/pkg/mod/golang.org/x/xerrors@v0.0.0-20191011141410-1b5146add898
golang.org/x/xerrors

Ah, wait, that doesn't work because getPkgPath is trying to infer the hypothetical package path for a directory that does not (yet) actually contain a Go package.

Thinking about this some more: is there a use-case for overlays outside of the main module? The module cache is generally read-only anyway, so folks shouldn't be adding files there.

I guess you might want to handle overlays for modules substituted in via replace directives..? But that won't work in vendor mode anyway, because the go command won't read the replacement directories: they really don't have a package path.

So that leaves:

1. Directories in the main module's vendor tree. You already know where the main module is, so you just need to detect that $DIR is within a subdirectory of its vendor tree, lop off the prefix, done. (No root search required.)

2. Directories in the main module. go list -m -f '{{.Dir}}' within $DIR to verify that it's actually still in the main module, lop off the prefix containing the main module directory, filepath.ToSlash, and done.

3. Directories in modules for which the main module contains a replace directive. But we currently require a go.mod file for replaced modules, and the replacement module's go.mod file is supposed to match the path that it replaces. So you can go list -m -f {{.Path}} {{.Dir}} within $DIR, then go list -m {{with .Replace}}{{.Dir}}{{end}} that module path from within the main module (to verify that that module really is the replacement for the module), and again lop off the directory prefix and filepath.ToSlash as above.

Thanks for the detailed analysis, @bcmills! I'll leave the decision-making up to @matloob, since this is not an area of go/packages that I'm particularly familiar with, but your suggestions sound good to me.

I'm trying to understand what's going on here. Some questions:

@bcmills, you're asking if there's a use-case for overlays outside the main module because you're trying to figure out if we can get rid of the go list -m call that determines module roots for determining import paths, right? If so, the suggestion is to determine which module each directory is by doing a go list -m call for each of them, right? If so, I'm wondering if the go list -m -f '{{.Dir}} invocation for #​2 will work if the directory doesn't exist.

My second question is why doesn't go list -m all work with -mod=vendor?

you're trying to figure out if we can get rid of the go list -m call that determines module roots for determining import paths, right?

Yep.

If so, the suggestion is to determine which module each directory is by doing a go list -m call for each of them, right?

Correct, with the exception of subdirectories of the main module's vendor directory.

If so, I'm wondering if the go list -m -f '{{.Dir}} invocation for #​2 will work if the directory doesn't exist.

It will not, but it should be ok in general to run go list -m -f '{{.Dir}}' on the deepest parent of the proposed directory that actually does exist.

My second question is why doesn't go list -m all work with -mod=vendor?

vendor/modules.txt only includes two sets of modules:

1. modules that provide packages transitively imported by the main module
2. (as of Go 1.14) modules named explicitly in the main module's go.mod file.

In contrast, at least up through Go 1.14, go list -m all lists all modules transitively required by the main module's go.mod file. In general, that can be an arbitrarily larger superset of the two sets listed in vendor/modules.txt.

That means that the output of go list -m all would vary depending on the -mod flag, and since the default value for the -mod flag changes automatically in Go 1.14, I felt that could be very confusing. So instead, it emits an error, and the user can make an explicit choice to either set -mod=mod explicitly or find a different query that works more consistently with -mod=vendor.

Thanks for the explanation. I'll start by implementing #​2. If we need support for directories in the vendor tree or those that are replaced, we can implement #​1 and #​3 if requested by users. (My intention is to keep the implementation of overlays as simple and restricted as possible.

Is there any temporary workaround for this issue?

@karuppiah7890: I'm sorry, but there's no workaround yet. @matloob should have a fix out soon, however.

Change https://golang.org/cl/227117 mentions this issue: go/packages: only list current module's directory in determineRootDirs

golang.org/cl/227717 should fix it!