Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: import errors when using go.work replace directives #66425

Closed
jonbarrow opened this issue Mar 16, 2024 · 17 comments
Closed

x/tools/gopls: import errors when using go.work replace directives #66425

jonbarrow opened this issue Mar 16, 2024 · 17 comments
Assignees
Labels
gopls Issues related to the Go language server, gopls. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone

Comments

@jonbarrow
Copy link

What version of Go, VS Code & VS Code Go extension are you using?

Version Information
  • Run go version to get version of Go from the VS Code integrated terminal.
    • go version go1.21.3 linux/amd64
  • Run gopls -v version to get version of Gopls from the VS Code integrated terminal.
Build info
----------
golang.org/x/tools/gopls v0.15.2
    golang.org/x/tools/gopls@v0.15.2 h1:4JKt4inO8JaFW3l/Fh9X1k/5JQn+iUOpdc4/Lpi0mOs=
    github.com/BurntSushi/toml@v1.2.1 h1:9F2/+DoOYIOksmaJFPw1tGFy1eDnIJXg+UHjuD8lTak=
    github.com/google/go-cmp@v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
    golang.org/x/exp/typeparams@v0.0.0-20221212164502-fae10dda9338 h1:2O2DON6y3XMJiQRAS1UWU+54aec2uopH3x7MAiqGW6Y=
    golang.org/x/mod@v0.15.0 h1:SernR4v+D55NyBH2QiEQrlBAnj1ECL6AGrA5+dPaMY8=
    golang.org/x/sync@v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
    golang.org/x/telemetry@v0.0.0-20240209200032-7b892fcb8a78 h1:vcVnuftN4J4UKLRcgetjzfU9FjjgXUUYUc3JhFplgV4=
    golang.org/x/text@v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
    golang.org/x/tools@v0.18.1-0.20240311201521-78fbdeb61842 h1:No0LMXYFkp3j4oEsPdtY8LUQz33gu79Rm9DE+izMeGQ=
    golang.org/x/vuln@v1.0.1 h1:KUas02EjQK5LTuIx1OylBQdKKZ9jeugs+HiqO5HormU=
    honnef.co/go/tools@v0.4.6 h1:oFEHCKeID7to/3autwsWfnuv69j3NsfcXbvJKuIcep8=
    mvdan.cc/gofumpt@v0.6.0 h1:G3QvahNDmpD+Aek/bNOLrFR2XC6ZAdo62dZu65gmwGo=
    mvdan.cc/xurls/v2@v2.5.0 h1:lyBNOm8Wo71UknhUs4QTFUNNMyxy2JEIaKKo0RWOh+8=
go: go1.21.3
  • Run code -v or code-insiders -v to get version of VS Code or VS Code Insiders.
1.85.2
8b3775030ed1a69b13e4f4c628c612102e30a681
x64

Though this happens on all tried versions, from 1.80 to 1.87

  • Check your installed extensions to get the version of the VS Code Go extension
    • v0.41.2
  • Run Ctrl+Shift+P (Cmd+Shift+P on Mac OS) > Go: Locate Configured Go Tools command.
# Tools Configuration


## Environment

GOBIN: undefined
toolsGopath: 
gopath: /home/jon/go
GOROOT: /usr/local/go
PATH: /home/jon/.local/share/pnpm:/home/jon/software:/home/jon/gems/bin:/opt/devkitpro/tools/bin:/home/jon/.nvm/versions/node/v21.6.2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/usr/local/go/bin:/opt/ghidra:/home/jon/go/bin:/home/jon/software/grpc/cmake/build

## Tools

	go:	/usr/local/go/bin/go: go version go1.21.3 linux/amd64

	gopls:	/home/jon/go/bin/gopls	(version: v0.15.2 built with go: go1.21.3)
	gotests:	/home/jon/go/bin/gotests	(version: v1.6.0 built with go: go1.21.3)
	gomodifytags:	/home/jon/go/bin/gomodifytags	(version: v1.16.0 built with go: go1.21.3)
	impl:	/home/jon/go/bin/impl	(version: v1.1.0 built with go: go1.21.3)
	goplay:	/home/jon/go/bin/goplay	(version: v1.0.0 built with go: go1.21.3)
	dlv:	/home/jon/go/bin/dlv	(version: v1.21.1 built with go: go1.21.3)
	golangci-lint:	/home/jon/go/bin/golangci-lint	(version: (devel) built with go: go1.21.3)

## Go env

Workspace Folder (nex-go-rewrite): /home/jon/pretendo/nex-go-rewrite

	GO111MODULE=''
	GOARCH='amd64'
	GOBIN=''
	GOCACHE='/home/jon/.cache/go-build'
	GOENV='/home/jon/.config/go/env'
	GOEXE=''
	GOEXPERIMENT=''
	GOFLAGS=''
	GOHOSTARCH='amd64'
	GOHOSTOS='linux'
	GOINSECURE=''
	GOMODCACHE='/home/jon/go/pkg/mod'
	GONOPROXY=''
	GONOSUMDB=''
	GOOS='linux'
	GOPATH='/home/jon/go'
	GOPRIVATE=''
	GOPROXY='direct'
	GOROOT='/usr/local/go'
	GOSUMDB='sum.golang.org'
	GOTMPDIR=''
	GOTOOLCHAIN='auto'
	GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
	GOVCS=''
	GOVERSION='go1.21.3'
	GCCGO='gccgo'
	GOAMD64='v1'
	AR='ar'
	CC='gcc'
	CXX='g++'
	CGO_ENABLED='1'
	GOMOD='/home/jon/pretendo/nex-go-rewrite/go.mod'
	GOWORK=''
	CGO_CFLAGS='-O2 -g'
	CGO_CPPFLAGS=''
	CGO_CXXFLAGS='-O2 -g'
	CGO_FFLAGS='-O2 -g'
	CGO_LDFLAGS='-O2 -g'
	PKG_CONFIG='pkg-config'
	GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build371863881=/tmp/go-build -gno-record-gcc-switches'
	
Workspace Folder (nex-protocols-go): /home/jon/pretendo/nex-protocols-go

	GO111MODULE=''
	GOARCH='amd64'
	GOBIN=''
	GOCACHE='/home/jon/.cache/go-build'
	GOENV='/home/jon/.config/go/env'
	GOEXE=''
	GOEXPERIMENT=''
	GOFLAGS=''
	GOHOSTARCH='amd64'
	GOHOSTOS='linux'
	GOINSECURE=''
	GOMODCACHE='/home/jon/go/pkg/mod'
	GONOPROXY=''
	GONOSUMDB=''
	GOOS='linux'
	GOPATH='/home/jon/go'
	GOPRIVATE=''
	GOPROXY='direct'
	GOROOT='/usr/local/go'
	GOSUMDB='sum.golang.org'
	GOTMPDIR=''
	GOTOOLCHAIN='auto'
	GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
	GOVCS=''
	GOVERSION='go1.21.3'
	GCCGO='gccgo'
	GOAMD64='v1'
	AR='ar'
	CC='gcc'
	CXX='g++'
	CGO_ENABLED='1'
	GOMOD='/home/jon/pretendo/nex-protocols-go/go.mod'
	GOWORK='/home/jon/pretendo/nex-protocols-go/go.work'
	CGO_CFLAGS='-O2 -g'
	CGO_CPPFLAGS=''
	CGO_CXXFLAGS='-O2 -g'
	CGO_FFLAGS='-O2 -g'
	CGO_LDFLAGS='-O2 -g'
	PKG_CONFIG='pkg-config'
	GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build152155767=/tmp/go-build -gno-record-gcc-switches'
	

Share the Go related settings you have added/edited

Run Preferences: Open Settings (JSON) command to open your settings.json file.
Share all the settings with the go. or ["go"] or gopls prefixes.

  • "go.toolsManagement.autoUpdate": true

Describe the bug

replace directive seems to be ignored in go.work file. Failing to load the correct local package. Only occurs in VS Code, go vet reports no issues.

This behavior was working up until recently. I do not have an exact time/version this stopped working, as it has been a few days since I touched these modules.

Steps to reproduce the behavior:

  1. Create 2 modules
  2. Add both to a VS Code workspace
  3. Create a go.work in ModuleA
  4. Set a replace directive in the go.work to point to ModuleB
  5. ModuleB fails to be loaded in VS Code

Screenshots or recordings

go.work contents:

go 1.21

use .

replace (
	github.com/PretendoNetwork/nex-go => ../nex-go-rewrite
)

Broken import/wrong package being referenced leading to many errors:
Screenshot from 2024-03-16 19-23-19

The local package does exist:
Screenshot from 2024-03-16 19-24-37

go vet command showing no errors (issue is only within VS Code):
Screenshot from 2024-03-16 19-25-24

@jonbarrow
Copy link
Author

Possibly related to golang/vscode-go#3258?

@findleyr
Copy link
Contributor

Hmm, while I believe that there may be a bug here (this logic changed with gopls@v0.15.0+), I'm not yet able to reproduce.

Specifically, I checked out https://go.googlesource.com/tools as well as https://go.googlesource.com/mod, and replaced golang.org/x/mod in the tools/go.work

> cat go.work
go 1.23

use (
        .
        ./gopls
)

replace golang.org/x/mod => ../mod

Everything worked as expected, and jumping to the definition of something imported from x/mod goes to the correct directory.

So, it seems there is something specific about your modules that exacerbate this bug -- it doesn't reproduce with an arbitrary go.work replace directive.

One question, which may seem counterintuitive: do things like jump-to-definition work in these buffers, even though they have import errors? In other words, can you jump to definitions in nex-go? If so, do they go to the locally replaced module?

@findleyr
Copy link
Contributor

Since it looks like you're working in open source, would it be possible to create two branches for your ongoing work? (one for the nex-protocols-go repo and another for the nex-go repo)? A personal fork should be fine.

Then I can try to reproduce in your specific workspace.

@jonbarrow
Copy link
Author

One question, which may seem counterintuitive: do things like jump-to-definition work in these buffers, even though they have import errors? In other words, can you jump to definitions in nex-go? If so, do they go to the locally replaced module?

When using Go to Definition it does jump to the locally replaced module, despite the import errors. Example GIF:

Peek 2024-03-18 15-11

Additionally it seems like even some local imports, not in the replaced module, also do not work. Example:

Screenshot from 2024-03-18 15-10-59-edit

I'm not yet able to reproduce

This is not surprising to be honest. I forgot to mention it in my original post, but this is inconsistent even in my setup. Here is an example from the same module where these errors do not occur (though you can see in the same screenshot, other files do have these errors despite identical imports and usage)

Screenshot from 2024-03-18 15-18-27

Since it looks like you're working in open source, would it be possible to create two branches for your ongoing work?

Both of these repositories already have public branches with the latest changes. They can be found here:

Not mentioned here, as I assumed it was unnecessary, but there is also a 3rd repository with this same setup having the same issues. I did not mention it, as I felt only the 1 was enough to demonstrate the issue. But that 3rd repository can be found here https://github.com/PretendoNetwork/nex-protocols-common-go/tree/nex-go-rewrite

It's also worth noting that this morning I started a new VS Code workspace from scratch, with ONLY those 2 repositories (the one shown here has many dozens of projects in it, some of which are confidential which is why they were not shown here), and in that new fresh workspace this error does not seem to happen.

@findleyr
Copy link
Contributor

Thanks, this looks like a variant of #66145, which was fixed in gopls@v0.15.2, albeit in a rather tricky and defensive way. There may be another way to exercise the bug.

Just to be clear: you are seeing this even with gopls@v0.15.2, right? If not, then the bug may have already been fixed.

@jonbarrow
Copy link
Author

Just to be clear: you are seeing this even with gopls@v0.15.2, right? If not, then the bug may have already been fixed.

Yes. The output from gopls -v version is posted in my original post. However I did update Go since then, so here is my new output

Build info
----------
golang.org/x/tools/gopls v0.15.2
    golang.org/x/tools/gopls@v0.15.2 h1:4JKt4inO8JaFW3l/Fh9X1k/5JQn+iUOpdc4/Lpi0mOs=
    github.com/BurntSushi/toml@v1.2.1 h1:9F2/+DoOYIOksmaJFPw1tGFy1eDnIJXg+UHjuD8lTak=
    github.com/google/go-cmp@v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
    golang.org/x/exp/typeparams@v0.0.0-20221212164502-fae10dda9338 h1:2O2DON6y3XMJiQRAS1UWU+54aec2uopH3x7MAiqGW6Y=
    golang.org/x/mod@v0.15.0 h1:SernR4v+D55NyBH2QiEQrlBAnj1ECL6AGrA5+dPaMY8=
    golang.org/x/sync@v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
    golang.org/x/telemetry@v0.0.0-20240209200032-7b892fcb8a78 h1:vcVnuftN4J4UKLRcgetjzfU9FjjgXUUYUc3JhFplgV4=
    golang.org/x/text@v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
    golang.org/x/tools@v0.18.1-0.20240311201521-78fbdeb61842 h1:No0LMXYFkp3j4oEsPdtY8LUQz33gu79Rm9DE+izMeGQ=
    golang.org/x/vuln@v1.0.1 h1:KUas02EjQK5LTuIx1OylBQdKKZ9jeugs+HiqO5HormU=
    honnef.co/go/tools@v0.4.6 h1:oFEHCKeID7to/3autwsWfnuv69j3NsfcXbvJKuIcep8=
    mvdan.cc/gofumpt@v0.6.0 h1:G3QvahNDmpD+Aek/bNOLrFR2XC6ZAdo62dZu65gmwGo=
    mvdan.cc/xurls/v2@v2.5.0 h1:lyBNOm8Wo71UknhUs4QTFUNNMyxy2JEIaKKo0RWOh+8=
go: go1.22.1

@findleyr
Copy link
Contributor

Yes. The output from gopls -v version is posted in my original post. However I did update Go since then, so here is my new output

Understood, and I saw that, but given the timing of the release wondered if perhaps the screenshots were taken at a different time.

Another question: do these errors persist even after running Developer: Reload Window?

Also, I'm curious why you are using a go.work replace directive rather than adding the nex-go-rewrite directory to your workspace. Of course gopls should not be broken, but I wonder if you see the same errors when using, rather than replacing, nex-go-rewrite.

@findleyr
Copy link
Contributor

Yet more questions: among the many other proprietary folders in your workspace do any have a local replace of nex-go or nex-go-protocol, or do they have a go.work file that uses either? Are any of your folders in a GOPATH directory?

Sorry for the barrage of questions, but the nature of the bug is as follows: the diagnostics you see are from one of your other workspace folders, complaining that it can't load the import for the package in nex-go-protocol. The problem is that this folder shouldn't care about that package, and therefore shouldn't produce diagnostics for it.

This is an unfortunately complicated problem, because with GOPATH directories, go.work files, and local replaces in both go.work and go.mod files, there are many ways for a file to be included in a build. In order to produce accurate and yet complete diagnostics, gopls must have accurate heuristics for when a package should be loadable from a workspace folder.

@jonbarrow
Copy link
Author

jonbarrow commented Mar 18, 2024

Understood, and I saw that, but given the timing of the release wondered if perhaps the screenshots were taken at a different time.

Understood. To be clear then, all information and screenshots are from the time of writing unless otherwise specified

Another question: do these errors persist even after running Developer: Reload Window?

Yes. This persists between window reloads, between closing and reopening VS Code entirely, and I've even tried rebooting my whole machine. The errors are both persistent and consistent where they appear (it doesn't affect random files each time, it affects the same ones each time despite the errors themselves being inconsistent)

Also, I'm curious why you are using a go.work replace directive rather than adding the nex-go-rewrite directory to your workspace. Of course gopls should not be broken, but I wonder if you see the same errors when using, rather than replacing, nex-go-rewrite.

nex-go-rewrite is part of my VS Code workspace. Breaking this out into it's own folder was just something I decided to do during a rewrite, as I still wanted to reference parts of the older implementation without constantly swapping branches. It's the same repository as nex-go, which is also in my VS Code workspace, just only the rewrite branch. We use rewrite directives during local development just as part of our workflow.

Though I don't believe that's really relevant. I have not tried directly importing the replaced module, mostly because this issue seems to not just be limited to that module. As stated in my first reply from this morning, this also affects locally imported packages from the same module (I was not aware of this until this morning). Here is the screenshot from earlier:

Screenshot from 2024-03-18 15-10-59-edit

Also when starting a new VS Code workspace from scratch these errors seem to not occur. I was planning to remake the workspace from scratch, seeing where it might break at, but have not found the time to do so yet.

Yet more questions: among the many other proprietary folders in your workspace do any have a local replace of nex-go or nex-go-protocol, or do they have a go.work file that uses either? Are any of your folders in a GOPATH directory?

Yes, many do have replaces for those 2, and many do not. Due to the aforementioned rewrite, many of these are currently broken for other reasons though so getting finer details on them would take some time. I did some touch ups on one just now for this question, and it looks like it's importing correctly. Here are 2 screenshots showing the rewrite directive being properly applied in this case (ignore all the other file errors, they are unrelated):

Screenshot from 2024-03-18 16-01-17
Screenshot from 2024-03-18 16-01-37

Sorry for the barrage of questions

It's no trouble, really. Ask as many as you need, I'll answer them as best I can (though I am no expert on these things). Whatever helps get things working smoothly again, as this setup worked for several years before now

@findleyr
Copy link
Contributor

Thanks. Let me think about the fastest way to figure this out. In the meantime, you should be able to get back to work by downgrading to v0.14.2:

go install golang.org/x/tools/gopls@v0.14.2

Sorry about the breakage.

@jonbarrow
Copy link
Author

No worries, thank you for taking the time to look into it.

Just to confirm by the way, downgrading to 0.14.2 did indeed fix the issue 👍

@findleyr findleyr changed the title replace directive in go.work seems to be ignored x/tools/gopls: import errors when using go.work replace directives Mar 20, 2024
@findleyr findleyr self-assigned this Mar 20, 2024
@findleyr findleyr transferred this issue from golang/vscode-go Mar 20, 2024
@findleyr findleyr added this to the gopls/v0.16.0 milestone Mar 20, 2024
@gopherbot gopherbot added Tools This label describes issues relating to any tools in the x/tools repository. gopls Issues related to the Go language server, gopls. labels Mar 20, 2024
@findleyr
Copy link
Contributor

This appears to be a gopls issue, so I transferred to the Go issue tracker.

One possibility is that, among your workspace folders, there is a folder where these import errors are actually correct, i.e. there's a workspace that legitimately can't load this module. In gopls@v0.14.2, we didn't merge diagnostics from multiple workspaces, so it would have been easy to miss such errors.

Since we don't have a simple repro, and your standard workspace setup sounds quite complicated, I think the thing to do is prepare a CL that more clearly identifies which folders are producing a diagnostic. Then we can perhaps narrow down the reproducer. I'll do this, but it might take a week or so as I'm traveling.

I suspect that this issue only affects a very small number of users: only a small fraction of users have multi-root workspaces, and among them only another fraction use multiple go.work files, and among them only a fraction use go.work replace directives (these are quite rare).

CC @adonovan for awareness. Alan: I don't think any action is required on your part unless this appears to be affecting a larger number of users, since @jonbarrow is unblocked by downgrading to gopls@v0.14.2. I want to get to the bottom of this, but it requires some work on my end to help diagnose.

@gopherbot
Copy link

Change https://go.dev/cl/574718 mentions this issue: gopls/internal/server: filter diagnostics to "best" views

@findleyr
Copy link
Contributor

findleyr commented Apr 1, 2024

@jonbarrow, I wonder if https://go.dev/cl/574718 improves the situation for you. Could you test?

Here's how to install that CL:

git clone https://go.googlesource.com/tools
cd tools/gopls
git fetch https://go.googlesource.com/tools refs/changes/18/574718/1 && git checkout FETCH_HEAD
go install

Does that version of gopls still have the spurious import errors?

@jonbarrow
Copy link
Author

@jonbarrow, I wonder if go.dev/cl/574718 improves the situation for you. Could you test?

Here's how to install that CL:

git clone https://go.googlesource.com/tools
cd tools/gopls
git fetch https://go.googlesource.com/tools refs/changes/18/574718/1 && git checkout FETCH_HEAD
go install

Does that version of gopls still have the spurious import errors?

So far so good! 👍

@findleyr
Copy link
Contributor

findleyr commented Apr 3, 2024

Thanks for testing. I think this change is safe to make, so I've tentatively marked this as fixed.

It would be good to understand exactly how the spurious diagnostics are being produced, but I think it's also safe to simply filter diagnostics to the most relevant builds, as I've done.

@gopherbot
Copy link

Change https://go.dev/cl/577261 mentions this issue: [gopls-release-branch.0.15] gopls/internal/server: filter diagnostics to "best" views

gopherbot pushed a commit to golang/tools that referenced this issue Apr 8, 2024
… to "best" views

Filter diagnostics only to the "best" view for a file. This reduces the
likelihood that we show spurious import diagnostics due to module graph
pruning, as reported by golang/go#66425.

Absent a reproducer this is hard to test, yet the change makes intuitive
sense (arguably): it is confusing if diagnostics are inconsistent with
other operations like jump-to-definition that find the "best" view.

Fixes golang/go#66425
Updates golang/go#66730

Change-Id: Iadb1a01518a30cc3dad2d412b1ded612ab35d6cc
Reviewed-on: https://go-review.googlesource.com/c/tools/+/574718
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit f345449)
Reviewed-on: https://go-review.googlesource.com/c/tools/+/577261
Auto-Submit: Robert Findley <rfindley@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gopls Issues related to the Go language server, gopls. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Development

No branches or pull requests

3 participants