Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: file corruption on case-insensitive filesystems #57081

Open
CompeyDev opened this issue Dec 1, 2022 · 20 comments
Open

x/tools/gopls: file corruption on case-insensitive filesystems #57081

CompeyDev opened this issue Dec 1, 2022 · 20 comments
Assignees
Labels
gopls/corruption Issues related to file corruption in gopls gopls Issues related to the Go language server, gopls. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone

Comments

@CompeyDev
Copy link

What version of Go, VS Code & VS Code Go extension are you using?

Version Information
  • Run go version to get version of Go from the VS Code integrated terminal.
    • go version go1.19.3 windows/amd64
  • Run gopls -v version to get version of Gopls from the VS Code integrated terminal.
    • golang.org/x/tools/gopls v0.10.1
  • Run code -v or code-insiders -v to get version of VS Code or VS Code Insiders.
    • 1.73.1
  • Check your installed extensions to get the version of the VS Code Go extension
    • 0.36.0
  • Run Ctrl+Shift+P (Cmd+Shift+P on Mac OS) > Go: Locate Configured Go Tools command.
GOBIN: undefined
toolsGopath: 
gopath: C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go
GOROOT: C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go
PATH: C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\dotnet\;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;%SYSTEMROOT%\System32\OpenSSH\;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\Programs\Python\Python310\Scripts\;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\Programs\Python\Python310\;%PNPM_HOME%;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\Programs\Python\Launcher\;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\Microsoft\WindowsApps;C:\Users\root.DESKTOP-FTCACMV\.aftman\bin;C:\Users\root.DESKTOP-FTCACMV\.deno\bin;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\JetBrains\Toolbox\scripts;C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\Microsoft\WindowsApps;C:\Users\root.DESKTOP-FTCACMV\VSCodium-win32-x64-1.72.2.22289;C:\Users\root.DESKTOP-FTCACMV\VSCodium-win32-x64-1.72.2.22289\bin;C:\Users\root.DESKTOP-FTCACMV\AppData\Local\Programs\Hyper\resources\bin;C:\Users\root.DESKTOP-FTCACMV\node\node-v18.12.1-win-x64\node_modules\npm\bin;C:\Users\root.DESKTOP-FTCACMV\node\node-v18.12.1-win-x64\node_modules\npm;C:\Users\root.DESKTOP-FTCACMV\node\node-v18.12.1-win-x64;C:\Users\root.DESKTOP-FTCACMV\AppData\Roaming\Spotify;C:\Users\root.DESKTOP-FTCACMV\git;C:\Users\root.DESKTOP-FTCACMV\git\bin;

	go:	C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\go.exe: go version go1.19.3 windows/amd64

	gotests:	C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\gotests.exe	(version: v1.6.0 built with go: go1.19.3)
	gomodifytags:	C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\gomodifytags.exe	(version: v1.16.0 built with go: go1.19.3)
	impl:	C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\impl.exe	(version: v1.1.0 built with go: go1.19.3)
	goplay:	C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\goplay.exe	(version: v1.0.0 built with go: go1.19.3)
	dlv:	C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\dlv.exe	(version: v1.9.1 built with go: go1.19.3)
	golint:	C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\golint.exe	(version: v0.0.0-20210508222113-6edffad5e616 built with go: go1.19.3)
	gopls:	C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\bin\gopls.exe	(version: v0.10.1 built with go: go1.19.3)

go env
Workspace Folder (nemo): c:\Users\root.DESKTOP-FTCACMV\dev\nemo
	set GO111MODULE=
	set GOARCH=amd64
	set GOBIN=
	set GOCACHE=C:\Users\root.DESKTOP-FTCACMV\AppData\Local\go-build
	set GOENV=C:\Users\root.DESKTOP-FTCACMV\AppData\Roaming\go\env
	set GOEXE=.exe
	set GOEXPERIMENT=
	set GOFLAGS=
	set GOHOSTARCH=amd64
	set GOHOSTOS=windows
	set GOINSECURE=
	set GOMODCACHE=C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\pkg\mod
	set GONOPROXY=
	set GONOSUMDB=
	set GOOS=windows
	set GOPATH=C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go
	set GOPRIVATE=
	set GOPROXY=https://proxy.golang.org,direct
	set GOROOT=C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go
	set GOSUMDB=sum.golang.org
	set GOTMPDIR=
	set GOTOOLDIR=C:\Users\root.DESKTOP-FTCACMV\go1.19.3.windows-amd64\go\pkg\tool\windows_amd64
	set GOVCS=
	set GOVERSION=go1.19.3
	set GCCGO=gccgo
	set GOAMD64=v1
	set AR=ar
	set CC=gcc
	set CXX=g++
	set CGO_ENABLED=1
	set GOMOD=c:\Users\root.DESKTOP-FTCACMV\dev\nemo\go.mod
	set GOWORK=
	set CGO_CFLAGS=-g -O2
	set CGO_CPPFLAGS=
	set CGO_CXXFLAGS=-g -O2
	set CGO_FFLAGS=-g -O2
	set CGO_LDFLAGS=-g -O2
	set PKG_CONFIG=pkg-config
	set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=C:\Users\ROOT~1.DES\AppData\Local\Temp\go-build2785085047=/tmp/go-build -gno-record-gcc-switches

Share the Go related settings you have added/edited

None, all default.

Describe the bug

The lint process of the extension is extremely bugged, and ends up deleting random bunches of code. If you save it during this process, the file's contents get discarded entirely.

Steps to reproduce the behavior:

  1. Create a large file, with over 300 lines of code.
  2. Make some changes, and save the file.
  3. If you instantly save the file once you input your code (possibly before the code can be checked by the extension), it discards all the changes made.
  4. If you wait for a minute before saving the file (possibly after the code has been checked by the extension), it removes random characters from your code and saves the file.

Screenshots or recordings

Following are screenshots of before and after saving a file with changes made. Notice how the fmt.Sprint(instanceCheckingErr) I added magically disappears once saved.

Before:

After:

@findleyr
Copy link
Contributor

findleyr commented Dec 2, 2022

Hi, I'm not aware of anything in the extension or language server that would cause the behavior you describe. I'm not saying it's impossible, but given the frequency with which you describe encountering this problem, it seems likely that there is something else going on (perhaps a bad interaction with another extension?).

If you are able to reliably repro, we would be very interested to get to the bottom of this. The easiest way for us to see what is going on is to look at LSP logs, which you can collect as described here:
https://github.com/golang/vscode-go/blob/master/docs/troubleshooting.md#collect-gopls-information

Could you try to collect those logs for a minimal repro?

@CompeyDev
Copy link
Author

CompeyDev commented Dec 2, 2022

Seems like the issue is occurring due to this extension itself, as I disabled all other extensions and this still occurs.

You can view the logs here.

EDIT: This is only occurring with one file, as all other files don't seem to face this. Even tried creating a new VSCode workspace to try and troubleshoot but does not seem to make any difference. This is the exact file.

@findleyr
Copy link
Contributor

findleyr commented Dec 2, 2022

Aha, I stared at this for a bit and believe that I have found a likely related issue.

Look at these two logs:

[Trace - 09:54:38.531 AM] Received notification 'textDocument/publishDiagnostics'.
Params: {"uri":"file:///C:/Users/root.DESKTOP-FTCACMV/dev/nemo/src/server/networking.go","diagnostics":[{"range":{"start":{"line":85,"character":15},"end":{"line":85,"character":33}},"severity":1,"code":"UndeclaredName","codeDescription":{"href":"https://pkg.go.dev/golang.org/x/tools/internal/typesinternal#UndeclaredName"},"source":"compiler","message":"undeclared name: jsonErrorResponse2"}]}


[Trace - 09:54:38.531 AM] Received notification 'textDocument/publishDiagnostics'.
Params: {"uri":"file:///C:/Users/root.DESKTOP-FTCACMV/dev/nemo/src/Server/networking.go","diagnostics":[]}

I noticed that we apparently have two different representations for this file: one with a capital 'S' for 'Server', and another with a lower-case 's' for 'server'. It looks like the 'didChangeWatchedFile' notification (arriving from the file watcher) has a lower-case 's', but notifications related to the open file have upper-case 'S'. Clearly gopls is confused about the identity of this file, because it sent different diagnostics for the two files.

I'm not yet sure how this causes the corruption you observe, but it seems likely that either the editor is confused about file identity, gopls is confused, or both.

Can you tell me more about your environment? What does the filesystem say about this path? Do you know where the capital 'S' originated?

I'll do more digging to try to understand how this could cause problems in gopls.

@findleyr
Copy link
Contributor

findleyr commented Dec 2, 2022

It would also be incredibly helpful if you could include a full set of logs for a session that repros the problem, starting from the beginning. That would let us see when and how the two different casings were communicated.

@findleyr
Copy link
Contributor

findleyr commented Dec 2, 2022

Hi, I believe I understand this now. The key is in the following logs:

[Trace - 09:54:37.094 AM] Sending request 'textDocument/codeAction - (115)'.
Params: {"textDocument":{"uri":"file:///c%3A/Users/root.DESKTOP-FTCACMV/dev/nemo/src/Server/networking.go"},"range":{"start":{"line":85,"character":33},"end":{"line":85,"character":33}},"context":{"diagnostics":[],"triggerKind":2}}


[Trace - 09:54:37.096 AM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2022/12/02 09:54:37 fixImports(filename=\"C:\\\\Users\\\\root.DESKTOP-FTCACMV\\\\dev\\\\nemo\\\\src\\\\server\\\\networking.go\"), abs=\"C:\\\\Users\\\\root.DESKTOP-FTCACMV\\\\dev\\\\nemo\\\\src\\\\server\\\\networking.go\", srcDir=\"C:\\\\Users\\\\root.DESKTOP-FTCACMV\\\\dev\\\\nemo\\\\src\\\\server\" ...\n"}


[Trace - 09:54:37.097 AM] Received response 'textDocument/codeAction - (115)' in 2ms.
Result: null

These show that gopls got a request for Server, but is actually looking for server. That means that gopls has canonicalized the URL internally to the version on disk (it uses the first version it sees as canonical).

But it looks like content invalidation does not canonicalize URIs. So a didOpen or didChange of the same URI will NOT invalidate on-disk contents inside of gopls. This is simply a bug.

In other words, I think the bug goes as follows:

  • Load a workspace with server on a case insensitive file-system.
  • Issue a didOpen and didChange for Server. This will fail to invalidate the content already loaded for server.
  • Issue a formatting request for Server. Gopls will canonicalize this request to server, and find the old contents (which have not been invalidated). If the resulting edits are non-empty, they will be incorrect and leave a corrupted state in the editor.

Now to find a case-insensitive filesystem to reproduce...

@CompeyDev
Copy link
Author

This seems correct. I did a bit of digging too, and attempted to copy and paste the contents of the file into another. This does not repro the issue, but if I directly copy the file itself (and not just its contents), the issue does reoccur, no matter where I store it.

@CompeyDev
Copy link
Author

It would also be incredibly helpful if you could include a full set of logs for a session that repros the problem, starting from the beginning. That would let us see when and how the two different casings were communicated.

I created a new workspace in the same folder and opened the db.go file, followed by the buggy networking.go file, and these are the logs it produced.

@findleyr findleyr changed the title Severely broken linting with extension x/tools/gopls: file corruption on case-insensitive filesystems Dec 5, 2022
@findleyr findleyr transferred this issue from golang/vscode-go Dec 5, 2022
@findleyr findleyr added this to the gopls/v0.11.0 milestone Dec 5, 2022
@gopherbot gopherbot added Tools This label describes issues relating to any tools in the x/tools repository. gopls Issues related to the Go language server, gopls. labels Dec 5, 2022
@findleyr
Copy link
Contributor

findleyr commented Dec 5, 2022

Thanks, while we haven't yet reproduced locally, I think we have enough information about the nature of the bug. I have transferred this to the Go issue tracker.

@CompeyDev
Copy link
Author

@findleyr Any updates?

@findleyr
Copy link
Contributor

@CompeyDev unfortunately this is going to take a little while to fix. We have ideas, but it is a rather low level bug. We will try to fix this for the next minor release, which is slated for ~February, but there is no guarantee.

@CompeyDev
Copy link
Author

@CompeyDev unfortunately this is going to take a little while to fix. We have ideas, but it is a rather low level bug. We will try to fix this for the next minor release, which is slated for ~February, but there is no guarantee.

Understood, is there at least a way I can mitigate this? It is quite disruptive at the moment.

@findleyr
Copy link
Contributor

@CompeyDev sorry for missing your last comment. If you can get your editor to agree about file casing that would avoid the problem.

I'm working on a fix now, but as I said it will not arrive this month. I will ping here when there is something ready for testing.

@CompeyDev
Copy link
Author

If you can get your editor to agree about file casing that would avoid the problem.

How exactly can I achieve this in VSCode? I'm not really sure what kind of casing VSCode on Windows follows.

@findleyr findleyr added the gopls/corruption Issues related to file corruption in gopls label Jan 19, 2023
@findleyr findleyr self-assigned this Jan 19, 2023
@gopherbot
Copy link

Change https://go.dev/cl/462819 mentions this issue: gopls: fix windows file corruption

@findleyr
Copy link
Contributor

How exactly can I achieve this in VSCode? I'm not really sure what kind of casing VSCode on Windows follows.

Sorry, I'm not sure why VS Code is choosing a different spelling of this file.

The CL above (not yet submitted) fixes the file corruption, but there is more work yet as other features won't work in this file: we need to identify that this is the same file as returned to us from the go command.

gopherbot pushed a commit to golang/tools that referenced this issue Jan 30, 2023
Fix the bug that gopls finds the wrong content when formatting an open
URI whose spelling does not match the spelling on disk (i.e. because of
case insensitivity).

Remove the whole View.filesByBase mechanism: it is problematic as we
can't generally know whether or not we want to associate two different
spellings of the same file: for the purposes of finding packages we may
want to treat Foo.go as foo.go, but we don't want to treat a symlink of
foo.go in another directory the same.

Instead, use robustio.FileID to de-duplicate content in the cache, and
otherwise treat URIs as we receive them. This fixes the formatting
corruption, but means that we don't find packages for the corresponding
file (because go/packages.Load("file=foo.go") fails).  A failing test is
added for the latter bug.

Also: use a seenFiles map in the view to satisfy the concern of tracking
relevant files, with a TODO to delete this problematic map.

Along the way, refactor somewhat to separate and normalize the
implementations of source.FileSource.

For golang/go#57081

Change-Id: I02971a1702f057b644fa18a873790e8f0d98a323
Reviewed-on: https://go-review.googlesource.com/c/tools/+/462819
gopls-CI: kokoro <noreply+kokoro@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
@findleyr findleyr modified the milestones: gopls/v0.12.0, gopls/v0.13.0 Mar 27, 2023
@CompeyDev
Copy link
Author

Heya, it's been a while now - any new updates, @findleyr?

@findleyr
Copy link
Contributor

@CompeyDev sorry, missed your ping.

We have been busy with the v0.12.0 release, but this is one of the largest issues we hope to fix for v0.13.0, as we believe it is the largest outstanding gopls bug.

@CompeyDev
Copy link
Author

Alright, as for now I've switched my environment to Linux & everything seems to work as expected.

Thanks for the update!

@findleyr findleyr added this to the gopls/v0.15.0 milestone Oct 9, 2023
@findleyr findleyr modified the milestones: gopls/v0.15.0, gopls/v0.16.0 Dec 12, 2023
@alesji
Copy link

alesji commented Dec 14, 2023

I am facing the same issue right now... can i at least somehow restart/clear gopls completely so it stops doing it? it happened when i renamed User.go to user.go on windows... I guess it has User.go stored somewhere, so some sort of clearing would help, how do i do that?

edit clearing c:/Users/%USER%/AppData/Local/gopls/ helped

@findleyr
Copy link
Contributor

@alesji are you experiencing corruption, or just a broken LSP server?

Your files should no longer be corrupted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gopls/corruption Issues related to file corruption in gopls gopls Issues related to the Go language server, gopls. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Development

No branches or pull requests

4 participants