Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: when working in vendor mode autoimport adds packages from vendor dir #56291

Closed
inliquid opened this issue Oct 18, 2022 · 21 comments
Assignees
Labels
gopls Issues related to the Go language server, gopls. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone

Comments

@inliquid
Copy link

inliquid commented Oct 18, 2022

gopls version

>gopls -v version
Build info
----------
golang.org/x/tools/gopls master
    golang.org/x/tools/gopls@(devel)
    github.com/BurntSushi/toml@v1.2.0 h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
    github.com/google/go-cmp@v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
    github.com/sergi/go-diff@v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
    golang.org/x/exp@v0.0.0-20220722155223-a9213eeb770e h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
    golang.org/x/exp/typeparams@v0.0.0-20220722155223-a9213eeb770e h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE=
    golang.org/x/mod@v0.6.0-dev.0.20220419223038-86c51ed26bb4 h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
    golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4 h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
    golang.org/x/sys@v0.0.0-20220808155132-1c4a2a72c664 h1:v1W7bwXHsnLLloWYTVEdvGvA7BHMeBYsPcF0GLDxIRs=
    golang.org/x/text@v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
    golang.org/x/tools@v0.1.13-0.20220928184430-f80e98464e27 => ../
    golang.org/x/vuln@v0.0.0-20221010193109-563322be2ea9 h1:KaYZQUtEEaV8aVADIHAuYBTjo77aUcCvC7KTGKM3J1I=
    honnef.co/go/tools@v0.3.3 h1:oDx7VAwstgpYpb3wv0oxiZlxY+foCpRAwY7Vk6XpAgA=
    mvdan.cc/gofumpt@v0.3.1 h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8=
    mvdan.cc/xurls/v2@v2.4.0 h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
go: go1.19.2

go env

>go env
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\***\AppData\Local\go-build
set GOENV=C:\Users\***\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\***\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\***\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.19.2
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=NUL
set GOWORK=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=C:\Users\***\AppData\Local\Temp\go-build1332515651=/tmp/go-build -gno-record-gcc-switches

What did you do?

When working in vendor module mode, and file does not contain imports yet, press Ctrl-S to force format and auto import actions.

What did you expect to see?

Imports added with correct package names, for example: google.golang.org/grpc

What did you see instead?

Packages added from vendor dir, for example: "path/to/my/module/vendor/google.golang.org/grpc"

@gopherbot gopherbot added Tools This label describes issues relating to any tools in the x/tools repository. gopls Issues related to the Go language server, gopls. labels Oct 18, 2022
@gopherbot gopherbot added this to the Unreleased milestone Oct 18, 2022
@findleyr
Copy link
Contributor

Thank you for the report.

CC @adonovan, as this coincidentally seems very similar to https://go.dev/cl/443636: clearly a package path and import path are being confused here (though I don't necessarily think this will be fixed by that CL -- goimports is mostly disconnected from the rest of gopls).

@inliquid are you by any chance working on an open-source project? If not, we can try to reproduce from scratch.

@findleyr findleyr modified the milestones: Unreleased, gopls/v0.10.1 Oct 19, 2022
@inliquid
Copy link
Author

@findleyr, unfortunately this is a private repo.

@adonovan adonovan self-assigned this Oct 20, 2022
@inliquid
Copy link
Author

I also found that when you use "References" or "Implementations" features, VS Code also displays paths from vendor:
изображение

@findleyr findleyr modified the milestones: gopls/v0.10.1, gopls/v0.10.2 Nov 1, 2022
@jstroem
Copy link

jstroem commented Nov 9, 2022

Working on a private repo I've noticed the same.

@gopherbot
Copy link

Change https://go.dev/cl/448377 mentions this issue: gopls/internal/lsp/cache: use Package.FileSet where possible

gopherbot pushed a commit to golang/tools that referenced this issue Nov 11, 2022
This change adds a FileSet field to Package, and uses it in
preference to Snapshot.FileSet wherever that is appropriate:
all but a handful of places.

For now, they must continue to refer to the same instance,
but once we do away with the Snapshot's cache of parsed files,
there will be no need for a global FileSet and each Package
will be able to create its own. (Some care will be required
to make sure it is always clear which package owns each
each token.Pos/ast.Node/types.Object when there are several
in play.)

Updates golang/go#56291

Change-Id: I017eb794ffb737550da6ae88462d23a8f5c1e1e7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/448377
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
@adonovan
Copy link
Member

Hi @inliquid, I've tried to reproduce this bug a number of ways but with no success. (For example, I vendored a number of packages, created a file with missing imports of packages among the vendored set, and ran 'gopls imports' on them, and each time it got the right answer. I tried various versions of gopls before recent changes in relevant logic. And I tried it without and without a go.work file. I also looked at the various places in our code that we've flagged as suspicious in the way in which they convert between import paths and package paths, and none of them looks like an obvious candidate to explain it.)

I understand this is private code you can't share, but next time you encounter the problem, would you be willing to spend a few minutes trying to create a reproducible test case in terms of packages you can share? As soon as I can reproduce this I can work on a fix. Thanks.

@adonovan adonovan added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Nov 17, 2022
@inliquid
Copy link
Author

@adonovan I work with gopls@master and update it regularly, does it say anything?

Just tried once again by adding glob.Glob in one of working files and pressing Ctrl-S:
изображение

If you have any idea what additional information I can provide please let me know. You are correct this is private repo so I can't share its code unfortunately.

@adonovan
Copy link
Member

adonovan commented Dec 2, 2022

Still no luck reproducing this.

The "implementations" issue you mention in #56291 (comment) is likely a dup of #56169, which should now be fixed.

@inliquid
Copy link
Author

inliquid commented Dec 2, 2022

Well the problem is definitely still present. Except for sharing the project files, what else can I do to help in solving it?

@bferullo
Copy link

bferullo commented Dec 15, 2022

I, and several of my coworkers, are also experiencing this issue. I hacked together a shell script that seems to reliably reproduce it (tested in git-bash and cygwin).

Requires gopls to be installed as a command (I ran go install golang.org/x/tools/gopls@latest).

#!/bin/sh

# create a new dir to hold our package
mkdir vendorbug
cd vendorbug

# create a simple main.go that uses github.com/google/uuid
cat > main.go <<EOT
package main

import (
	"fmt"

	"github.com/google/uuid"
)

func main() {
	fmt.Printf("This is a test package.")

	myUUID, _ := uuid.NewRandom()
	fmt.Printf("The UUID generated was: %s", myUUID)
}
EOT

# create a module
go mod init github.com/bferullo/vendorbug
go mod tidy
go mod vendor

# remove github.com/google/uuid import from main.go
sed --in-place -E '/github.com\/google\/uuid/d' ./main.go

# (verify here that main.go no longer has that import, if you want)

# run gopls imports
gopls imports -d main.go

# EXPECTED RESULT: "github.com/google/uuid" reappears in import block
# OBSERVED RESULT: "github.com/bferullo/vendorbug/vendor/github.com/google/uuid" appears in import block

@findleyr
Copy link
Contributor

Huh, FWIW that script didn't reproduce the bug for me. I completely believe this bug exists, but it must depend on state (likely the state of the module cache).

@adonovan
Copy link
Member

adonovan commented Dec 15, 2022

Same here. github.com/google/uuid appears each time. I tried with GOMODCACHE=$(mktemp -d) too. Could you @bferullo try with that env var and report whether it still happens?

@bferullo
Copy link

Same here. github.com/google/uuid appears each time. I tried with GOMODCACHE=$(mktemp -d) too. Could you @bferullo try with that env var and report whether it still happens?

It does still repro. I also imagine the issue has to do with some persistent state, but I can't figure it out either -- I even installed go, gopls, and gitbash on a machine that previously had none of those, and was able to repro it there as well.

@seankhliao
Copy link
Member

can you give the exact versions of go and gopls you used?

@bferullo
Copy link

$ go version
go version go1.19.4 windows/amd64
$ gopls version
golang.org/x/tools/gopls v0.11.0
    golang.org/x/tools/gopls@v0.11.0 h1:/nvKHdTtePQmrv9XN3gIUN9MOdUrKzO/dcqgbG6x8EY=
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\bferu\AppData\Local\go-build
set GOENV=C:\Users\bferu\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\bferu\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\bferu\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.19.4
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=NUL
set GOWORK=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=C:\Users\bferu\AppData\Local\Temp\go-build375371210=/tmp/go-build -gno-record-gcc-switches

@inliquid
Copy link
Author

I also can reproduce the issue with the above script. Since I also run it on Windows, can OS be a problem?

@bferullo
Copy link

Anecdotally, some colleagues who run OSX also encounter this issue. I'll see if I can get them to test the script specifically.

@seankhliao seankhliao removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Dec 17, 2022
@adonovan
Copy link
Member

adonovan commented May 23, 2023

Still no luck reproducing this on macOS using go1.19 (or go1.19.5) and the script and environment above, and either gopls@v0.11.0 or today's gopls@latest.

@inliquid
Copy link
Author

inliquid commented May 23, 2023

I still can reproduce this with the script from above comments on a brand new Windows 11 machine (old one was Windows 10):

 import (
        "fmt"

+       "github.com/bferullo/vendorbug/vendor/github.com/google/uuid"
 )

PS: I'm using gopls@master with latest commits.

@gopherbot
Copy link

Change https://go.dev/cl/498695 mentions this issue: gopls/internal/regtest: add a test for goimports and windows vendoring

@findleyr
Copy link
Contributor

The test in the above CL (based on the script provided by @bferullo reproduces the failure on our windows trybot. I'll fix this for gopls@v0.12.1 (which we expect not long after v0.12.0...)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gopls Issues related to the Go language server, gopls. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Development

No branches or pull requests

8 participants