Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: gopls implementation result does not include all types in vendor directory #56169

Closed
x-yuri opened this issue Oct 4, 2022 · 13 comments
Assignees
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@x-yuri
Copy link

x-yuri commented Oct 4, 2022

What version of Go, VS Code & VS Code Go extension are you using?

Version Information
  • Run go version to get version of Go from the VS Code integrated terminal.
    • go version go1.19.1 linux/amd64
  • Run gopls -v version to get version of Gopls from the VS Code integrated terminal.
    Build info
    ----------
    golang.org/x/tools/gopls v0.9.5
        golang.org/x/tools/gopls@(devel)
        github.com/BurntSushi/toml@v1.2.0 h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
        github.com/google/go-cmp@v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
        github.com/sergi/go-diff@v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
        golang.org/x/exp@v0.0.0-20220722155223-a9213eeb770e h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
        golang.org/x/exp/typeparams@v0.0.0-20220722155223-a9213eeb770e h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE=
        golang.org/x/mod@v0.6.0-dev.0.20220419223038-86c51ed26bb4 h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
        golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4 h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
        golang.org/x/sys@v0.0.0-20220722155257-8c9f86f7a55f h1:v4INt8xihDGvnrfjMDVXGxw9wrfxYyCjk0KbXjhR55s=
        golang.org/x/text@v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
        golang.org/x/tools@v0.1.13-0.20220908144252-ce397412b6a4 h1:glzimF7qHZuKVEiMbE7UqBu44MyTjt5u6j3Jz+rfMRM=
        golang.org/x/vuln@v0.0.0-20220901221904-62b0186a1058 h1:YnB27EXBD8XxB0JcaOeluuvhF2kS4DrH0k+lpopG2xc=
        honnef.co/go/tools@v0.3.2 h1:ytYb4rOqyp1TSa2EPvNVwtPQJctSELKaMyLfqNP4+34=
        mvdan.cc/gofumpt@v0.3.1 h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8=
        mvdan.cc/xurls/v2@v2.4.0 h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
    go: go1.19.1
    
  • Run code -v or code-insiders -v to get version of VS Code or VS Code Insiders.
    1.71.2
    74b1f979648cc44d385a2286793c226e611f59e7
    x64
    
  • Check your installed extensions to get the version of the VS Code Go extension
    • v0.35.2
  • Run Ctrl+Shift+P (Cmd+Shift+P on Mac OS) > Go: Locate Configured Go Tools command.
    Checking configured tools....
    GOBIN: undefined
    toolsGopath: 
    gopath: /home/yuri/go
    GOROOT: /usr/lib/go
    PATH: /home/yuri/perl5/bin:/home/yuri/bin:/home/yuri/Dropbox/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
    
        go: /usr/bin/go: go version go1.19.1 linux/amd64
    
        gotests:    not installed
        gomodifytags:   not installed
        impl:   not installed
        goplay: not installed
        dlv:    /usr/bin/dlv    (version: (devel) built with go: go1.19)
        staticcheck:    /usr/bin/staticcheck    (version: (devel) built with go: go1.18.5)
        gopls:  /usr/bin/gopls  (version: (devel) built with go: go1.19.1)
    
    go env
    Workspace Folder (docker): /home/yuri/go/src/github.com/docker/docker
        GO111MODULE=""
        GOARCH="amd64"
        GOBIN=""
        GOCACHE="/home/yuri/.cache/go-build"
        GOENV="/home/yuri/.config/go/env"
        GOEXE=""
        GOEXPERIMENT=""
        GOFLAGS=""
        GOHOSTARCH="amd64"
        GOHOSTOS="linux"
        GOINSECURE=""
        GOMODCACHE="/home/yuri/go/pkg/mod"
        GONOPROXY=""
        GONOSUMDB=""
        GOOS="linux"
        GOPATH="/home/yuri/go"
        GOPRIVATE=""
        GOPROXY="https://proxy.golang.org,direct"
        GOROOT="/usr/lib/go"
        GOSUMDB="sum.golang.org"
        GOTMPDIR=""
        GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
        GOVCS=""
        GOVERSION="go1.19.1"
        GCCGO="gccgo"
        GOAMD64="v1"
        AR="ar"
        CC="gcc"
        CXX="g++"
        CGO_ENABLED="1"
        GOMOD="/dev/null"
        GOWORK=""
        CGO_CFLAGS="-g -O2"
        CGO_CPPFLAGS=""
        CGO_CXXFLAGS="-g -O2"
        CGO_FFLAGS="-g -O2"
        CGO_LDFLAGS="-g -O2"
        PKG_CONFIG="pkg-config"
        GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1841263837=/tmp/go-build -gno-record-gcc-switches"
    

Share the Go related settings you have added/edited

I didn't change any settings.

Describe the bug

I trying to examine the moby project, and the extension seems to ignore the vendor dir. Possibly because there's no go.mod at the root of the repository. E.g. it can't find the implementation of p.repo.Blobs, that is located here. As long as I need to navigate within the main module it seems to work fine.

Steps to reproduce the behavior:

  1. Clone https://github.com/moby/moby to ~/go/src/github.com/docker/docker.
  2. Open the directory in code, then distribution/pull_v2.go (line 903).
  3. Right click on Blobs, choose "Go to Implementations".

You'll see only some mock implementations from distribution/push_v2_test.go.

Screenshots or recordings

@hyangah
Copy link
Contributor

hyangah commented Oct 6, 2022

Can you try with the gopls at master?

git clone https://go.googlesource.com/tools
cd tools/gopls
go install

And restart the gopls (Go: Restart Language Server) or reload the vscode window.

I could reproduce the issue with the released version of gopls. But I see the gopls built from master now finds code in the vendor location correctly.

@hyangah hyangah added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Oct 6, 2022
@x-yuri
Copy link
Author

x-yuri commented Oct 6, 2022

I've just tried, but see no difference. It can't find that particular implementation in the vendor dir. I tried both restarting the server and code.

$ ~/go/bin/gopls version
golang.org/x/tools/gopls master
    golang.org/x/tools/gopls@(devel)

Is there a way to make a curl request to gopls or something to confirm this? Can you possibly tell what's the extension's part in it? Does it simply tell code how to connect to the server? Or there's more to it? Any links to the code?

@suzmue
Copy link
Contributor

suzmue commented Oct 6, 2022

Could you please collect a trace and share it here?

You can do this by setting "go.languageServerFlags": ["-rpc.trace"] in your settings.json and then sharing the logs you see in the output channel:

Screen Shot 2021-01-27 at 2 53 49 PM

@hyangah
Copy link
Contributor

hyangah commented Oct 6, 2022

To check the detailed version of gopls -

$ go version -m  /usr/bin/gopls 

The gopls your extension is using is printed in the Go: Locate Configured Go Tools output.

@x-yuri
Copy link
Author

x-yuri commented Oct 6, 2022

The trace: https://gist.github.com/x-yuri/985f8289b15e07fe1038c1eafe679b0b

$ go version -m  /usr/bin/gopls 
/usr/bin/gopls: go1.19.1
	path	golang.org/x/tools/gopls
	mod	golang.org/x/tools/gopls	(devel)	
	dep	github.com/BurntSushi/toml	v1.2.0	h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
	dep	github.com/google/go-cmp	v0.5.8	h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
	dep	github.com/sergi/go-diff	v1.1.0	h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
	dep	golang.org/x/exp	v0.0.0-20220722155223-a9213eeb770e	h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
	dep	golang.org/x/exp/typeparams	v0.0.0-20220722155223-a9213eeb770e	h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE=
	dep	golang.org/x/mod	v0.6.0-dev.0.20220419223038-86c51ed26bb4	h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
	dep	golang.org/x/sync	v0.0.0-20220722155255-886fb9371eb4	h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
	dep	golang.org/x/sys	v0.0.0-20220722155257-8c9f86f7a55f	h1:v4INt8xihDGvnrfjMDVXGxw9wrfxYyCjk0KbXjhR55s=
	dep	golang.org/x/text	v0.3.7	h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
	dep	golang.org/x/tools	v0.1.13-0.20220908144252-ce397412b6a4	h1:glzimF7qHZuKVEiMbE7UqBu44MyTjt5u6j3Jz+rfMRM=
	dep	golang.org/x/vuln	v0.0.0-20220901221904-62b0186a1058	h1:YnB27EXBD8XxB0JcaOeluuvhF2kS4DrH0k+lpopG2xc=
	dep	honnef.co/go/tools	v0.3.2	h1:ytYb4rOqyp1TSa2EPvNVwtPQJctSELKaMyLfqNP4+34=
	dep	mvdan.cc/gofumpt	v0.3.1	h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8=
	dep	mvdan.cc/xurls/v2	v2.4.0	h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
	build	-compiler=gc
	build	-trimpath=true
	build	CGO_ENABLED=1
	build	GOARCH=amd64
	build	GOOS=linux
	build	GOAMD64=v1
$ go version -m  ~/go/bin/gopls
/home/yuri/go/bin/gopls: go1.19.1
	path	golang.org/x/tools/gopls
	mod	golang.org/x/tools/gopls	(devel)	
	dep	github.com/BurntSushi/toml	v1.2.0	h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
	dep	github.com/google/go-cmp	v0.5.8	h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
	dep	github.com/sergi/go-diff	v1.1.0	h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
	dep	golang.org/x/exp	v0.0.0-20220722155223-a9213eeb770e	h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
	dep	golang.org/x/exp/typeparams	v0.0.0-20220722155223-a9213eeb770e	h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE=
	dep	golang.org/x/mod	v0.6.0-dev.0.20220419223038-86c51ed26bb4	h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
	dep	golang.org/x/sync	v0.0.0-20220722155255-886fb9371eb4	h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
	dep	golang.org/x/sys	v0.0.0-20220808155132-1c4a2a72c664	h1:v1W7bwXHsnLLloWYTVEdvGvA7BHMeBYsPcF0GLDxIRs=
	dep	golang.org/x/text	v0.3.7	h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
	dep	golang.org/x/tools	v0.1.13-0.20220928184430-f80e98464e27
	=>	../	(devel)	
	
	dep	golang.org/x/vuln	v0.0.0-20221004232641-2aa0553d353b	h1:8Tu9pgIV7kt8ulNtzidzpLl9E9l1i+U4QLdKG0ZzHyE=
	dep	honnef.co/go/tools	v0.3.3	h1:oDx7VAwstgpYpb3wv0oxiZlxY+foCpRAwY7Vk6XpAgA=
	dep	mvdan.cc/gofumpt	v0.3.1	h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8=
	dep	mvdan.cc/xurls/v2	v2.4.0	h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
	build	-compiler=gc
	build	CGO_ENABLED=1
	build	CGO_CFLAGS=
	build	CGO_CPPFLAGS=
	build	CGO_CXXFLAGS=
	build	CGO_LDFLAGS=
	build	GOARCH=amd64
	build	GOOS=linux
	build	GOAMD64=v1
	build	vcs=git
	build	vcs.revision=bd8c28ff5c76b957a23f93a67337acc0c5a4f67e
	build	vcs.time=2022-10-06T14:25:42Z
	build	vcs.modified=false

Can you try with the gopls at master?

Now that I think about it, maybe I should uninstall gopls and try again.

@x-yuri
Copy link
Author

x-yuri commented Oct 6, 2022

But Go: Locate Configured Go Tools says:

gopls: /home/yuri/go/bin/gopls (version: (devel) built with go: go1.19.1)

All in all looks like a gopls issue. Should I report it there?

@hyangah
Copy link
Contributor

hyangah commented Oct 12, 2022

Thanks for capturing the log.

I also verified this is reproducible - sorry that I was confused by "Find all references" reporting the interface definition correctly. From your original report, I see that you are reporting that the "Find all implementations" (or Implementations) result is missing a type in the vendor directory.

In the example you shared, p.repo.Blobs implements a method of the interface distribution.Repository, so you wanted the implementations query to return all implementations found not only in the main workspace, including distribution.repository. (not distribution.blobs.Get). Here what I mean by main workspace is the code excluding vendor or third-party dependencies.

I think this happens because the implementation type (distribution.repository) in vendor is unexported. For performance, gopls employs various heuristics; one of them is to avoid full type-checking when a package is outside the workspace (i.e. packages in dependency). (relevant code: workspaceParseMode). Packages under vendor seem to be considered packages outside workspace, then gopls will operate in exported symbol only mode. This can affect the implementations query search space and results in skipping unexported types.

I verified gopls returns the implementation type if I rename distribution.repository to an exported name. (See the one with red underline)

Screen Shot 2022-10-11 at 8 44 36 PM

I will transfer this to gopls issue tracker, but I am afraid this is subtle and may not be fixable easily.

@hyangah hyangah changed the title It ignores the vendor dir (code navigation), owing to missing go.mod? x/tools/gopls: gopls implementation result does not include all types in vendor directory Oct 12, 2022
@hyangah hyangah transferred this issue from golang/vscode-go Oct 12, 2022
@hyangah hyangah removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Oct 12, 2022
@gopherbot gopherbot added this to the Unreleased milestone Oct 12, 2022
@x-yuri
Copy link
Author

x-yuri commented Oct 12, 2022

Oh, I just noticed that one of the links in my report is wrong (the second one):

E.g. it can't find the implementation of p.repo.Blobs, that is located here.

And that's probably what you meant here:

In the example you shared, p.repo.Blobs implements a method of the interface distribution.Repository, so you wanted the implementations query to return all implementations found not only in the main workspace, including distribution.repository. (not distribution.blobs.Get).

So to make sure we're on the same page...

I know that p.repo.Blobs() here (distribution/pull_v2.go, v2Puller.pullSchema2Config()) invokes repository.Blobs() here (vendor/github.com/docker/distribution/registry/client/repository.go). According to vscode p.repo (from v2Puller.pullSchema2Config()) implements the Repository interface from vendor/github.com/docker/distribution/registry.go.

In the same vein blobs.Get() here (distribution/pull_v2.go, v2Puller.pullSchema2Config()) invokes blobs.Get() here (vendor/github.com/docker/distribution/registry/client/repository.go). According to vscode blobs (from v2Puller.pullSchema2Config()) implements the BlobProvider interface from vendor/github.com/docker/distribution/blobs.go.

@joedian joedian added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 14, 2022
@findleyr findleyr modified the milestones: Unreleased, gopls/v0.10.1 Oct 24, 2022
@findleyr findleyr modified the milestones: gopls/v0.10.1, gopls/v0.10.2 Nov 1, 2022
@gopherbot
Copy link

Change https://go.dev/cl/454637 mentions this issue: gopls/internal/regtest/misc: test Implementations + vendor

@hyangah
Copy link
Contributor

hyangah commented Dec 2, 2022

I am afraid this isn't fixed. I still don't see the implementation in vendor.
From my initial analysis in #56169 (comment), I thought the root cause was the handling of unexported types outside the main module (regardless it's vendored or not).

@hyangah
Copy link
Contributor

hyangah commented Dec 5, 2022

Reopen for continued investigation & fix verification. cc @adonovan

@hyangah hyangah reopened this Dec 5, 2022
@findleyr findleyr modified the milestones: gopls/v0.11.0, gopls/v0.12.0 Dec 8, 2022
@findleyr
Copy link
Contributor

It looks like the issue may be related to file watching: gopls doesn't ask VS Code to watch the vendor directory. (I discovered this after implementing proper support for glob patterns in our fake LSP client).

@adonovan
Copy link
Member

I cloned moby into $(go env GOPATH)/src and was able to build it (with GO111MODULE=auto) and use gopls@latest within it. In particular, the implementations operation applied to the p.repo.Blobs call in pull_v2.go reported three results:

distribution/push_v2_test.go
559:func (m *mockRepoWithBlob) Blobs(ctx context.Context) distribution.BlobStore {
652:func (m *mockRepo) Blobs(ctx context.Context) distribution.BlobStore {
vendor/github.com/docker/distribution/registry/client/repository.go
161:func (r *repository) Blobs(ctx context.Context) distribution.BlobStore {

I confirm that with v0.11.0 only the first two were reported, so this issue appears to have been fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

7 participants