Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/tools/gopls: gopls walks entire filesystem of a go module #52993

Closed
slon opened this issue May 19, 2022 · 6 comments
Closed

x/tools/gopls: gopls walks entire filesystem of a go module #52993

slon opened this issue May 19, 2022 · 6 comments
Assignees
@dle8 @findleyr
Labels
FrozenDueToAge gopls/workspace Issues related to support for modules or multi-module workspaces. gopls Issues related to the Go language server, gopls. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone
gopls/v0.9.2

Comments

@slon
Copy link
Contributor

slon commented May 19, 2022

gopls version

Build info
----------
golang.org/x/tools/gopls v0.8.3
    golang.org/x/tools/gopls@(devel)
    github.com/BurntSushi/toml@v1.0.0 h1:dtDWrepsVPfW9H/4y7dDgFc2MBUSeJhlaDtK13CxFlU=
    github.com/google/go-cmp@v0.5.7 h1:81/ik6ipDQS2aGcBfIN5dHDB36BwrStyeAQquSYCV4o=
    github.com/sergi/go-diff@v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
    golang.org/x/exp/typeparams@v0.0.0-20220218215828-6cf2b201936e h1:qyrTQ++p1afMkO4DPEeLGq/3oTsdlvdH4vqZUBWzUKM=
    golang.org/x/mod@v0.6.0-dev.0.20220106191415-9b9b3d81d5e3 h1:kQgndtyPBW/JIYERgdxfwMYh3AVStj88WQTlNDi2a+o=
    golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c h1:5KslGYwFpkhGh+Q16bwMP3cOontH8FOep7tGV86Y7SQ=
    golang.org/x/sys@v0.0.0-20220209214540-3681064d5158 h1:rm+CHSpPEEW2IsXUib1ThaHIjuBVZjxNgSKmBLFfD4c=
    golang.org/x/text@v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
    golang.org/x/tools@v0.1.11-0.20220407163324-91bcfb1bdf9c h1:rYiuK/jn6TCxjAq1hQ6MR6pgOeP3A5gIHDxgrQ8Vbys=
    golang.org/x/vuln@v0.0.0-20220324005316-18fd808f5c7f h1:9dMzk88fnONra7zrEalqkRMGa9jMGf9B5mdzhYVyI28=
    golang.org/x/xerrors@v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
    honnef.co/go/tools@v0.3.0 h1:2LdYUZ7CIxnYgskbUZfY7FPggmqnh6shBqfWa8Tn3XU=
    mvdan.cc/gofumpt@v0.3.0 h1:kTojdZo9AcEYbQYhGuLf/zszYthRdhDNDUi2JKTxas4=
    mvdan.cc/xurls/v2@v2.4.0 h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
go: go1.18

go env

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/prime/.cache/go-build"
GOENV="/home/prime/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/prime/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/prime/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/prime/.ya/tools/v4/3102076311"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/prime/.ya/tools/v4/3102076311/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.2"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/prime/arc/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/var/tmp/go-build3354655002=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Our internal version control system is working over fuse. Big mono-repository is mounted as a virtual file system and loaded lazily on demand. Whole repository is a single go module with single go.mod file at the root.

When I open subset of big mono-repository in vscode, gopls starts walking all directories of the repository, breaking lazy load.

What did you expect to see?

gopls should interact only with directories added to workspace.

What did you see instead?

gopls scans all directories of a go module.

This issue originates imports cache. https://github.com/golang/tools/blob/cf66aec62b42a726b36e083c001d3b3231cba2b3/internal/imports/fix.go#L675-L689

It loads all directories from disk without any way of setting up excludes.

I tried adding exclude filter to gopls configuration. This patch fixed issue for me: slon/tools@bfcfbb1

Editor and settings

{
	"folders": [
		{
			"path": "arc/yt"
		},
		{
			"path": "arc/mapreduce"
		},
		{
			"path": "arc/library/cpp"
		},
		{
			"path": "arc/junk/prime"
		},
		{
			"path": "arc/yp"
		},
		{
			"path": "arc/devtools/yamaker"
		},
		{
			"path": "arc/devtools/ya"
		},
		{
			"path": "arc/web/app_host/api/service"
		},
		{
			"path": "arc/library"
		},
		{
			"path": "arc/contrib/libs/tcmalloc"
		}
	],
	"settings": {
        "go.toolsEnvVars": {
            "CGO_ENABLED": "0"
        }, 
        "go.logging.level": "verbose", 
        "go.goroot": "/home/prime/.ya/tools/v4/1272507071", 
        "gopls": {
            "verboseOutput": true, 
            "formatting.local": "a.yandex-team.ru", 
            "build.env": {
                "CGO_ENABLED": "0"
            }, 
            "build.expandWorkspaceToModule": false,
        }, 
        "files.watcherExclude": {
            "**": true
        }
    }
}

Logs
@gopherbot gopherbot added Tools This label describes issues relating to any tools in the x/tools repository. gopls Issues related to the Go language server, gopls. labels May 19, 2022
@gopherbot gopherbot added this to the Unreleased milestone May 19, 2022
@findleyr
Copy link
Contributor

Thanks for the issue, and for the patch. We should reuse the directoryFilters setting, or go.work files, rather than add a new setting. But otherwise what you did looks good.

Rationalizing these settings with go.work is in the plan for gopls@v0.9.0, so I'm including this in that milestone (our next release).

@findleyr findleyr modified the milestones: Unreleased, gopls/v0.9.0 May 19, 2022
@slon
Copy link
Contributor Author

slon commented May 20, 2022

directoryFilters setting is broken in my case.

My workspace consists of multiple folders. directoryFilter is evaluated relative to each directory. But import cache filter should be evaluated relative to module root.

Because of this issue, I added separate setting in my proof-of-concept patch.

It is possible to reuse directoryFilter, but this would require changing semantics in case where workspace directory is not module root.

@findleyr
Copy link
Contributor

I see! Thank you for clarifying. We will fix this.

@findleyr
Copy link
Contributor

Our plan to fix this is to extend directoryFilters to support wildcards, and pass this to goimports. @dle8 is working on the former feature.

https://go.dev/issue/52784 also fixed a bug in gopls that caused scanning of GOPATH when using go.work.

@gopherbot
Copy link

Change https://go.dev/cl/414454 mentions this issue: internal/lsp: Add exclude filters to gopls ModuleResolver.scan

@findleyr findleyr modified the milestones: gopls/v0.9.0, gopls/v0.9.1 Jun 28, 2022
@findleyr findleyr modified the milestones: gopls/v0.9.1, gopls/v0.9.2 Jul 13, 2022
@findleyr findleyr added the gopls/workspace Issues related to support for modules or multi-module workspaces. label Jul 14, 2022
@findleyr
Copy link
Contributor

findleyr commented Aug 4, 2022

This was completed by https://go.dev/cl/420959, and so should be in the v0.9.2 release scheduled for next week.

@findleyr findleyr closed this as completed Aug 4, 2022
@golang golang locked and limited conversation to collaborators Aug 4, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@dle8 dle8

@findleyr findleyr

Labels
FrozenDueToAge gopls/workspace Issues related to support for modules or multi-module workspaces. gopls Issues related to the Go language server, gopls. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Milestone
gopls/v0.9.2
Development

No branches or pull requests
4 participants
@slon @gopherbot @dle8 @findleyr