Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/tools/go/packages: high CPU usage while processing cgo file #32821

Open
RadhiFadlillah opened this issue Jun 28, 2019 · 5 comments
Open

x/tools/go/packages: high CPU usage while processing cgo file #32821

RadhiFadlillah opened this issue Jun 28, 2019 · 5 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Performance Tools This label describes issues relating to any tools in the x/tools repository.
Milestone
Unreleased

Comments

@RadhiFadlillah
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.12.5 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN="/home/radhi/Go/bin"
GOCACHE="/home/radhi/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/radhi/Go"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build763115998=/tmp/go-build -gno-record-gcc-switches"

What did you do?

This issue is related with vscode-go#2598. While working on cgo project, both gocode and gopls will use huge amount of CPU when making autocomplete suggestion. It can be seen here for gocode and here for gopls.

After looking around, I found out the reason for this is because both gocode and gopls uses LoadSyntax mode, which make go/packages run go list with -compiled flag set to true, which lead to high CPU usage in cgo project.

What did you expect to see?

The cgo file can be parsed as light as the normal Go code.

What did you see instead?

The process uses huge amount of CPU.
@gopherbot gopherbot added this to the Unreleased milestone Jun 28, 2019
@RadhiFadlillah RadhiFadlillah mentioned this issue Jun 28, 2019
Closed
@stamblerre stamblerre changed the title x/tools/go/packages: High CPU usage while processing cgo file x/tools/go/packages: high CPU usage while processing cgo file Jun 28, 2019
@bcmills
Copy link
Contributor

bcmills commented Jul 1, 2019

CC @matloob @ianthehat

@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Performance labels Jul 1, 2019
@bcmills
Copy link
Contributor

bcmills commented Jul 1, 2019

I'm going to assume for now that this is something that can be addressed within go/packages, but please let me and @jayconrod know if this is looking like a bug in go list -compiled instead.

@matloob
Copy link
Contributor

matloob commented Jul 1, 2019

When requesting source in go/packages, we always return the cgo-processed source code. That's where we chose to draw the abstraction boundary. So I don't think there's too much we can do from the go/packages side. We're going to have to run go list -compiled, and once go list runs, go/packages does minimal work to copy the outputs of go/packages into the Packages struct it returns in a Load call (since we don't need to do parsing/typechecking/etc).

The one strange thing is that it's my understanding that the LSP gets the package information only once (unless new packages are opened) so there shouldn't be consistent high cpu usage.

@stamblerre
Copy link
Contributor

stamblerre commented Jul 1, 2019 •
edited

gopls will run go/packages whenever imports are changed, so it could be happening more frequently.

@segevfiner
Copy link
Contributor

Probably related if not duplicate of #29427

@gopherbot gopherbot added the Tools This label describes issues relating to any tools in the x/tools repository. label Sep 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Performance Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
6 participants
@bcmills @stamblerre @RadhiFadlillah @gopherbot @matloob @segevfiner