Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/tools/cmd/guru: implements can take extreme amount of resources (e.g. when run from vscode) depending on working directory #41243

Open
Dieterbe opened this issue Sep 6, 2020 · 1 comment
Open

x/tools/cmd/guru: implements can take extreme amount of resources (e.g. when run from vscode) depending on working directory #41243

Dieterbe opened this issue Sep 6, 2020 · 1 comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone
Unreleased

Comments

@Dieterbe
Copy link
Contributor

Dieterbe commented Sep 6, 2020 •
edited

Hello, first of all, thanks for the work that's been put into guru. It's certainly been quite helpful to me over the years.
I know guru is no longer actively maintained (and I don't expect a resolution), but considering gopls is still in alpha, i thought it might be helpful to others (or to my future self) to document this issue.

What version of Go are you using (go version)?

go version go1.15.1 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/dieter/.cache/go-build"
GOENV="/home/dieter/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/dieter/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/dieter/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/dieter/go-build095063144=/tmp/go-build -gno-record-gcc-switches"

I found out that based on the working directory when guru implements is run, it can take an extreme amount of cpu and take a very long time.

In particular, consider these 3 types of runs:

1) cd /home/dieter && \
    guru -json implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'
2) cd /home/dieter/go/src/github.com/prometheus/prometheus && \
    guru -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
3) cd /home/dieter/go/src/github.com/prometheus/prometheus && \
    guru -json -scope github.com/prometheus/prometheus/... implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'

only 1) returns in a reasonable amount of time. 2) is how visual studio code invokes it (when CWD is $GOPATH/src/github.com/prometheus/prometheus), and 3) is my attempt to fix the problem with 2) by constraining the scope.

More info about each run:

Note:
I moved guru to guru-real and replaced guru with this script:

$ cat /home/dieter/go/bin/guru
#!/bin/bash
file="/home/dieter/guru-$(date '+%F_%T')"
echo "guru $@" > $file
env >> $file
guru-real "$@"

this allows me to track the exact environment of the process each time it is run (particularly when the parent is vscode and not my terminal), but it turns out the issue was reproducible perfectly from a terminal, so at this point it's no longer useful, but it does explain why i have mentions of "guru-real" in the pstree output below.

run 1

time /home/dieter/go/bin/guru -json implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'
{
	"type": {
		"name": "github.com/prometheus/prometheus/storage.Storage",
		"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:43:6",
		"kind": "interface"
	},
	"to": [
		{
			"name": "*github.com/prometheus/prometheus/cmd/prometheus.readyStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:898:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/storage.fanout",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout.go:33:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/storage/remote.Storage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:48:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/tsdb.DB",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/tsdb/db.go:120:6",
			"kind": "pointer"
		},
		{
			"name": "github.com/prometheus/prometheus/storage/fanout.errStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout/fanout_test.go:152:6",
			"kind": "struct"
		},
		{
			"name": "github.com/prometheus/prometheus/util/teststorage.testStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/util/teststorage/storage.go:46:6",
			"kind": "struct"
		}
	],
	"from": [
		{
			"name": "github.com/cortexproject/cortex/vendor/github.com/prometheus/common/expfmt.Closer",
			"pos": "/home/dieter/go/src/github.com/cortexproject/cortex/vendor/github.com/prometheus/common/expfmt/encode.go:40:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/storage.Appendable",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:35:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/storage.Queryable",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:56:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt.Closer",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt/encode.go:40:6",
			"kind": "interface"
		},
		{
			"name": "io.Closer",
			"pos": "/usr/lib/go/src/io/io.go:98:6",
			"kind": "interface"
		}
	]
}
/home/dieter/go/bin/guru -json implements   95.38s user 26.11s system 733% cpu 16.554 total

pstree -a -t -l | less
  |   |   `-guru /home/dieter/go/bin/guru -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |       `-guru-real -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |           `-13*[{guru-real}]
run 2

I didn't let it complete. I killed it after it ran for hours.

pstree -a -t -l | less
  |   |   `-guru /home/dieter/go/bin/guru -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |       `-guru-real -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/share
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/hello
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/shape
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/peter-rabbit
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/showdeps
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/share2
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/go.tools/godoc/redirect
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/go.tools/godoc/static
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/unsafeunmarshaler
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/marshalto
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/populate
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/stringer
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/unmarshal
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/addons/kube-proxy
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/gce/manifests
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/juju/prereqs
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/kubemark/pre-existing
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/docs/yaml/kubectl
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cmd/kube-apiserver/app
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cmd/kube-controller-manager/app
  |   |           |   `-11*[{go}]
  |   |           `-35*[{guru-real}]

run 3

time /home/dieter/go/bin/guru -json -scope github.com/prometheus/prometheus/... implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'
{
	"type": {
		"name": "github.com/prometheus/prometheus/storage.Storage",
		"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:43:6",
		"kind": "interface"
	},
	"to": [
		{
			"name": "*github.com/prometheus/prometheus/cmd/prometheus.readyStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:898:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/storage.fanout",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout.go:33:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/storage/remote.Storage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:48:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/tsdb.DB",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/tsdb/db.go:120:6",
			"kind": "pointer"
		},
		{
			"name": "github.com/prometheus/prometheus/storage/fanout.errStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout/fanout_test.go:152:6",
			"kind": "struct"
		},
		{
			"name": "github.com/prometheus/prometheus/util/teststorage.testStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/util/teststorage/storage.go:46:6",
			"kind": "struct"
		}
	],
	"from": [
		{
			"name": "github.com/prometheus/common/expfmt.Closer",
			"pos": "/home/dieter/go/pkg/mod/github.com/prometheus/common@v0.10.0/expfmt/encode.go:40:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/storage.Appendable",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:35:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/storage.Queryable",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:56:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt.Closer",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt/encode.go:40:6",
			"kind": "interface"
		},
		{
			"name": "io.Closer",
			"pos": "/usr/lib/go/src/io/io.go:98:6",
			"kind": "interface"
		}
	]
}
/home/dieter/go/bin/guru -json -scope github.com/prometheus/prometheus/...    5478.10s user 886.50s system 246% cpu 42:57.99 total

$ pstree -a -t -l
  |   |   `-guru /home/dieter/go/bin/guru -json -scope github.com/prometheus/prometheus/... implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |       `-guru-real -json -scope github.com/prometheus/prometheus/... implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- google.golang.org/protobuf/internal/descopts
  |   |           |   `-10*[{go}]
  |   |           |-(go)
  |   |           |   `-{go}
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- google.golang.org/protobuf/runtime/protoimpl
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/apimachinery/pkg/util/intstr
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/apimachinery/pkg/apis/meta/v1
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- google.golang.org/protobuf/internal/flags
  |   |           |   `-9*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/apimachinery/pkg/runtime/schema
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- github.com/prometheus/prometheus/vendor/k8s.io/client-go/plugin/pkg/client
  |   |           |   `-10*[{go}]
  |   |           `-19*[{guru-real}]
@gopherbot gopherbot added the Tools This label describes issues relating to any tools in the x/tools repository. label Sep 6, 2020
@gopherbot gopherbot added this to the Unreleased milestone Sep 6, 2020
@stamblerre
Copy link
Contributor

Thank you for this thorough investigation, but you are right to say that we will not be fixing guru issues. gopls is close to reaching beta, so it is the recommended alternative. If you are interested in using CLI tools, you can also try gopls implementation command, which will tell you both the implementations of a given interface or the interfaces implemented by a concrete type.

@andybons andybons added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 8, 2020
@seankhliao seankhliao changed the title x/tools/cmd/guru implements can take extreme amount of resources (e.g. when run from vscode) depending on working directory x/tools/cmd/guru: implements can take extreme amount of resources (e.g. when run from vscode) depending on working directory Jun 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
4 participants
@Dieterbe @andybons @stamblerre @gopherbot