x/tools/cmd/godoc: godoc HTTP server taking > 8 GB of memory #17344
Comments
|
Why guess? Move |
bradfitz
added
Suggested
|
Removing |
|
Back up to 8.63 GB. |
|
Removed |
|
Could symlinks cause issues here? |
|
I've added some profiling on godoc with go-issue-mirror installed in $GOPATH and using the command line provided in reproduction. A fair amount of time is spent in GC on the cpu profile. There's a good amount of object creation in the indexer paths which started leading me towards GC or memory allocation issues, but I don't have enough expertise in those parts of Go. If there's a tool that can do like cpu profile for memory allocation that would be great. I tried passing it through valgrind but can't seem to get debugging info from the go files in question to give me caller info to track down more on.
|
|
@cwgem Thanks for doing this! I'm not familiar with this process – can you also run a memory profile? It was using significant memory. |
|
With the following $GOPATH usage, I was seeing 1.2GB of memory being used by godoc. |
|
What's the status here? Has anybody investigated this further? I see a CPU profile above. Has anybody done a memory profile? |
|
@bradfitz This is still happening for me. I pulled some info from |
|
@parkr, older versions of Go's memory samples weren't self-contained. Can you either include the binary or ideally use Go 1.8? |
|
@bradfitz My apologies! I just updated the gist above with the new data after re-compiling godoc via |
|
Here is some more info. Current $GOPATH usage. Pprof Output: Here is the svg output.
|
|
@parkr was this fixed by #21061? Your comment (#17344 (comment)) suggested you may have been affected by the symlink bug. This can probably be closed regardless? I don't think there's enough information here for anyone else to reproduce. |
|
That looks likely! I stopped using godoc as much because of this. I’ll give it a whirl in the latest go tip. Thanks, all! |
|
@bradfitz I still have the problem of godoc using more memory than it seems like it should use. Should I open a new issue? |
|
The memory usage is proportional to the size of your GOPATH. If you think it is using more memory than it should, then yes please open a new issue with all relevant details possible that can help us debug the issue. Thanks. |
|
@agnivade When you say,
Is there a way to calculate what a sane value would be? My current GOPATH usage is 134M, but |
|
Most possibly, you have the search index enabled. In that case, memory usage can be significantly high. I can't deduce anything more without further info. I suggest that you open a new issue with all the details if you believe there is a bug somewhere. |
I tweeted about this yesterday and it was suggested by @adg that I file this bug here.
What version of Go are you using (
go version)?go version go1.7.1 darwin/amd64
What operating system and processor architecture are you using (
go env)?What did you do?
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.
Command run:
/usr/local/opt/go/libexec/bin/godoc -http=localhost:6060 -index -index_throttle=0.25 -index_interval=1hGOPATH is fairly large:
Size of various folders & files in GOPATH
github.com/bradfitz/go-issue-mirroris 57 MB of Git data, and 414 MB of.jsonfiles inissues/. I forked that togithub.com/parkr/jekyll-issue-mirrorwhich is 46 MB of Git data and 6.9 MB of.jsonfiles.What did you expect to see?
I expected to see the process take less memory. I expected to see the process's memory stay stable over time if I make no modifications to the contents of my GOPATH.
What did you see instead?
It is using well over 4 times as much memory as it takes on disk. It continues to grow over time.
My hunch is that it's indexing these
.jsonfiles, or the.gitdata.The text was updated successfully, but these errors were encountered: