Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/pprof: runtime error: slice bounds out of range on "view source" #46636

Closed
prattmic opened this issue Jun 7, 2021 · 7 comments
Closed

cmd/pprof: runtime error: slice bounds out of range on "view source" #46636

prattmic opened this issue Jun 7, 2021 · 7 comments
Assignees
@prattmic
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Milestone
Go1.17

Comments

@prattmic
Copy link
Member

prattmic commented Jun 7, 2021

Reported at https://groups.google.com/g/golang-dev/c/MH_e2xnSWAE/m/cCPpbtb8AQAJ

What version of Go are you using (go version)?

$ go version
go version devel go1.17-8212707871 Mon Jun 7 15:19:57 2021 +0000 linux/amd64

Does this issue reproduce with the latest release?

No

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/usr/local/google/home/mpratt/.cache/go-build"
GOENV="/usr/local/google/home/mpratt/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/usr/local/google/home/mpratt/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/usr/local/google/home/mpratt/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/google/home/mpratt/src/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/google/home/mpratt/src/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="devel go1.17-8212707871 Mon Jun 7 15:19:57 2021 +0000"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3999737285=/tmp/go-build -gno-record-gcc-switches"

What did you do?

https://play.golang.org/p/ASWkOjkKSN-

$ go tool pprof -http :8080 ./prof cpu.profile, then browse to http://localhost:8080/ui/source

What did you expect to see?

Source

What did you see instead?

2021/06/07 12:56:09 http: panic serving 10.129.52.196:49259: runtime error: slice bounds out of range [:18446744073705353216] with capacity 663772
goroutine 6 [running]:                                                                                                                       
net/http.(*conn).serve.func1()                                                                                                               
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:1801 +0xb9                                                 
panic({0x8e0580, 0xc0000e1770})                                                                                                              
        /usr/local/google/home/mpratt/src/go/src/runtime/panic.go:1047 +0x266     
cmd/internal/objfile.(*Disasm).Decode(0xc000119920, 0xc00031a530, 0x6, {0x0, 0x1, 0xc000198fd0}, 0xd9, 0x30)
        /usr/local/google/home/mpratt/src/go/src/cmd/internal/objfile/disasm.go:276 +0x42a                                         
main.(*objTool).Disasm(0x1, {0xc00031a530, 0x0}, 0xc00036f320, 0xc000199130, 0xd0)                                    
        /usr/local/google/home/mpratt/src/go/src/cmd/pprof/pprof.go:183 +0x9d                                            
cmd/vendor/github.com/google/pprof/driver.(*internalObjTool).Disasm(0xc000312f50, {0xc00031a530, 0x2}, 0x0, 0x1000067049a8c18, 0x0)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/driver/driver.go:273 +0x3b                               
cmd/vendor/github.com/google/pprof/internal/report.(*sourcePrinter).expandAddresses(0xc00037c140, 0xc000314520, 0x0, 0x0)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:352 +0x376              
cmd/vendor/github.com/google/pprof/internal/report.newSourcePrinter(0xc000314520, {0x9f8118, 0xc000119930}, {0xc00002e0f4, 0x27})
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:312 +0x5f7       
cmd/vendor/github.com/google/pprof/internal/report.PrintWebList({0x9f1320, 0xc00036eae0}, 0xc000314520, {0x9f8118, 0xc000119930}, 0x2)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:208 +0xcc
cmd/vendor/github.com/google/pprof/internal/driver.(*webInterface).source(0xc000243740, {0x9fa330, 0xc0003222a0}, 0xc0003b2200)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/driver/webui.go:397 +0x178                      
net/http.HandlerFunc.ServeHTTP(0x88fce0, {0x9fa330, 0xc0003222a0}, 0x7)                                               
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2046 +0x2f                                                               
cmd/vendor/github.com/google/pprof/internal/driver.defaultWebServer.func1({0x9fa330, 0xc0003222a0}, 0xc0003b2200)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/driver/webui.go:190 +0xb7                                                                                                                                                                    
net/http.HandlerFunc.ServeHTTP(0x4d1bd3, {0x9fa330, 0xc0003222a0}, 0x72)                                                                     
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2046 +0x2f                                                               
net/http.StripPrefix.func1({0x9fa330, 0xc0003222a0}, 0xc0002b8300)                                                                           
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2089 +0x330                                                              
net/http.HandlerFunc.ServeHTTP(0xc000199b40, {0x9fa330, 0xc0003222a0}, 0x68c60f)
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2046 +0x2f                              
net/http.(*ServeMux).ServeHTTP(0x4681ca, {0x9fa330, 0xc0003222a0}, 0xc0002b8300)          
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2424 +0x149   
net/http.serverHandler.ServeHTTP({0xc000094570}, {0x9fa330, 0xc0003222a0}, 0x6)                                                      
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2865 +0x8e                                                     
net/http.(*conn).serve(0xc0000960a0, {0x9fcf80, 0xc000243d70})                                                                               
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:1929 +0xb08                                                              
created by net/http.(*Server).Serve
        /usr/local/google/home/mpratt/src/go/src/net/http/server.go:2991 +0x4e8
@prattmic prattmic added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker labels Jun 7, 2021
@prattmic prattmic self-assigned this Jun 7, 2021
@prattmic
Copy link
Member Author

prattmic commented Jun 7, 2021

Same thing without the webserver complexity:

$ go tool pprof -weblist . ./prof cpu.profile
panic: runtime error: slice bounds out of range [:18446744073705353216] with capacity 759594

goroutine 1 [running]:
cmd/internal/objfile.(*Disasm).Decode(0xc000073930, 0xc00002b7f8, 0x6, {0x0, 0x1, 0xc0001731f8}, 0xd9, 0x30)
        /usr/local/google/home/mpratt/src/go/src/cmd/internal/objfile/disasm.go:276 +0x42a
main.(*objTool).Disasm(0x1, {0xc00002b7f8, 0x0}, 0xc00020c4e0, 0xc000173358, 0xf8)
        /usr/local/google/home/mpratt/src/go/src/cmd/pprof/pprof.go:183 +0x9d
cmd/vendor/github.com/google/pprof/driver.(*internalObjTool).Disasm(0xc000073f50, {0xc00002b7f8, 0x2}, 0x0, 0x100004b36650fd8, 0x0)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/driver/driver.go:273 +0x3b
cmd/vendor/github.com/google/pprof/internal/report.(*sourcePrinter).expandAddresses(0xc000021040, 0xc00007d220, 0x0, 0x0)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:352 +0x376
cmd/vendor/github.com/google/pprof/internal/report.newSourcePrinter(0xc00007d220, {0x9f8118, 0xc000073940}, {0xc00002e0f4, 0x27})
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:312 +0x5f7
cmd/vendor/github.com/google/pprof/internal/report.PrintWebList({0x9f1320, 0xc0001afad0}, 0xc00007d220, {0x9f8118, 0xc000073940}, 0x8)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:208 +0xcc
cmd/vendor/github.com/google/pprof/internal/report.printWebSource({0x9f1320, 0xc0001afad0}, 0xc0001b1540, {0x9f8118, 0xc000073940})
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/source.go:125 +0x5b
cmd/vendor/github.com/google/pprof/internal/report.Generate({0x9f1320, 0xc0001afad0}, 0x0, {0x9f8118, 0xc000073940})
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/report/report.go:115 +0x1c5
cmd/vendor/github.com/google/pprof/internal/driver.generateReport(_, {_, _, _}, {{0x0, 0x0}, 0x0, 0x0, {0x9123ec, 0x7}, ...}, ...)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/driver/driver.go:117 +0xd4
cmd/vendor/github.com/google/pprof/internal/driver.PProf(0xc000176980)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/internal/driver/driver.go:53 +0x225
cmd/vendor/github.com/google/pprof/driver.PProf(0x4601db)
        /usr/local/google/home/mpratt/src/go/src/cmd/vendor/github.com/google/pprof/driver/driver.go:33 +0x1e
main.main()
        /usr/local/google/home/mpratt/src/go/src/cmd/pprof/pprof.go:38 +0xc5

@cherrymui
Copy link
Member

It is interesting that list doesn't panic, whereas weblist does...

@cherrymui cherrymui added this to the Go1.17 milestone Jun 7, 2021
@prattmic
Copy link
Member Author

prattmic commented Jun 7, 2021 •
edited

I don't think list invokes the disassembler, while weblist does.

The immediate problem seems to be that vendored pprof is invoking cmd/pprof.objTool.Disasm with start 0x401000, end 0x0.

Still digging into why that is.

@ALTree
Copy link
Member

ALTree commented Jun 7, 2021

A possible culprit is 68327e1. Does reverting it fixes it?

@prattmic
Copy link
Member Author

prattmic commented Jun 7, 2021

I believe the problem is that https://go-review.googlesource.com/c/go/+/318049/3/src/cmd/pprof/pprof.go makes ObjAddr unconditionally return 0, but https://go-review.googlesource.com/c/go/+/318049/3/src/cmd/vendor/github.com/google/pprof/internal/report/source.go passes the result of ObjAddr to Disasm as-is. I believe that ObjAddr should return addr to remain a no-op as before.

@prattmic
Copy link
Member Author

prattmic commented Jun 7, 2021

Yup, that does it. I'll send a CL.

@gopherbot
Copy link

Change https://golang.org/cl/325809 mentions this issue: cmd/pprof: make ObjAddr a no-op

@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jun 7, 2021
@heschi heschi added okay-after-beta1 Used by release team to mark a release-blocker issue as okay to resolve either before or after beta1 and removed okay-after-beta1 Used by release team to mark a release-blocker issue as okay to resolve either before or after beta1 labels Jun 8, 2021
@hyangah hyangah mentioned this issue Jun 11, 2021
Closed
@golang golang locked and limited conversation to collaborators Jun 17, 2022
@prattmic prattmic self-assigned this Jun 24, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@prattmic prattmic

Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Projects
None yet
Milestone
Go1.17
Development

No branches or pull requests
6 participants
@prattmic @dmitshur @ALTree @gopherbot @cherrymui @heschi