Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: "panic: runtime error: growslice: cap out of range" when creating slice from an unsafe.Pointer in another part of program #34288

Open
Lomanic opened this issue Sep 13, 2019 · 6 comments
Open

runtime: "panic: runtime error: growslice: cap out of range" when creating slice from an unsafe.Pointer in another part of program #34288

Lomanic opened this issue Sep 13, 2019 · 6 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@Lomanic
Copy link

Lomanic commented Sep 13, 2019 •
edited
Loading

What version of Go are you using (go version)?

$ go version
go version go1.13 windows/amd64

Does this issue reproduce with the latest release?

Yes (1.13, also reproducible on 1.12.9)

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\user\AppData\Local\go-build
set GOENV=C:\Users\user\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\user\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\user\AppData\Local\Temp\go-build617861358=/tmp/go-build -gno-record-gcc-switches

What did you do?

As we are calling specific win32 API not available on previous versions, Windows 8.1+ is needed to reproduce.

go get -u github.com/shirou/gopsutil/process
cd  "%HOME%\go\src\github.com\shirou\gopsutil\"
git remote add fork https://github.com/Lomanic/gopsutil.git
git checkout fork/issue250-bugreport
go run <path_to_main.go>

main.go

package main

import (
        "fmt"
        "os"
        "time"

        "github.com/shirou/gopsutil/process"
)

func main() {
        start := time.Now()
        procs, err := process.Processes()
        if err != nil {
                fmt.Println("err", err)
                os.Exit(1)
        }
        for _, proc := range procs {
                fmt.Println("PID", proc.Pid)
                fmt.Println(proc.Cmdline())
        }
        elapsed := time.Now().Sub(start)
        fmt.Println("Elapsed time", elapsed, "(", elapsed/(time.Duration(len(procs))), "per process).", len(procs), "processes")
}

What did you expect to see?

The commandline for all (non-elevated) processes on the host. Extract sample using WMI (current implementation in gopsutil master):

PID 4580
"C:\Users\user\Downloads\cmder\vendor\conemu-maximus5\ConEmu64.exe"  <nil>
PID 4616
 <nil>
PID 4652
"C:\Users\user\Downloads\cmder\vendor\conemu-maximus5\ConEmu\ConEmuC64.exe"  /CINMODE=600020 /AID=4636 /GID=4580 /GHWND=00030172 /BW=238 /BH=53 /BZ=1000 "/FN=Lucida Console" /FW=3 /FH=5 /TA=10100007 /HIDE /ROOT cmd /k ""%ConEmuDir%\..\init.bat" " <nil>
PID 4660
\??\C:\Windows\system32\conhost.exe 0x4 <nil>
PID 4756
C:\Users\user\AppData\Local\Temp\go-build614473794\b001\exe\proccmdline.exe <nil>
PID 4760
"C:\Windows\SYSTEM32\cmd.exe"  /k ""C:\Users\user\Downloads\cmder\vendor\conemu-maximus5\..\init.bat" " <nil>
PID 4852
"C:\Windows\SYSTEM32\cmd.exe"  /k ""C:\Users\user\Downloads\cmder\vendor\conemu-maximus5\..\init.bat" " <nil>
PID 4884
 <nil>
PID 4948
go  run proccmdline.go  <nil>
Elapsed time 3.5995457s ( 52.167328ms per process). 69 processes

What did you see instead?

When running as an elevated user, the main.go sample is able to retrieve every possible process commandline it can retrieve like the WMI implementation. The issue occurs with non-elevated user with a panic runtime as explained in shirou/gopsutil#250 (comment).

PID 0
 <nil>
PID 4
 <nil>
PID 68
 <nil>
PID 140
sihost.exe <nil>
PID 184
"C:\Windows\system32\cmd.exe"  <nil>
PID 304
 <nil>
PID 332
 <nil>
PID 392
 <nil>
PID 460
 <nil>
PID 468
 <nil>
PID 528
 <nil>
PID 552
 <nil>
PID 560
 <nil>
PID 672
 <nil>
PID 680
 <nil>
PID 688
 <nil>
PID 696
\??\C:\Windows\system32\conhost.exe 0x4 <nil>
PID 776
 <nil>
PID 784
"C:\Users\user\Downloads\cmder\vendor\conemu-maximus5\ConEmu64.exe"  <nil>
PID 856
panic: runtime error: growslice: cap out of range

goroutine 1 [running]:
fmt.(*buffer).writeString(...)
        c:/go/src/fmt/print.go:82
fmt.(*fmt).padString(0xc000044e10, 0x75006d0045006e, 0x65002e00340036)
        c:/go/src/fmt/format.go:110 +0xfa
fmt.(*fmt).fmtS(0xc000044e10, 0x75006d0045006e, 0x65002e00340036)
        c:/go/src/fmt/format.go:359 +0x68
fmt.(*pp).fmtString(0xc000044dd0, 0x75006d0045006e, 0x65002e00340036, 0xc000000076)
        c:/go/src/fmt/print.go:447 +0x138
fmt.(*pp).printArg(0xc000044dd0, 0x51bfc0, 0xc0000f8300, 0x76)
        c:/go/src/fmt/print.go:698 +0x87e
fmt.(*pp).doPrintln(0xc000044dd0, 0xc000075e80, 0x2, 0x2)
        c:/go/src/fmt/print.go:1173 +0xb9
fmt.Fprintln(0x578480, 0xc000006020, 0xc000075e80, 0x2, 0x2, 0x0, 0x0, 0x0)
        c:/go/src/fmt/print.go:264 +0x5f
fmt.Println(...)
        c:/go/src/fmt/print.go:274
main.main()
        C:/Users/user/go/src/github.com/Lomanic/gopsutiltest/proccmdline.go:25 +0x1b2
exit status 2

Commenting out this line doesn't trigger the panic and hints that the issue resides in the unicodeString.String() function (got it from an UNICODE_STRING implementation in hillu/go-ntdll). What's puzzling is that it works properly running as an elevated user and that the stacktrace looks completely unrelated (in fmt stdlib), the stacktrace appears even for a PID where buf.String() is not even called, hinting about some memory/stack corruption maybe?
@randall77
Copy link
Contributor

Your unicodeString.String method looks unsafe.
The docs of reflect.SliceHeader say:

the Data field is not sufficient to guarantee the data it references will not be garbage collected, so programs must keep a separate, correctly typed pointer to the underlying data.

Are you sure you're doing that?

A safer bet is to do:

s := (*[1e9]uint16)(unsafe.Pointer(u.Buffer))[:u.Length/2]

Then there's no chance a missing write barrier could corrupt the world (like I think might be happening with your code).

@Lomanic
Copy link
Author

Lomanic commented Sep 13, 2019

Thanks a lot for your help.

Unfortunately changing var s []uint16 to s := (*[1e9]uint16)(unsafe.Pointer(u.Buffer))[:u.Length/2] on this line doesn't fix the issue (exact same crash at runtime). Also tried defer runtime.KeepAlive(hdr) (and for s) to no effect.

diff --git a/process/process_windows.go b/process/process_windows.go
index 1abf7ac..3582b72 100644
--- a/process/process_windows.go
+++ b/process/process_windows.go
@@ -7,6 +7,7 @@ import (
        "fmt"
        "os"
        "reflect"
+       "runtime"
        "strings"
        "syscall"
        "time"
@@ -304,8 +305,10 @@ type unicodeString struct {
 }

 func (u unicodeString) String() string {
-       var s []uint16
+       s := (*[1e9]uint16)(unsafe.Pointer(u.Buffer))[:u.Length/2]
+       defer runtime.KeepAlive(s)
        hdr := (*reflect.SliceHeader)(unsafe.Pointer(&s))
+       defer runtime.KeepAlive(hdr)
        hdr.Data = uintptr(unsafe.Pointer(u.Buffer))
        hdr.Len = int(u.Length / 2)
        hdr.Cap = int(u.MaximumLength / 2)

@randall77
Copy link
Contributor

You want just:

func (u unicodeString) String() string {
	s := (*[1e9]uint16)(unsafe.Pointer(u.Buffer))[:u.Length/2]
	return string(utf16.Decode(s))
}

@Lomanic
Copy link
Author

Lomanic commented Sep 13, 2019

Same behavior when redefining the unicodeString.String() function like this unfortunately (and it even still works properly as an elevated user also).

@toothrot toothrot changed the title "panic: runtime error: growslice: cap out of range" when creating slice from an unsafe.Pointer in another part of program runtime: "panic: runtime error: growslice: cap out of range" when creating slice from an unsafe.Pointer in another part of program Sep 20, 2019
@toothrot toothrot added this to the Go1.14 milestone Sep 20, 2019
@toothrot toothrot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 20, 2019
@randall77
Copy link
Contributor

I don't know what the problem might be then.
Probably needs a Windows person to take a look.
@alexbrainman

@alexbrainman
Copy link
Member

Probably needs a Windows person to take a look.
@alexbrainman

I don't have time to debug this code.

Sorry.

Alex

@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 7, 2022
@mknyszek mknyszek moved this to Triage Backlog in Go Compiler / Runtime Jul 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
Go Compiler / Runtime
Status: Triage Backlog
Milestone
Backlog
Development

No branches or pull requests
6 participants
@toothrot @rsc @Lomanic @randall77 @gopherbot @alexbrainman