Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: slice allocation with 1<<33 cap not showing up in profile #33342

Closed
zeast opened this issue Jul 29, 2019 · 2 comments
Closed

runtime: slice allocation with 1<<33 cap not showing up in profile #33342

zeast opened this issue Jul 29, 2019 · 2 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@zeast
Copy link

zeast commented Jul 29, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.1 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"

What did you do?

func main() {
	go func() {
		l, _ := net.Listen("tcp", "0.0.0.0:8989")
		http.Serve(l, http.DefaultServeMux)
	}()

	bigMem := make([]byte, 0, 1<<33)

	time.Sleep(time.Hour)

	_ = bigMem
}

runnable program: https://play.golang.org/p/RPebfcdzpvI

What did you expect to see?

In normal, I think bigMem will be shown when I use go tool pprof xxx/heap
The program has been simplified, In my case bigMem in other package, and I want to find who use the memory from a lot of heap recordes.

What did you see instead?

I did't see information about bigMem

What I found

I think the reason in malloc.go +1014

	if rate := MemProfileRate; rate > 0 {
		if rate != 1 && int32(size) < c.next_sample {
			c.next_sample -= int32(size)
		} else {
			mp := acquirem()
			profilealloc(mp, x, size)
			releasem(mp)
		}
	}

In fact size bigger than maxint32, so int32(size) maybe negative or zero, and never be recorded, perhaps definition c.next_sample uint64 is better ? or always record when size bigger than a threshold.

@ALTree ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 29, 2019
@ALTree ALTree added this to the Go1.14 milestone Jul 29, 2019
@ALTree ALTree changed the title pprof/heap maybe missing record pprof/heap: slice allocation with 1<<33 cap not showing up in profile Jul 29, 2019
@ALTree ALTree changed the title pprof/heap: slice allocation with 1<<33 cap not showing up in profile cmd/pprof: slice allocation with 1<<33 cap not showing up in profile Jul 29, 2019
@ianlancetaylor ianlancetaylor changed the title cmd/pprof: slice allocation with 1<<33 cap not showing up in profile runtime: slice allocation with 1<<33 cap not showing up in profile Jul 29, 2019
@randall77
Copy link
Contributor

I think changing the type to uintptr is the right fix.
Testing is a bit challenging - we'd need a multi-gig machine, which isn't always available.

@gopherbot
Copy link

Change https://golang.org/cl/188017 mentions this issue: runtime: use uintptr instead of int32 for counting to next heap profile sample

@golang golang locked and limited conversation to collaborators Jul 28, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

4 participants