runtime: possible map-related memory leak #5864
Labels
Comments
package main
import (
"math/rand"
"runtime"
)
func run2(){
m:=make(map[int32]int32)
for i:=0;i<1000000;i++{
m[rand.Int31()]=rand.Int31()
}
}
func main() {
for i:=0;i<100;i++ {
run2()
runtime.GC()
}
}
//this code will prevent leak, however, without runtime.GC(), the memory never stops
growing
|
I believe this was fixed when issue #6119 was closed. Please confirm if the problem has been fixed for you. Status changed to WaitingForReply. |
I run hg pull and ./all.bash
I re-run the code. The memory still keep growing.
The code I run is
package main
import (
"math/rand"
)
func run2(){
m:=make(map[int32]int32)
for i:=0;i<1000000;i++{
m[rand.Int31()]=rand.Int31()
}
}
func main() {
for i:=0;i<100;i++ {
run2()
}
}
|
What does 'go version' print, before you build the program? How do you run or build the program? I ran this on Linux/amd64 at tip, at 3338bbc9c09b, and at go1.1.2. In go1.1.2 it does in fact grow without bound. At tip and at 3338bbc9c09b it does not; it tops out around 100 MB. I believe that somehow your test of re-running the program was still running an old copy, either because you are using a different 'go' command than you think, or because the program was not rebuilt after the update. Labels changed: removed go1.2maybe. |
I've tested on both darwin/amd64 and linux/amd64 and don't see this problem at tip. The process grows to 104MB and is quite stable until the program completes. GC seems to be working fine. gc19(1): 14+2+0 ms, 77 -> 37 MB 89651 -> 29086 (657207-628121) objects, 0(0) handoff, 0(0) steal, 0/0/0 yields gc20(1): 14+2+0 ms, 82 -> 37 MB 89224 -> 29031 (717345-688314) objects, 0(0) handoff, 0(0) steal, 0/0/0 yields gc21(1): 14+2+0 ms, 82 -> 37 MB 89214 -> 29133 (777528-748395) objects, 0(0) handoff, 0(0) steal, 0/0/0 yields gc22(1): 14+2+0 ms, 82 -> 37 MB 89511 -> 29254 (837906-808652) objects, 0(0) handoff, 0(0) steal, 0/0/0 yields gc23(1): 14+2+0 ms, 82 -> 37 MB 89448 -> 29110 (898100-868990) objects, 0(0) handoff, 0(0) steal, 0/0/0 yields gc24(1): 14+2+0 ms, 82 -> 37 MB 89395 -> 29153 (958385-929232) objects, 0(0) handoff, 0(0) steal, 0/0/0 yields 8d19586af772+ tip |
Possibly it was related to issue #6119, the GC would occasionally conservatively scan the internal map structures, causing otherwise unreachable objects to be retained. Labels changed: removed priority-later. Status changed to Retracted. |
This issue was closed.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
by lionghostshop:
Thanks. What steps will reproduce the problem? If possible, include a link to a program on play.golang.org. Run code below package main import ( "text/scanner" "os" //"io" "math/rand" "fmt" "time" //"strconv" "runtime/pprof" ) func run() int { var cin scanner.Scanner start:=time.Now().UnixNano() f,_:=os.Open("/dev/shm/benchmark.txt") defer f.Close() cin.Init(f) tok:=cin.Scan() m:=make(map[string]string) for tok!=scanner.EOF{ key:=cin.TokenText() cin.Scan() val:=cin.TokenText() //fmt.Printf("(%s) (%s)\n",key,val) m[key]=val //delete(m,key) tok=cin.Scan() } end:=time.Now().UnixNano() fmt.Printf("First half %d\n", (end-start)/1000000 ) start=time.Now().UnixNano() sum:=0 for key,_:=range m{ a,_:=strconv.Atoi(key) b,_:=strconv.Atoi(val) //a:=1 //b:=2 sum+=a*b //delete(m,key) } end=time.Now().UnixNano() fmt.Printf("Second half %d\n", (end-start)/1000000 ) m=nil return sum } func main() { f,_:=os.OpenFile("/dev/shm/benchmark.txt",os.O_APPEND|os.O_CREATE|os.O_TRUNC|os.O_WRONLY,0600) for i:=0;i<1000000;i++{ f.WriteString(fmt.Sprintf("%d %d\n",rand.Int31(),rand.Int31() )) } f.Close() sum:=0 for i:=0;i<100;i++ { sum+=run() if i == 10 { w,_:=os.OpenFile("memory.dump",os.O_APPEND|os.O_CREATE|os.O_TRUNC|os.O_WRONLY,0600) pprof.WriteHeapProfile(w) w.Close() // break } } fmt.Println(sum) } What is the expected output? The memory should not grow continuously What do you see instead? The memory grows continuously Which compiler are you using (5g, 6g, 8g, gccgo)? AMD 64 bit go under linux Which operating system are you using? Fedora 19 64 bit Which version are you using? (run 'go version') 1.1.1 In fact, I checked out the source code. So the version should be slightly newer than 1.1.1 Please provide any additional information below.The text was updated successfully, but these errors were encountered: