cmd/compile: big array not allocated on heap, which make program panic. #20021
Comments
|
Worth noting that the spec does not talk of heap nor stack, so if this is a bug it would likely be a compiler one. Did you try forcing an allocation at run-time, like |
ianlancetaylor
changed the title
package main
const N = 1000 * 1000 * 537
func main() {
// make is ok
p := make([]byte, N)
q := make([]byte, N)
_, _ = p, q
// new is also ok
var u = new([N]byte)
var v = new([N]byte)
_, _ = u, v
// blow also stack overflow
var x [N]byte
var y [N]byte
_, _ = x, y
} |
package main
const N = 1000 * 1000 * 537
var a [N]byte
func main() {
for _, v := range a {
_ = v
}
}This is a typical trap that the #19817 should address. The workaround is to rewrite the line with for _, v := range a[:] { |
|
Go tip (as of 992ce90) has a better error message: So, some progress. But bumping to Go 1.11. |
bradfitz
added
the
NeedsFix
|
As of Go 1.12, it does not crash if the code is for _, v := range a {
_ = v
}But if the variable is assigned to something else and printed later - package main
import "fmt"
const N = 1000 * 1000 * 537
var a [N]byte
var tmp byte
func main() {
for _, v := range a {
tmp = v
}
fmt.Println(tmp)
}it goes back to the original error Same behavior with tip (go version devel +f947c4dcfe Fri Apr 5 00:52:55 2019 +0000 linux/amd64) |
|
I believe this is, or is closely related to, #27447. |
|
It looks this problem has already been fixed. Anyone may close it if it is verified. |
|
It looks there are still some cases which might be hard to detected at compile time. An example: package main
const N = 1000 * 1000 * 100
var a [N]byte
var n int
func g(x interface{}, n int) {
type _ int // avoid g being inlined
if n > 0 {
var y = a // moved to heap: y
g(y, n-1) // y does not escape
}
}
func main() {
var x = a // moved to heap: x
g(x, 4) // x does not escape
}But the following program doesn't overflow stack: package main
const N = 1000 * 1000 * 100
var a [N]byte
var n int
func g(x interface{}, n int) {
type _ int // avoid g being inlined
if n > 0 {
g(a, n-1) // a does not escape
}
}
func main() {
g(a, 100) // a does not escape
} |
|
A simpler one, which almost makes my 8g notetbook hang. package main
const N = 1000 * 1000 * 513
var a [N]byte
func g(vs ...interface{}) {
type _ int // avoid g being inlined
println(len(vs))
}
func main() {
g(a, a, a, a, a)
} |
|
Every assignment in Go is a copy, passing It would be interesting to see the output of |
|
It would be better to allocate the 5 copies of a to heap. However, It looks the gc compiler has changed the strategy much since version 1.8. |
|
BTW, maybe the compiler should make an optimization for var i interface{} = a
g(i, i, i, i, i) |
I think this might be a reporting error, 512mb values are too large for the stack. |
|
You are right, which could be proved by the following code. package main
import t "testing"
const N = 1000 * 1000 * 1 // 513
var a [N]byte
var k int
func g(vs ...interface{}) {
type _ int // avoid g being inlined
_ = vs[k]
}
func main() {
var y int
println(&y) // 0xc000045f60
x := t.AllocsPerRun(1, func() {
g(a, a, a, a)
})
println(int(x)) // 4
println(&y) // 0xc000045f60
}One wired thing is, if |
|
So many weird things in the following code: package main
import t "testing"
const N = 1000 * 1000 * 100
var a [N]byte
var n int
func g(x interface{}, n int) {
type _ int // avoid g being inlined
var y = a // moved to heap: y
if n > 0 {
g(y, n-1) // y does not escape
panic("unreachable")
}
}
func main() {
var x int
println(&x) // 0xc000045f68
z := t.AllocsPerRun(1, func() {
g(nil, 0)
})
println(int(z)) // 2
println(&x) // 0xc00dffbf68
}Weirdness 1: why 2 allocations in the call
|
I only get 1 when I run your code. (There is a second allocation site for converting
Probably because |
It is always 2 on my machine. But if I change the first argument of
It prints: The second run only allocates once. If I add a |
Remove the |
Note that "a does not escape" AFAIK does not mean there is an allocation on the stack. By looking at Each time |
g itself uses a huge amount of stack: looking at This would also explain why moving |
|
The following program sometimes prints 1, sometimes prints 2 on my machine. package main
import "testing"
const N = 10 * 1024 * 1024 // 10M
func declare10Mplus1() {
var a [N+1]byte // moved to heap: a
_ = a
}
func main() {
stat := func( f func() ) int {
allocs := testing.AllocsPerRun(1, f)
return int(allocs)
}
println(stat(declare10Mplus1))
} |
gopherbot
added
the
compiler/runtime
Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (
go version)?go version go1.8.1 linux/amd64
What did you do?
[edit]: now(Go SDK 1.12.2), the above program doesn't crash. But below @agnivade shows one case which will still crash.
What did you expect to see?
runs ok
What did you see instead?
crash
The text was updated successfully, but these errors were encountered: