Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: autotemps can make stack frame too large #27447

Open
randall77 opened this issue Sep 2, 2018 · 5 comments
Open

cmd/compile: autotemps can make stack frame too large #27447

randall77 opened this issue Sep 2, 2018 · 5 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@randall77
Copy link
Contributor

randall77 commented Sep 2, 2018

When the order pass introduces temporaries, it always allocates them on the stack, even if they are too big for the stack.

package main

import "reflect"

func f() {
	g(reflect.TypeOf([2000000000]*byte{}))
}

//go:noescape
func g(p reflect.Type)

We should use the same rules as escape analysis does to decide if we should put the temporary on the stack or the heap.

Before order:

.   .   .   AS2-list
.   .   .   .   NAME-reflect.i a(true) l(1376) x(0) class(PAUTO) tc(1) addrtaken assigned used INTER-interface {}
.   .   .   AS2-rlist
.   .   .   .   CONVIFACE l(6) esc(no) tc(1) implicit(true) INTER-interface {}
.   .   .   .   .   ARRAYLIT l(6) esc(h) tc(1) ARRAY-[2000000000]*byte

After order:

.   AS l(6) tc(1)
.   .   NAME-main..autotmp_5 a(true) l(6) x(0) class(PAUTO) esc(N) tc(1) assigned used ARRAY-[2000000000]*byte
.   .   ARRAYLIT l(6) esc(h) tc(1) ARRAY-[2000000000]*byte

.   AS2 l(6) tc(1)
.   AS2-list
.   .   NAME-reflect.i a(true) l(1376) x(0) class(PAUTO) tc(1) addrtaken assigned used INTER-interface {}
.   AS2-rlist
.   .   CONVIFACE l(6) esc(no) tc(1) implicit(true) INTER-interface {}
.   .   .   NAME-main..autotmp_5 a(true) l(6) x(0) class(PAUTO) esc(N) tc(1) assigned used ARRAY-[2000000000]*byte

.   VARKILL l(6) tc(1)
.   .   NAME-main..autotmp_5 a(true) l(6) x(0) class(PAUTO) esc(N) tc(1) assigned used ARRAY-[2000000000]*byte
@randall77
Copy link
Contributor Author

This is the error:

$ go tool compile tmp2.go
tmp2.go:5:6: stack frame too large (>1GB)

@josharian
Copy link
Contributor

The cutoff (1<<16) is also duplicated in a couple of places in walk.go (func isSmallMakeSlice, case ONEW in func walkexpr). Should unify all of them.

@cuonglm
Copy link
Member

cuonglm commented Apr 6, 2020

@randall77 @josharian @mdempsky How can we make autotemps heap alloc this case? The order happens after escape analysis was done.

@randall77
Copy link
Contributor Author

@cuonglm This can be tricky. On first glance, there's not immediate problem - order can introduce calls to newobject. If the object is pointerless, it works fine. But if the object has pointers, then things that previously didn't escape now do. I think the right, athough difficult, solution would be to treat these heap-allocated things as stack somehow (no write barriers, scanned as part of stack scanning, etc.).
A good first step would be to solve this issue just for pointerless objects.

@go101
Copy link

go101 commented Jun 28, 2021

It looks this is also true for slice literals.

package main

import t "testing"

const M = 1024 * 1024 * 100

var n int = M - 1
var b byte

func f() {
	type _ int
	var bs = []byte{M+M: 0} 
	b = bs[n]
}

func main() {
	x := t.AllocsPerRun(2, f)
	println(int(x)) // 0
}

{edit] Some array literal corner cases:

package main

const N = 100 * 1024 * 1024

var m = make(map[byte][N]byte, 1)
func foo() {
	m[0] = [N]byte{} // copy stack to heap
}

var i interface{}
func bar() {
	i = [N]byte{} // copy stack to heap
}

func pen(vx ...interface{}) {
	type _ int
}

func qin(m map[byte][N]byte) {
	m[1] = [N]byte{} // copy stack to heap
}

func main() {
	run := func(f func(), c chan struct{}) {
		defer close(c)
		var x int
		println(&x) // <address 1>
		f()
		println(&x) // <address 2>
	}
	c1 := make(chan struct{})
	go run(foo, c1)
	<-c1
	c2 := make(chan struct{})
	go run(bar, c2)
	<-c2
	c3 := make(chan struct{})
	go run(func() {qin(map[byte][N]byte{})}, c3)
	<-c3
	c4 := make(chan struct{})
	go run(func() {pen([N]byte{})}, c4)
	<-c4
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

6 participants