Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: provide error message for giant strings instead of crashing #16394

Closed
bmizerany opened this issue Jul 16, 2016 · 26 comments
Closed
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. ToolSpeed
Milestone

Comments

@bmizerany
Copy link
Contributor

Please answer these questions before submitting your issue. Thanks!

  1. What version of Go are you using (go version)?

go version devel +fca9fc5 Thu Jun 16 19:45:33 2016 +0000 darwin/amd64

  1. What operating system and processor architecture are you using (go env)?

$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/bmizerany"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/dx/jcs03lw901vc05cjjqyjf8y00000gn/T/go-build243723307=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"

  1. What did you do?

I generated this file (165mb) and ran go build:

https://drive.google.com/open?id=0BwlBkts6X4NueEgzUVZiLUJwYms

  1. What did you expect to see?

A successful build.

  1. What did you see instead?

This

$ go build table.go 
# command-line-arguments
runtime: goroutine stack exceeds 1000000000-byte limit
fatal error: stack overflow

runtime stack:
runtime.throw(0x4f62ab, 0xe)
    /usr/local/go/src/runtime/panic.go:566 +0x95
runtime.newstack()
    /usr/local/go/src/runtime/stack.go:1054 +0x416
runtime.morestack()
    /usr/local/go/src/runtime/asm_amd64.s:366 +0x7f

goroutine 1 [running]:
runtime.heapBitsSetType(0xc436720e10, 0x90, 0x90, 0x4e77c0)
    /usr/local/go/src/runtime/mbitmap.go:867 fp=0xc4912b0260 sp=0xc4912b0258
runtime.mallocgc(0x90, 0x4e77c0, 0xfe01, 0xc436720d80)
    /usr/local/go/src/runtime/malloc.go:690 +0x5ae fp=0xc4912b0300 sp=0xc4912b0260
runtime.newobject(0x4e77c0, 0x0)
    /usr/local/go/src/runtime/malloc.go:785 +0x38 fp=0xc4912b0330 sp=0xc4912b0300
cmd/compile/internal/gc.treecopy(0xc422f31cb0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:495 +0x4ae fp=0xc4912b03e8 sp=0xc4912b0330
cmd/compile/internal/gc.treecopy(0xc422f31dd0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b04a0 sp=0xc4912b03e8
cmd/compile/internal/gc.treecopy(0xc422f31ef0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b0558 sp=0xc4912b04a0
cmd/compile/internal/gc.treecopy(0xc422f32090, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b0610 sp=0xc4912b0558
cmd/compile/internal/gc.treecopy(0xc422f321b0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b06c8 sp=0xc4912b0610
cmd/compile/internal/gc.treecopy(0xc422f322d0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b0780 sp=0xc4912b06c8
cmd/compile/internal/gc.treecopy(0xc422f323f0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b0838 sp=0xc4912b0780
cmd/compile/internal/gc.treecopy(0xc422f32510, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b08f0 sp=0xc4912b0838
cmd/compile/internal/gc.treecopy(0xc422f32630, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b09a8 sp=0xc4912b08f0
cmd/compile/internal/gc.treecopy(0xc422f32750, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b0a60 sp=0xc4912b09a8
cmd/compile/internal/gc.treecopy(0xc422f32870, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b0b18 sp=0xc4912b0a60
cmd/compile/internal/gc.treecopy(0xc422f32990, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b0bd0 sp=0xc4912b0b18
cmd/compile/internal/gc.treecopy(0xc422f32ab0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b0c88 sp=0xc4912b0bd0
cmd/compile/internal/gc.treecopy(0xc422f32bd0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b0d40 sp=0xc4912b0c88
cmd/compile/internal/gc.treecopy(0xc422f32cf0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b0df8 sp=0xc4912b0d40
cmd/compile/internal/gc.treecopy(0xc422f32e10, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b0eb0 sp=0xc4912b0df8
cmd/compile/internal/gc.treecopy(0xc422f32f30, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b0f68 sp=0xc4912b0eb0
cmd/compile/internal/gc.treecopy(0xc422f33050, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1020 sp=0xc4912b0f68
cmd/compile/internal/gc.treecopy(0xc422f33170, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b10d8 sp=0xc4912b1020
cmd/compile/internal/gc.treecopy(0xc422f33290, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1190 sp=0xc4912b10d8
cmd/compile/internal/gc.treecopy(0xc422f333b0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1248 sp=0xc4912b1190
cmd/compile/internal/gc.treecopy(0xc422f334d0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1300 sp=0xc4912b1248
cmd/compile/internal/gc.treecopy(0xc422f335f0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b13b8 sp=0xc4912b1300
cmd/compile/internal/gc.treecopy(0xc422f33710, 0xc400000000, 0x1000000000000)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1470 sp=0xc4912b13b8
cmd/compile/internal/gc.treecopy(0xc422f33830, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1528 sp=0xc4912b1470
cmd/compile/internal/gc.treecopy(0xc422f33950, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b15e0 sp=0xc4912b1528
cmd/compile/internal/gc.treecopy(0xc422f33a70, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1698 sp=0xc4912b15e0
cmd/compile/internal/gc.treecopy(0xc422f33b90, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1750 sp=0xc4912b1698
cmd/compile/internal/gc.treecopy(0xc422f33cb0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1808 sp=0xc4912b1750
cmd/compile/internal/gc.treecopy(0xc422f33dd0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b18c0 sp=0xc4912b1808
cmd/compile/internal/gc.treecopy(0xc422f33ef0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1978 sp=0xc4912b18c0
cmd/compile/internal/gc.treecopy(0xc422f38090, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1a30 sp=0xc4912b1978
cmd/compile/internal/gc.treecopy(0xc422f381b0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1ae8 sp=0xc4912b1a30
cmd/compile/internal/gc.treecopy(0xc422f382d0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1ba0 sp=0xc4912b1ae8
cmd/compile/internal/gc.treecopy(0xc422f383f0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1c58 sp=0xc4912b1ba0
cmd/compile/internal/gc.treecopy(0xc422f38510, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1d10 sp=0xc4912b1c58
cmd/compile/internal/gc.treecopy(0xc422f38630, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1dc8 sp=0xc4912b1d10
cmd/compile/internal/gc.treecopy(0xc422f38750, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1e80 sp=0xc4912b1dc8
cmd/compile/internal/gc.treecopy(0xc422f38870, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1f38 sp=0xc4912b1e80
cmd/compile/internal/gc.treecopy(0xc422f38990, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b1ff0 sp=0xc4912b1f38
cmd/compile/internal/gc.treecopy(0xc422f38ab0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b20a8 sp=0xc4912b1ff0
cmd/compile/internal/gc.treecopy(0xc422f38bd0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2160 sp=0xc4912b20a8
cmd/compile/internal/gc.treecopy(0xc422f38cf0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2218 sp=0xc4912b2160
cmd/compile/internal/gc.treecopy(0xc422f38e10, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b22d0 sp=0xc4912b2218
cmd/compile/internal/gc.treecopy(0xc422f38f30, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2388 sp=0xc4912b22d0
cmd/compile/internal/gc.treecopy(0xc422f39050, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2440 sp=0xc4912b2388
cmd/compile/internal/gc.treecopy(0xc422f39170, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b24f8 sp=0xc4912b2440
cmd/compile/internal/gc.treecopy(0xc422f39290, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b25b0 sp=0xc4912b24f8
cmd/compile/internal/gc.treecopy(0xc422f393b0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2668 sp=0xc4912b25b0
cmd/compile/internal/gc.treecopy(0xc422f394d0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2720 sp=0xc4912b2668
cmd/compile/internal/gc.treecopy(0xc422f395f0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b27d8 sp=0xc4912b2720
cmd/compile/internal/gc.treecopy(0xc422f39710, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2890 sp=0xc4912b27d8
cmd/compile/internal/gc.treecopy(0xc422f39830, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2948 sp=0xc4912b2890
cmd/compile/internal/gc.treecopy(0xc422f39950, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2a00 sp=0xc4912b2948
cmd/compile/internal/gc.treecopy(0xc422f39a70, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2ab8 sp=0xc4912b2a00
cmd/compile/internal/gc.treecopy(0xc422f39b90, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2b70 sp=0xc4912b2ab8
cmd/compile/internal/gc.treecopy(0xc422f39cb0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2c28 sp=0xc4912b2b70
cmd/compile/internal/gc.treecopy(0xc422f39dd0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2ce0 sp=0xc4912b2c28
cmd/compile/internal/gc.treecopy(0xc422f39ef0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2d98 sp=0xc4912b2ce0
cmd/compile/internal/gc.treecopy(0xc422f3c090, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2e50 sp=0xc4912b2d98
cmd/compile/internal/gc.treecopy(0xc422f3c1b0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2f08 sp=0xc4912b2e50
cmd/compile/internal/gc.treecopy(0xc422f3c2d0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b2fc0 sp=0xc4912b2f08
cmd/compile/internal/gc.treecopy(0xc422f3c3f0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3078 sp=0xc4912b2fc0
cmd/compile/internal/gc.treecopy(0xc422f3c510, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3130 sp=0xc4912b3078
cmd/compile/internal/gc.treecopy(0xc422f3c630, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b31e8 sp=0xc4912b3130
cmd/compile/internal/gc.treecopy(0xc422f3c750, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b32a0 sp=0xc4912b31e8
cmd/compile/internal/gc.treecopy(0xc422f3c870, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3358 sp=0xc4912b32a0
cmd/compile/internal/gc.treecopy(0xc422f3c990, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3410 sp=0xc4912b3358
cmd/compile/internal/gc.treecopy(0xc422f3cab0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b34c8 sp=0xc4912b3410
cmd/compile/internal/gc.treecopy(0xc422f3cbd0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3580 sp=0xc4912b34c8
cmd/compile/internal/gc.treecopy(0xc422f3ccf0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3638 sp=0xc4912b3580
cmd/compile/internal/gc.treecopy(0xc422f3ce10, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b36f0 sp=0xc4912b3638
cmd/compile/internal/gc.treecopy(0xc422f3cf30, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b37a8 sp=0xc4912b36f0
cmd/compile/internal/gc.treecopy(0xc422f3d050, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3860 sp=0xc4912b37a8
cmd/compile/internal/gc.treecopy(0xc422f3d170, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3918 sp=0xc4912b3860
cmd/compile/internal/gc.treecopy(0xc422f3d290, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b39d0 sp=0xc4912b3918
cmd/compile/internal/gc.treecopy(0xc422f3d3b0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3a88 sp=0xc4912b39d0
cmd/compile/internal/gc.treecopy(0xc422f3d4d0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3b40 sp=0xc4912b3a88
cmd/compile/internal/gc.treecopy(0xc422f3d5f0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3bf8 sp=0xc4912b3b40
cmd/compile/internal/gc.treecopy(0xc422f3d710, 0xc400000000, 0x1000000000000)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3cb0 sp=0xc4912b3bf8
cmd/compile/internal/gc.treecopy(0xc422f3d830, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3d68 sp=0xc4912b3cb0
cmd/compile/internal/gc.treecopy(0xc422f3d950, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3e20 sp=0xc4912b3d68
cmd/compile/internal/gc.treecopy(0xc422f3da70, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3ed8 sp=0xc4912b3e20
cmd/compile/internal/gc.treecopy(0xc422f3db90, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b3f90 sp=0xc4912b3ed8
cmd/compile/internal/gc.treecopy(0xc422f3dcb0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b4048 sp=0xc4912b3f90
cmd/compile/internal/gc.treecopy(0xc422f3ddd0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b4100 sp=0xc4912b4048
cmd/compile/internal/gc.treecopy(0xc422f3def0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b41b8 sp=0xc4912b4100
cmd/compile/internal/gc.treecopy(0xc422f3e090, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b4270 sp=0xc4912b41b8
cmd/compile/internal/gc.treecopy(0xc422f3e1b0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b4328 sp=0xc4912b4270
cmd/compile/internal/gc.treecopy(0xc422f3e2d0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b43e0 sp=0xc4912b4328
cmd/compile/internal/gc.treecopy(0xc422f3e3f0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b4498 sp=0xc4912b43e0
cmd/compile/internal/gc.treecopy(0xc422f3e510, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b4550 sp=0xc4912b4498
cmd/compile/internal/gc.treecopy(0xc422f3e630, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b4608 sp=0xc4912b4550
cmd/compile/internal/gc.treecopy(0xc422f3e750, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b46c0 sp=0xc4912b4608
cmd/compile/internal/gc.treecopy(0xc422f3e870, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b4778 sp=0xc4912b46c0
cmd/compile/internal/gc.treecopy(0xc422f3e990, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b4830 sp=0xc4912b4778
cmd/compile/internal/gc.treecopy(0xc422f3eab0, 0xc400000000, 0x0)
    /usr/local/go/src/cmd/compile/internal/gc/subr.go:497 +0x523 fp=0xc4912b48e8 sp=0xc4912b4830
...additional frames elided...
@dr2chase
Copy link
Contributor

Is this a regression?

@bmizerany
Copy link
Contributor Author

@dr2chase If you're asking me: I don't know.

@bmizerany
Copy link
Contributor Author

Update: I changed each string from 20 escaped bytes to 1000 and now go build table.go is hanging it seems. It's been a few minutes and my laptop is slowing to a crawl - I could barely get this message out. If you don't hear back soon go on without me.

@bmizerany
Copy link
Contributor Author

bmizerany commented Jul 16, 2016

Okay. I'm back. It exited with:

$ go build table.go 
go build command-line-arguments: /usr/local/go/pkg/tool/darwin_amd64/compile: signal: killed

@josharian
Copy link
Contributor

Can you provide the script used to generate the file?

@dr2chase
Copy link
Contributor

Am I correct that this is

const table = 

followed by a single gigantic string constant broken into the +-concatenation of over 3 million strings?

Have you tried it as a single string constant? It's clearly machine-generated, and this is a little outside the norm even for machine-generated code.

@dr2chase
Copy link
Contributor

Workaround: emit it as a single line.

time go build htablej.go

real    0m3.920s
user    0m3.677s
sys 0m0.260s

./htablej 
Len table is  65865934

Code to rewrite into a single string constant:

package main

import (
    "bufio"
    "fmt"
    "os"
    "strings"
)

func main() {
    scanner := bufio.NewScanner(os.Stdin)
    join := false
    for scanner.Scan() {
        t := strings.TrimSpace(scanner.Text())
        l := len(t)
        if join && l > 1 && t[0] == '"' { // eat leading "
            t = t[1:]
            l--
        }
        join = false
        if l < 2 || t[l-1] != '+' || t[l-2] != '"' {
            fmt.Println(t)
            continue
        }
        join = true
        t = t[:l-2] // lose trailing "+
        fmt.Print(t)
    }
}

@bmizerany
Copy link
Contributor Author

It is machine generated so yes, I can make it a single line. It was just an experiment, though. I thought this would be of interest. I'll let you know if the single line doesn't work for some reason.

@ianlancetaylor
Copy link
Contributor

This is not a regression. Go 1.5 and 1.6 fail in the same way. Go 1.1 through 1.4 (before the compiler was itself written in Go) simply fail with segmentation violations.

@ianlancetaylor ianlancetaylor added this to the Go1.8 milestone Jul 17, 2016
@ianlancetaylor
Copy link
Contributor

It may be that this file is too large to compile, but the compiler should fail with a better error message.

@josharian
Copy link
Contributor

Related: #7755. See also the discussion in #7754.

I don't see a reasonable approach here for getting good error messages other than setting an arbitrary max string size, the way we eventually declare other constants too large. The fix would need to be in several places—first encounter, the OADDSTR case in const.go, maybe others.

@josharian josharian changed the title cmd/go: go build crashes with "goroutine stack exceeds 1000000000-byte limit" cmd/compile: provide error message for giant strings instead of crashing Jul 17, 2016
@ianlancetaylor
Copy link
Contributor

I guess I think that even a crash saying "compiler is out of memory" would be better than a crash saying "goroutine stack exceeds 1000000000-byte limit".

@josharian
Copy link
Contributor

That error is a throw, not a panic, so we can't do an easy defer/recover.

I suppose we could add recursion depth tracking to treecopy.

But maybe the best plan for this particular problem is changing the AST for OADD to be like OADDSTR—instead of Left+Right, flatten and use List to store all the Nodes to be added. This is one of many parts of the AST that would benefit from flattening. cc @mdempsky who I believe has been thinking about the future of the AST.

@dr2chase
Copy link
Contributor

It's not an issue of string size, it is instead a problem with the size of the 3-million deep expression tree, of the form "a0"+"a1"+"a2"+...+"a3,000,000". If you convert that into the single string that it could be, it compiles in a few seconds. It's machine-generated code, I recommend fixing the machine-generator. It's also possible that judicious use of parentheses could get past this, along the lines of:

("a0"+"a1"+"a2"+...+"a2,999")+("a3,000"+"a3,001"+"a3,002"+...+"a5,999")+...+
("a2,997,000"+...+"a2,999,999")+"a3,000,000"

@bradfitz
Copy link
Contributor

it is instead a problem with the size of the 3-million deep expression tree

That kinda seems like a lazy excuse, shifting the problem to the user.

@randall77
Copy link
Contributor

I think @dr2chase's recommendation should be considered as a workaround for the problem for now, not a permanent solution. We should still fix this problem if we can.

@josharian
Copy link
Contributor

Flattening OADD as suggested would solve the deep tree. I'm happy to look at it during 1.8 if @mdempsky thinks it won't be wasted work.

@dr2chase
Copy link
Contributor

Reassociating OADD is okay if the type is not floating point or complex.
We might be able to recognize this in the parser, given ..." + "...

@odeke-em
Copy link
Member

odeke-em commented Oct 3, 2016

Perhaps this issue is a duplicate of older issue #1700?

@griesemer griesemer removed this from the Go1.9 milestone Jun 6, 2017
@griesemer
Copy link
Contributor

The compiler-internal syntax package can parse table.go (in cmd/compile/internal/syntax: go test -run Parse -src=$HOME/tmp/table.go). The (first) problem in the compiler is that the conversion to nodes requires recursion which leads to stack overflow. It may be possible to avoid that recursion, but that may still cause problems down the road due to sheer size.

Recognizing string concatenation in the parser (and doing the concatenation there) may be possible but is a bit tricky.

Bumping forward to 1.10. This is an esoteric corner case.

@griesemer
Copy link
Contributor

@odeke-em Not a duplicate of #1700, but related. Fixing the first part of this issue may lead to #1700.

tmthrgd added a commit to tmthrgd/go-bindata that referenced this issue Jul 2, 2017
The compiler has trouble compiling the concatenation of many strings, s0 + s1 +
s2 + ... + sN, for large N. We insert redundant, explicit parentheses to work
around that, lowering the N at any given step.

This bug is golang/go#16394 and the fix is taken from golang/text@5c6cf4f9a2.

Fixes #8
@gopherbot
Copy link

Change https://golang.org/cl/73880 mentions this issue: cmd/compile: stop crashing on huge string concat

@gopherbot
Copy link

Change https://golang.org/cl/76450 mentions this issue: cmd/compile: optimize noding of long summation expressions

@mdempsky
Copy link
Member

mdempsky commented Nov 8, 2017

This should be reasonably fixed now. During noding, we now specially recognize expressions with concatenated string literals like

x + "a" + "b" + "c" + y

and efficiently collapse them down into

x + "abc" + y

@mvdan
Copy link
Member

mvdan commented Nov 8, 2017

@mdempsky is that done for constants in general when noding, or just as a special case for strings to avoid the internal compiler crash later? I would assume that constant folding happens at a later stage.

@mdempsky
Copy link
Member

mdempsky commented Nov 8, 2017

@mvdan During noding, we only special case string literal concatenation.

General constant folding (including string concatenation involving named string constants or parenthesized subexpressions) is still done during typechecking.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. ToolSpeed
Projects
None yet
Development

No branches or pull requests