Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: internal compiler error: bvbulkalloc too big #26560

Closed
meilihao opened this issue Jul 24, 2018 · 5 comments
Closed

cmd/compile: internal compiler error: bvbulkalloc too big #26560

meilihao opened this issue Jul 24, 2018 · 5 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Milestone

Comments

@meilihao
Copy link

What version of Go are you using (go version)?

go version go1.11beta2 linux/amd64

Does this issue reproduce with the latest release?

using go version go1.10.3 linux/amd6 gets the same error.

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/chen/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/chen/git/go"
GOPROXY=""
GORACE=""
GOROOT="/opt/go"
GOTMPDIR=""
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build035443180=/tmp/go-build -gno-record-gcc-switches"

What did you do?

go run github.com/meilihao/pinyin/blob/master/example/case.go

What did you expect to see?

go run or go build can work.

What did you see instead?

# github.com/meilihao/pinyin
<autogenerated>:1: internal compiler error: bvbulkalloc too big: nbit=20900 count=877877 nword=654 size=574131558

Please file a bug report including a short program that triggers the error.
https://golang.org/issue/new
@mvdan mvdan added the NeedsFix The path to resolution is known, but the work has not been done. label Jul 24, 2018
@randall77
Copy link
Contributor

This is also a problem at tip.

It's the really big pinyin_dict.go file. Compiling the initializer for PinyinDict fails. It has a 20,000 line map initializer, and that function has too many stack slots (~20,000) times blocks (125,000). The resulting live bitmap is too big.

There are things we can do that might fix this, like #26552. Unfortunately, they will be 1.12 at best.

But if you want to make progress now, you might try reorganizing your code. Use a slice as the underlying data structure and copy it into a map in an init function. Presumably that code is autogenerated so it might require only a small change to the generator.

@meilihao
Copy link
Author

@randall77 Thanks for your answer, i put it into func init(), and it works for now.

@agnivade
Copy link
Contributor

@randall77 - Since OP has resolved the issue, do we want to track this separately or mark as dup of #26552 ?

@randall77
Copy link
Contributor

Let's track this separately. #26552 might fix it but I'm not sure.

@randall77 randall77 added this to the Go1.12 milestone Jul 24, 2018
@ALTree ALTree mentioned this issue Jul 31, 2018
@ianlancetaylor ianlancetaylor changed the title internal compiler error: bvbulkalloc too big cmd/compile: internal compiler error: bvbulkalloc too big Nov 28, 2018
@randall77
Copy link
Contributor

This seems fixed at tip. CL 140301 helped, but even without that CL tip can compile the OP's code (which is now the commented-out initializer in that github repository, right?). So something else has helped also, not sure what. It does still take ~4GB or so to compile, but the live variable map is not >2^31 any more.

I'm going to close this now. #26552 might help some more, but we're ok for now.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Projects
None yet
Development

No branches or pull requests

5 participants