Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: compiler crashes on programs with 1<<30 array key #23780

Closed
dotaheor opened this issue Feb 11, 2018 · 13 comments
Closed

cmd/compile: compiler crashes on programs with 1<<30 array key #23780

dotaheor opened this issue Feb 11, 2018 · 13 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@dotaheor
Copy link

What version of Go are you using (go version)?

go version go1.9.3 linux/amd64

Does this issue reproduce with the latest release?

yes

What did you do?

a.go:

package main

import (
	"fmt"
)

func main() {
	fmt.Println("Hello, playground")
	const k uint64 = 1 << 30
	_ = []int{k:1}
}}

go build a.go

What did you expect to see?

produce a binary in one second.

What did you see instead?

computer becomes lagging, almost hanging.
I must press CTRL-C for several times to stop the building.

@cznic
Copy link
Contributor

cznic commented Feb 11, 2018

I can reproduce the issue with go version go1.9.3 linux/amd64 when replacing the final }} with } to make the code syntactically valid.

@dotaheor
Copy link
Author

dotaheor commented Feb 11, 2018

The hang may be caused by my computer (8G memory) has not enough memory.
But I think this should only happen at run time, not compile time.

If I change the slice element type to byte, it fails to compile with the following error:

./a.go:10:15: prepwrite: bad off=1073741824 siz=1 s="".statictmp_1

@agnivade
Copy link
Contributor

agnivade commented Feb 11, 2018

You are trying to create a slice of 1073741824 integers. Which would mean you are trying to get 1073741824 * 8 bytes = 8GB of memory approximately.

@dotaheor
Copy link
Author

Yes. But I think the hang should occur at run time, instead of compile time.

@ALTree
Copy link
Member

ALTree commented Feb 11, 2018

The second issue you reported (the fact that with byte the compiler just prints

./a.go:10:15: prepwrite: bad off=1073741824 siz=1 s="".statictmp_1

) looks more serious.

I suspect it would do the same thing with int64 on a machine with enough memory.

@ALTree ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 11, 2018
@ALTree ALTree changed the title cmd/compile: "go build" makes computer lag, almost hang. cmd/compile: compiler crashes on programs with 1<<30 array key Feb 11, 2018
@ALTree
Copy link
Member

ALTree commented Feb 11, 2018

I changed the issue's title to highlight the linker(?) crash you reported in your second post, since I believe it is more interesting that your machine getting slow on the program having the int64 array.

@agnivade
Copy link
Contributor

I think it is to do with the way the slice is being initialized. []int{k:1} creates a slice of 0,0,...1. But if you change the code to make([]int, k) everything works fine. Frankly, I have never seen syntax like this - []int{k:1}.

@bradfitz bradfitz added this to the Go1.11 milestone Feb 11, 2018
@cznic
Copy link
Contributor

cznic commented Feb 12, 2018

FTR: foo := []T{n:1} makes foo with n+1 elements in contrast to foo := make([]t, n).

Note to the compiler guys: I guess the compiler tries to be nice wrt to initialization time and puts the images for composite literals in the text and/or data segments - triggering this issue. If that's the case I think the solution would be to detect when the composite literal is very sparse/mostly zero-values and perform instead a run time all-zero value creation and initialization of the non zero-valued items only all in code.

@odeke-em
Copy link
Member

odeke-em commented May 3, 2018

/cc @griesemer @mdempsky @josharian

@griesemer
Copy link
Contributor

@cznic Yes, the compiler could probably be a bit smarter here - on the other hand, perhaps one shouldn't write code like this. But I agree that the compiler shouldn't just appear to be hanging.

To summarize:

  1. $ go tool compile x.go
    x.go:4:22: prepwrite: bad off=1073741824 siz=1 s="".statictmp_0
    for x.go:
package p

func f() {
	_ = []byte{1 << 30: 0}
}
  1. $ go tool compile x.go
    x.go:4:21: prepwrite: bad off=8589934592 siz=8 s="".statictmp_0
    for x.go (same as above, but int instead of byte):
package p

func f() {
	_ = []int{1 << 30: 0}
}
  1. There's no issue with:
package p

var X = []int{1 << 30: 0}

@griesemer
Copy link
Contributor

Not urgent. Moving to 1.12.

@griesemer griesemer modified the milestones: Go1.11, Go1.12 Jun 13, 2018
@randall77
Copy link
Contributor

Related to #27447

@gopherbot
Copy link

Change https://golang.org/cl/151319 mentions this issue: cmd/compile: initialize sparse slice literals dynamically

@golang golang locked and limited conversation to collaborators Nov 26, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

9 participants