Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/compile: hang in ssa.(*regAllocState).computeLive when optimizations are disabled #52180

Closed
pioz opened this issue Apr 6, 2022 · 11 comments
Closed

cmd/compile: hang in ssa.(*regAllocState).computeLive when optimizations are disabled #52180

pioz opened this issue Apr 6, 2022 · 11 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Unplanned

Comments

@pioz
Copy link

pioz commented Apr 6, 2022

What version of Go are you using (go version)?

$ go version
go version go1.18 darwin/arm64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN="/Users/pioz/Code/go/bin"
GOCACHE="/Users/pioz/Library/Caches/go-build"
GOENV="/Users/pioz/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/pioz/Code/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/pioz/Code/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.18"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/pioz/Code/go/src/github.com/pioz/faker/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ym/tzf6py4518q66sqncq3nbczw0000gn/T/go-build2384475117=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

To reproduce pull my package faker:

$ git clone https://github.com/pioz/faker.git
$ cd faker
$ go tool compile -o faker.a -lang=go1.14 -goversion go1.18 -N -l -c 4 -complete -shared -nolocalimports -pack address.go builder.go color.go country.go currency.go faker.go gender.go init.go internet.go lang.go misc.go name.go number.go pool.go random.go sentence.go slice.go string.go time.go unique.go utils.go

What did you expect to see?

I expect the output file faker.a.

What did you see instead?

The command run, I think forever (I let it run all night).

If I remove the option -N (disable optimizations) the command compiles the file faker.a as expected.
@pioz pioz changed the title affected/package: go tool compile run forever with option -N Apr 6, 2022
@bcmills bcmills changed the title go tool compile run forever with option -N cmd/compile: hang with -N option Apr 6, 2022
@bcmills
Copy link
Contributor

bcmills commented Apr 6, 2022 •
edited

With a few flag tweaks (removing -c 4, which is not supported with -N on Go 1.17), this also reproduces with Go 1.17.8:

faker$ go1.17.8 tool compile -o faker.a -lang=go1.14 -goversion go1.17.8 -N -l -complete -shared -nolocalimports -pack address.go builder.go color.go country.go currency.go faker.go gender.go init.go internet.go lang.go misc.go name.go number.go pool.go random.go sentence.go slice.go string.go time.go unique.go utils.go
^\SIGQUIT: quit
PC=0x78dee9 m=0 sigcode=128

goroutine 1 [running]:
cmd/compile/internal/ssa.(*sparseMap).contains(...)
        /usr/local/go/src/cmd/compile/internal/ssa/sparsemap.go:38
cmd/compile/internal/ssa.(*regAllocState).computeLive(0xc001e46900)
        /usr/local/go/src/cmd/compile/internal/ssa/regalloc.go:2638 +0x1909 fp=0xc00166f9b8 sp=0xc00166f678 pc=0x78dee9
cmd/compile/internal/ssa.(*regAllocState).init(0xc001e46900, 0xc001f88380)
        /usr/local/go/src/cmd/compile/internal/ssa/regalloc.go:677 +0x850 fp=0xc00166fae8 sp=0xc00166f9b8 pc=0x77e6d0
cmd/compile/internal/ssa.regalloc(0xc001f88380)
        /usr/local/go/src/cmd/compile/internal/ssa/regalloc.go:147 +0x34 fp=0xc00166fb18 sp=0xc00166fae8 pc=0x77c214
cmd/compile/internal/ssa.Compile(0xc001f88380)
        /usr/local/go/src/cmd/compile/internal/ssa/compile.go:96 +0x9e7 fp=0xc001673788 sp=0xc00166fb18 pc=0x71ddc7
cmd/compile/internal/ssagen.buildssa(0xc001297a20, 0x0)
        /usr/local/go/src/cmd/compile/internal/ssagen/ssa.go:642 +0x1e50 fp=0xc001673ad0 sp=0xc001673788 pc=0xa54f50
cmd/compile/internal/ssagen.Compile(0xc001297a20, 0xc000723120)
        /usr/local/go/src/cmd/compile/internal/ssagen/pgen.go:165 +0x4c fp=0xc001673b98 sp=0xc001673ad0 pc=0xa4bd8c
cmd/compile/internal/gc.compileFunctions.func4.1(0x1)
        /usr/local/go/src/cmd/compile/internal/gc/compile.go:153 +0x3a fp=0xc001673bd8 sp=0xc001673b98 pc=0xc02f5a
cmd/compile/internal/gc.compileFunctions.func2(0x0)
        /usr/local/go/src/cmd/compile/internal/gc/compile.go:125 +0x1e fp=0xc001673bf0 sp=0xc001673bd8 pc=0xc0765e
cmd/compile/internal/gc.compileFunctions.func4({0xc0041f8800, 0xf7, 0x100})
        /usr/local/go/src/cmd/compile/internal/gc/compile.go:152 +0x53 fp=0xc001673c38 sp=0xc001673bf0 pc=0xc02e53
cmd/compile/internal/gc.compileFunctions()
        /usr/local/go/src/cmd/compile/internal/gc/compile.go:163 +0x162 fp=0xc001673c98 sp=0xc001673c38 pc=0xc02d22
cmd/compile/internal/gc.Main(0xd16700)
        /usr/local/go/src/cmd/compile/internal/gc/main.go:287 +0xf3a fp=0xc001673f20 sp=0xc001673c98 pc=0xc04e7a
main.main()
        /usr/local/go/src/cmd/compile/main.go:55 +0xdd fp=0xc001673f80 sp=0xc001673f20 pc=0xc2747d
runtime.main()
        /usr/local/go/src/runtime/proc.go:255 +0x227 fp=0xc001673fe0 sp=0xc001673f80 pc=0x437c67
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc001673fe8 sp=0xc001673fe0 pc=0x467241

rax    0xc485
rbx    0x1
rcx    0xc000fc4810
rdx    0xc001e46900
rdi    0xc026553680
rsi    0xc000fc4780
rbp    0xc00166f9a8
rsp    0xc00166f678
r8     0xf13a5
r9     0xd421
r10    0xdc16
r11    0xc0372b4000
r12    0x1b0b00
r13    0xa
r14    0xc0000001a0
r15    0xf13a5
rip    0x78dee9
rflags 0x212
cs     0x33
fs     0x0
gs     0x0

faker$ go1.17.8 tool compile -o faker.a -lang=go1.14 -goversion go1.17.8 -l -complete -shared -nolocalimports -pack address.go builder.go color.go country.go currency.go faker.go gender.go init.go internet.go lang.go misc.go name.go number.go pool.go random.go sentence.go slice.go string.go time.go unique.go utils.go

faker$

@bcmills
Copy link
Contributor

bcmills commented Apr 6, 2022

The hang occurs in the same place with go1.18:

faker$ go1.18 tool compile -o faker.a -lang=go1.14 -goversion go1.18 -N -l -c 4 -complete -shared -nolocalimports -pack address.go builder.go color.go country.go currency.go faker.go gender.go init.go internet.go lang.go misc.go name.go number.go pool.go random.go sentence.go slice.go string.go time.go unique.go utils.go
^\SIGQUIT: quit
PC=0x469121 m=0 sigcode=128

goroutine 0 [idle]:
runtime.futex()
        /usr/local/go/src/runtime/sys_linux_amd64.s:552 +0x21
runtime.futexsleep(0x100020319a8?, 0x0?, 0x7ffd311ebac8?)
        /usr/local/go/src/runtime/os_linux.go:66 +0x36
runtime.notesleep(0x139d028)
        /usr/local/go/src/runtime/lock_futex.go:159 +0x87
runtime.mPark(...)
        /usr/local/go/src/runtime/proc.go:1449
runtime.stopm()
        /usr/local/go/src/runtime/proc.go:2228 +0x8d
runtime.findrunnable()
        /usr/local/go/src/runtime/proc.go:2804 +0x865
runtime.schedule()
        /usr/local/go/src/runtime/proc.go:3187 +0x239
runtime.park_m(0xc000002d00?)
        /usr/local/go/src/runtime/proc.go:3336 +0x14d
runtime.mcall()
        /usr/local/go/src/runtime/asm_amd64.s:425 +0x43

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc001038840?)
        /usr/local/go/src/runtime/sema.go:56 +0x25
sync.(*WaitGroup).Wait(0xc0000bd000?)
        /usr/local/go/src/sync/waitgroup.go:136 +0x52
cmd/compile/internal/gc.compileFunctions()
        /usr/local/go/src/cmd/compile/internal/gc/compile.go:165 +0x19a
cmd/compile/internal/gc.Main(0xd534b0)
        /usr/local/go/src/cmd/compile/internal/gc/main.go:310 +0x109a
main.main()
        /usr/local/go/src/cmd/compile/main.go:55 +0xdd

goroutine 97 [runnable]:
cmd/compile/internal/ssa.(*regAllocState).computeLive(0xc011ff0480)
        /usr/local/go/src/cmd/compile/internal/ssa/regalloc.go:2649 +0x1b66
cmd/compile/internal/ssa.(*regAllocState).init(0xc011ff0480, 0xc00f077a40)
        /usr/local/go/src/cmd/compile/internal/ssa/regalloc.go:680 +0x80d
cmd/compile/internal/ssa.regalloc(0xc00f077a40?)
        /usr/local/go/src/cmd/compile/internal/ssa/regalloc.go:147 +0x34
cmd/compile/internal/ssa.Compile(0xc00f077a40)
        /usr/local/go/src/cmd/compile/internal/ssa/compile.go:98 +0x9be
cmd/compile/internal/ssagen.buildssa(0xc005f866e0, 0x3)
        /usr/local/go/src/cmd/compile/internal/ssagen/ssa.go:602 +0x1fe7
cmd/compile/internal/ssagen.Compile(0xc005f866e0, 0xc000684790?)
        /usr/local/go/src/cmd/compile/internal/ssagen/pgen.go:183 +0x4c
cmd/compile/internal/gc.compileFunctions.func4.1(0x0?)
        /usr/local/go/src/cmd/compile/internal/gc/compile.go:153 +0x3a
cmd/compile/internal/gc.compileFunctions.func3.1()
        /usr/local/go/src/cmd/compile/internal/gc/compile.go:140 +0x4d
created by cmd/compile/internal/gc.compileFunctions.func3
        /usr/local/go/src/cmd/compile/internal/gc/compile.go:138 +0x78

rax    0xca
rbx    0x0
rcx    0x469123
rdx    0x0
rdi    0x139d028
rsi    0x80
rbp    0x7ffd311eba80
rsp    0x7ffd311eba38
r8     0x0
r9     0x0
r10    0x0
r11    0x286
r12    0x56
r13    0x4
r14    0x139c8c0
r15    0x12ea4a0
rip    0x469121
rflags 0x286
cs     0x33
fs     0x0
gs     0x0

@bcmills bcmills changed the title cmd/compile: hang with -N option cmd/compile: hang in ssa.(*regAllocState).computeLive when optimizations are disabled Apr 6, 2022
@bcmills
Copy link
Contributor

bcmills commented Apr 6, 2022

(CC @randall77 @josharian @martisch)

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 6, 2022
@bcmills bcmills added this to the Go1.19 milestone Apr 6, 2022
@pioz
Copy link
Author

pioz commented Apr 6, 2022

With a few flag tweaks (removing -c 4, which is not supported with -N on Go 1.17),

Another thing: the command is generated with the flag -N and -c 4 from delve.

@bcmills
Copy link
Contributor

bcmills commented Apr 6, 2022

(FWIW I think the -c 4 argument is unrelated, except for the fact that in 1.17.8 the compiler explicitly rejects it. 😅)

@cherrymui
Copy link
Member

This code has a huge init function (perhaps from the map literal in init.go?). In -N mode the function has 330000+ basic blocks. It appears to me that the compiler isn't stuck in an infinite loop, but that loop has something quadratic. It runs slower and slower and eventually got killed (probably OOM) on my machine. So perhaps something quadratic in memory as well.

@dr2chase
Copy link
Contributor

dr2chase commented Jun 8, 2022

This is similar to #51543, perhaps the fix ( https://go-review.googlesource.com/c/go/+/397318 ) or an improved version of it would apply here.

@cherrymui
Copy link
Member

It is still slow/hanging at tip. I think CL 397318 improves the liveness computation for debug info generation, but this is one is in the register allocator. I'll take another look.

@ianlancetaylor
Copy link
Contributor

Rolling forward to 1.20. Please comment if you disagree. Thanks.

@ianlancetaylor ianlancetaylor modified the milestones: Go1.19, Go1.20 Jun 24, 2022
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 13, 2022
@mknyszek mknyszek modified the milestones: Go1.20, Unplanned Nov 30, 2022
@pioz
Copy link
Author

pioz commented Feb 14, 2023

Any update on this?

@gopherbot
Copy link

Change https://go.dev/cl/475339 mentions this issue: cmd/compile: tighten for huge functions in -N mode

@golang golang locked and limited conversation to collaborators Mar 12, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
Release Status & Release Blockers
Status: Done
Milestone
Unplanned
Development

No branches or pull requests
7 participants
@pioz @mknyszek @dr2chase @ianlancetaylor @bcmills @gopherbot @cherrymui