Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime/trace: TestTraceStress and TestTraceStressStartStop running out of memory on plan9 and windows #21873

Closed
0intro opened this issue Sep 13, 2017 · 3 comments
Closed

runtime/trace: TestTraceStress and TestTraceStressStartStop running out of memory on plan9 and windows #21873

0intro opened this issue Sep 13, 2017 · 3 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Plan9 OS-Windows
Milestone
Backlog

Comments

@0intro
Copy link
Member

0intro commented Sep 13, 2017

TestTraceStress and TestTraceStressStartStop are regularly running out of memory on plan9/amd64.

This issue is related to the issue #19388 about the plan9-amd64-9front builder.

runtime: out of memory: cannot allocate 154009600-byte block (467501056 in use)
fatal error: out of memory

runtime stack:
runtime.throw(0x36fd71, 0xd)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/panic.go:606 +0x86
runtime.largeAlloc(0x92e0000, 0x440630001, 0x427fd2805)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/malloc.go:829 +0x115
runtime.mallocgc.func1()
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/malloc.go:722 +0x4b
runtime.systemstack(0x4207aa600)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/asm_amd64.s:344 +0x7c
runtime.mstart()
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/proc.go:1140

goroutine 27 [running]:
runtime.systemstack_switch()
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/asm_amd64.s:298 fp=0x42067b748 sp=0x42067b740 pc=0x252f50
runtime.mallocgc(0x92e0000, 0x352f60, 0x1, 0x4280be150)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/malloc.go:721 +0x7c9 fp=0x42067b7f0 sp=0x42067b748 pc=0x20fbe9
runtime.growslice(0x352f60, 0x434fae000, 0x2f0000, 0x2f0000, 0x2f0001, 0x4280be150, 0x1, 0x2)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/slice.go:154 +0x144 fp=0x42067b858 sp=0x42067b7f0 pc=0x23dd14
internal/trace.readTrace(0x38b240, 0x4206c6380, 0x420660cc0, 0x231e75, 0x263b05, 0x420660cf0, 0x42063ef00, 0x420660cd8, 0x420660cd8)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/internal/trace/parser.go:240 +0x77b fp=0x42067bc78 sp=0x42067b858 pc=0x2fe14b
internal/trace.parse(0x38b240, 0x4206c6380, 0x0, 0x0, 0x22b923, 0x377850, 0x5, 0x4206be000, 0x42063ef00, 0x420660da8)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/internal/trace/parser.go:79 +0x52 fp=0x42067bd20 sp=0x42067bc78 pc=0x2fd5c2
internal/trace.Parse(0x38b240, 0x4206c6380, 0x0, 0x0, 0x420658000, 0x420660e18, 0x23bd2c, 0x4206883c0, 0x36f401)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/internal/trace/parser.go:66 +0x52 fp=0x42067bd80 sp=0x42067bd20 pc=0x2fd432
runtime/trace_test.parseTrace(0x4206c84b0, 0x38b240, 0x4206c6380, 0xf, 0x1, 0x0, 0x0)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/trace/trace_test.go:73 +0x64 fp=0x42067be88 sp=0x42067bd80 pc=0x322b24
runtime/trace_test.TestTraceStress(0x4206c84b0)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/trace/trace_test.go:246 +0x80b fp=0x42067bfa8 sp=0x42067be88 pc=0x3237fb
testing.tRunner(0x4206c84b0, 0x377a70)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/testing/testing.go:748 +0xd5 fp=0x42067bfd0 sp=0x42067bfa8 pc=0x2b6095
runtime.goexit()
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/asm_amd64.s:2303 +0x1 fp=0x42067bfd8 sp=0x42067bfd0 pc=0x255ae1
created by testing.(*T).Run
	/tmp/workdir-host-plan9-amd64-0intro/go/src/testing/testing.go:791 +0x2d9

goroutine 1 [chan receive]:
testing.(*T).Run(0x4206c8000, 0x3701b9, 0xf, 0x377a70, 0x26ac01)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/testing/testing.go:792 +0x2fa
testing.runTests.func1(0x4206c8000)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/testing/testing.go:1015 +0x69
testing.tRunner(0x4206c8000, 0x42067fdd8)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/testing/testing.go:748 +0xd5
testing.runTests(0x4206520e0, 0x615180, 0x7, 0x7, 0x42067fed0)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/testing/testing.go:1013 +0x2b5
testing.(*M).Run(0x4206c6000, 0x0)
	/tmp/workdir-host-plan9-amd64-0intro/go/src/testing/testing.go:933 +0x16f
main.main()
	runtime/trace/_test/_testmain.go:58 +0x15b

goroutine 8 [select (no cases)]:
runtime/trace_test.TestTraceSymbolize.func1()
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/trace/trace_stack_test.go:38 +0x25
created by runtime/trace_test.TestTraceSymbolize
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/trace/trace_stack_test.go:37 +0x131

goroutine 9 [chan send (nil chan)]:
runtime/trace_test.TestTraceSymbolize.func2()
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/trace/trace_stack_test.go:42 +0x3b
created by runtime/trace_test.TestTraceSymbolize
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/trace/trace_stack_test.go:40 +0x149

goroutine 10 [chan receive (nil chan)]:
runtime/trace_test.TestTraceSymbolize.func3()
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/trace/trace_stack_test.go:46 +0x38
created by runtime/trace_test.TestTraceSymbolize
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/trace/trace_stack_test.go:44 +0x161

goroutine 55 [select (no cases)]:
runtime/trace_test.TestTraceStress.func9()
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/trace/trace_test.go:232 +0x2a
created by runtime/trace_test.TestTraceStress
	/tmp/workdir-host-plan9-amd64-0intro/go/src/runtime/trace/trace_test.go:230 +0x6f0
FAIL	runtime/trace	7.761s

See https://build.golang.org/log/cc30d23255254e6ca1f7d772c6b0482baf47eb06
@0intro 0intro added this to the Go1.10 milestone Sep 13, 2017
@0intro 0intro self-assigned this Sep 13, 2017
@rsc rsc modified the milestones: Go1.10, Go1.11 Nov 29, 2017
@bcmills
Copy link
Contributor

bcmills commented Jun 1, 2018

It's not just plan9: there is a similar failure in readTrace on the windows-amd64-race builder:
https://build.golang.org/log/2bf896ddb4f7478b5635b87d4bc1bc033bc803fa

runtime: VirtualAlloc of 29360128 bytes failed with errno=1455
fatal error: out of memory

runtime stack:
runtime.throw(0x670dbd, 0xd)
	C:/workdir/go/src/runtime/panic.go:589 +0x71
runtime.sysMap(0xc024c00000, 0x1c00000, 0x806b38)
	C:/workdir/go/src/runtime/mem_windows.go:122 +0x138
runtime.(*mheap).sysAlloc(0x7ec2e0, 0x1c00000, 0x0, 0x48)
	C:/workdir/go/src/runtime/malloc.go:611 +0x1cb
runtime.(*mheap).grow(0x7ec2e0, 0xc51, 0x0)
	C:/workdir/go/src/runtime/mheap.go:920 +0x49
runtime.(*mheap).allocSpanLocked(0x7ec2e0, 0xc51, 0x806b48, 0x4000)
	C:/workdir/go/src/runtime/mheap.go:848 +0x361
runtime.(*mheap).alloc_m(0x7ec2e0, 0xc51, 0xffffffffffff0100, 0x533fd90)
	C:/workdir/go/src/runtime/mheap.go:692 +0x127
runtime.(*mheap).alloc.func1()
	C:/workdir/go/src/runtime/mheap.go:759 +0x53
runtime.(*mheap).alloc(0x7ec2e0, 0xc51, 0xc000010100, 0xc000003200)
	C:/workdir/go/src/runtime/mheap.go:758 +0x91
runtime.largeAlloc(0x18a2000, 0xc001d90001, 0x478b20)
	C:/workdir/go/src/runtime/malloc.go:1011 +0x9e
runtime.mallocgc.func1()
	C:/workdir/go/src/runtime/malloc.go:906 +0x4d
runtime.systemstack(0x0)
	C:/workdir/go/src/runtime/asm_amd64.s:351 +0x6b
runtime.mstart()
	C:/workdir/go/src/runtime/proc.go:1223

goroutine 43 [running]:
runtime.systemstack_switch()
	C:/workdir/go/src/runtime/asm_amd64.s:311 fp=0xc001d9f740 sp=0xc001d9f738 pc=0x490710
runtime.mallocgc(0x18a2000, 0x652520, 0x5c7201, 0x478b20)
	C:/workdir/go/src/runtime/malloc.go:905 +0x8f0 fp=0xc001d9f7e0 sp=0xc001d9f740 pc=0x440a50
runtime.growslice(0x652520, 0xc022e74000, 0x4ed00, 0x4ed00, 0x4ed01, 0xc0243cbee0, 0x1, 0x2)
	C:/workdir/go/src/runtime/slice.go:204 +0x1b1 fp=0xc001d9f858 sp=0xc001d9f7e0 pc=0x478cd1
internal/trace.readTrace(0x6a2dc0, 0xc00bff9950, 0xc00bff98f0, 0x8, 0x445190, 0x63ad40, 0x67b898, 0x3002f, 0x5c39c8)
	C:/workdir/go/src/internal/trace/parser.go:268 +0x92d fp=0xc001d9fbf0 sp=0xc001d9f858 pc=0x5c720d
internal/trace.parse(0x6a2dc0, 0xc00bff9950, 0x0, 0x0, 0x0, 0xc0000c8380, 0x1b1c00, 0x3413e0, 0x0, 0x16a306, ...)
	C:/workdir/go/src/internal/trace/parser.go:103 +0x65 fp=0xc001d9fce0 sp=0xc001d9fbf0 pc=0x5c6445
internal/trace.Parse(0x6a2dc0, 0xc00bff9950, 0x0, 0x0, 0x0, 0x0, 0x638980, 0xc00bff82a0, 0xc000008900, 0xc000008b40)
	C:/workdir/go/src/internal/trace/parser.go:90 +0x7f fp=0xc001d9fd88 sp=0xc001d9fce0 pc=0x5c623f
runtime/trace_test.testBrokenTimestamps(0xc00018c500, 0xc0008de000, 0x3d12be, 0x3e01bf)
	C:/workdir/go/src/runtime/trace/trace_test.go:172 +0x145 fp=0xc001d9fe18 sp=0xc001d9fd88 pc=0x606c45
runtime/trace_test.TestTraceStress(0xc00018c500)
	C:/workdir/go/src/runtime/trace/trace_test.go:319 +0x8c5 fp=0xc001d9ff78 sp=0xc001d9fe18 pc=0x607685
testing.tRunner(0xc00018c500, 0x67bc78)
	C:/workdir/go/src/testing/testing.go:827 +0x16a fp=0xc001d9ffd0 sp=0xc001d9ff78 pc=0x54bc8a
runtime.goexit()
	C:/workdir/go/src/runtime/asm_amd64.s:1333 +0x1 fp=0xc001d9ffd8 sp=0xc001d9ffd0 pc=0x4928d1
created by testing.(*T).Run
	C:/workdir/go/src/testing/testing.go:878 +0x658

goroutine 1 [chan receive]:
testing.(*T).Run(0xc0000a0000, 0x6714a3, 0xf, 0x67bc78, 0xc000067c01)
	C:/workdir/go/src/testing/testing.go:879 +0x690
testing.runTests.func1(0xc0000a0000)
	C:/workdir/go/src/testing/testing.go:1119 +0xb0
testing.tRunner(0xc0000a0000, 0xc000067d88)
	C:/workdir/go/src/testing/testing.go:827 +0x16a
testing.runTests(0xc0000044c0, 0x7e2f00, 0x9, 0x9, 0x2030000)
	C:/workdir/go/src/testing/testing.go:1117 +0x4ee
testing.(*M).Run(0xc000090000, 0x0)
	C:/workdir/go/src/testing/testing.go:1034 +0x2f6
main.main()
	_testmain.go:64 +0x229

goroutine 11 [select (no cases)]:
runtime/trace_test.TestTraceSymbolize.func1()
	C:/workdir/go/src/runtime/trace/trace_stack_test.go:41 +0x37
created by runtime/trace_test.TestTraceSymbolize
	C:/workdir/go/src/runtime/trace/trace_stack_test.go:40 +0x137

goroutine 12 [chan send (nil chan)]:
runtime/trace_test.TestTraceSymbolize.func2()
	C:/workdir/go/src/runtime/trace/trace_stack_test.go:45 +0x52
created by runtime/trace_test.TestTraceSymbolize
	C:/workdir/go/src/runtime/trace/trace_stack_test.go:43 +0x14f

goroutine 13 [chan receive (nil chan)]:
runtime/trace_test.TestTraceSymbolize.func3()
	C:/workdir/go/src/runtime/trace/trace_stack_test.go:49 +0x3e
created by runtime/trace_test.TestTraceSymbolize
	C:/workdir/go/src/runtime/trace/trace_stack_test.go:47 +0x167

goroutine 99 [select (no cases)]:
runtime/trace_test.TestTraceStress.func9()
	C:/workdir/go/src/runtime/trace/trace_test.go:304 +0x3c
created by runtime/trace_test.TestTraceStress
	C:/workdir/go/src/runtime/trace/trace_test.go:302 +0x717
FAIL	runtime/trace	33.170s

@bcmills bcmills changed the title runtime/trace: TestTraceStress and TestTraceStressStartStop running out of memory on plan9/amd64 runtime/trace: TestTraceStress and TestTraceStressStartStop running out of memory on plan9 and windows Jun 1, 2018
@ianlancetaylor ianlancetaylor added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. help wanted labels Jun 28, 2018
@ianlancetaylor ianlancetaylor modified the milestones: Go1.11, Go1.12 Jun 28, 2018
@bradfitz
Copy link
Contributor

I have vague memories of this being fixed. @aclements, ring a bell?

@andybons andybons modified the milestones: Go1.12, Go1.13 Feb 12, 2019
@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@rsc rsc unassigned 0intro Jun 23, 2022
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 7, 2022
@bcmills
Copy link
Contributor

bcmills commented Mar 16, 2023

This issue is very old and it's been years since the last update. If there are any remaining failures, watchflakes can open an issue for them.

@bcmills bcmills closed this as not planned Won't fix, can't repro, duplicate, stale Mar 16, 2023
@golang golang locked and limited conversation to collaborators Mar 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Plan9 OS-Windows
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
7 participants
@bradfitz @rsc @andybons @ianlancetaylor @bcmills @0intro @gopherbot