Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: linux-mips-rtrk builder consistently failing with "bad prune" as of CL 233497 #39128

Closed
bcmills opened this issue May 18, 2020 · 8 comments
Closed

runtime: linux-mips-rtrk builder consistently failing with "bad prune" as of CL 233497 #39128

bcmills opened this issue May 18, 2020 · 8 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Milestone
Go1.15

Comments

@bcmills
Copy link
Contributor

bcmills commented May 18, 2020

CL 233497, for #38966, introduced a consistent failure in the linux-mips-rtrk builder:

fatal error: bad prune

runtime stack:
runtime.throw(0x68f98c, 0x9)
	/tmp/gobuilder-mips64/go/src/runtime/panic.go:1116 +0x60 fp=0x6f1fee40 sp=0x6f1fee2c pc=0x440fb4
runtime.addrRange.subtract(0x7fc00000, 0x80000000, 0x7fffffff, 0x7fffffff, 0x4, 0x0)
	/tmp/gobuilder-mips64/go/src/runtime/mranges.go:63 +0x10c fp=0x6f1fee4c sp=0x6f1fee40 pc=0x439608
runtime.(*addrRanges).removeGreaterEqual(0xcc6a74, 0x7fffffff)
	/tmp/gobuilder-mips64/go/src/runtime/mranges.go:284 +0x188 fp=0x6f1fee80 sp=0x6f1fee4c pc=0x439efc
runtime.(*pageAlloc).scavengeStartGen(0xcc6a20)
	/tmp/gobuilder-mips64/go/src/runtime/mgcscavenge.go:470 +0x8c fp=0x6f1fee94 sp=0x6f1fee80 pc=0x429eac
runtime.NewPageAlloc.func1()
	/tmp/gobuilder-mips64/go/src/runtime/export_test.go:840 +0x58 fp=0x6f1feea0 sp=0x6f1fee94 pc=0x47ecc0
runtime.systemstack(0x6f1ff920)
	/tmp/gobuilder-mips64/go/src/runtime/asm_mipsx.s:207 +0x9c fp=0x6f1feea4 sp=0x6f1feea0 pc=0x483120
runtime.mstart()
	/tmp/gobuilder-mips64/go/src/runtime/proc.go:1100 fp=0x6f1feea4 sp=0x6f1feea4 pc=0x447480

goroutine 23014 [running]:
runtime.systemstack_switch()
	/tmp/gobuilder-mips64/go/src/runtime/asm_mipsx.s:159 +0x8 fp=0xe206bc sp=0xe206b8 pc=0x483068
runtime.NewPageAlloc(0xc10d60, 0xc10d80, 0x14aae844)
	/tmp/gobuilder-mips64/go/src/runtime/export_test.go:838 +0x304 fp=0xe20738 sp=0xe206bc pc=0x47962c
runtime_test.TestPageAllocAlloc.func1(0xcbf040)
	/tmp/gobuilder-mips64/go/src/runtime/mpagealloc_test.go:619 +0x64 fp=0xe207a4 sp=0xe20738 pc=0x61ef3c
testing.tRunner(0xcbf040, 0xc115c0)
	/tmp/gobuilder-mips64/go/src/testing/testing.go:1053 +0x108 fp=0xe207e4 sp=0xe207a4 pc=0x50e974
runtime.goexit()
	/tmp/gobuilder-mips64/go/src/runtime/asm_mipsx.s:651 +0x4 fp=0xe207e4 sp=0xe207e4 pc=0x485528
created by testing.(*T).Run
	/tmp/gobuilder-mips64/go/src/testing/testing.go:1104 +0x32c

…
FAIL	runtime	24.824s

2020-05-18T13:11:11-afd477f/linux-mips-rtrk
2020-05-18T09:13:38-bb59a13/linux-mips-rtrk
2020-05-15T16:15:25-2b70ffe/linux-mips-rtrk
2020-05-14T21:41:46-881d540/linux-mips-rtrk
2020-05-14T18:58:17-f474e9e/linux-mips-rtrk
2020-05-14T17:50:18-5c802c1/linux-mips-rtrk
2020-05-14T16:53:34-1f325af/linux-mips-rtrk
2020-05-14T16:20:19-796786c/linux-mips-rtrk

CC @mknyszek @cherrymui @aclements
@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker labels May 18, 2020
@bcmills bcmills added this to the Go1.15 milestone May 18, 2020
@mknyszek
Copy link
Contributor

On it.

@mknyszek mknyszek self-assigned this May 18, 2020
@mknyszek
Copy link
Contributor

OK, I see what's wrong. mips32 platforms only have access to the low 2 GiB of virtual memory, so heapAddrBits is 31 and maxOffAddr is set to (2^31)-1. The tests on 32-bit platforms, however, provide dummy addresses in the top half. This didn't break before because maxOffAddr was ^uintptr(0). The fix is to make the tests use addresses in the bottom half only on mips32, I think.

@mknyszek
Copy link
Contributor

Ah! And in fact, there's a comment on BaseChunkIdx already about mips32 platforms, which says to keep the address low. Looks like at least one of the tests violates this.

@mknyszek
Copy link
Contributor

Ah hah, even more specifically, it looks like the test never goes beyond the 2 GiB boundary, but does hit the top of it. maxOffAddr is inclusive but the limit in the range is exclusive. When we try to prune, the range ends up being entirely contained in the last chunk, hence the failure. I don't know the likelihood that the OS on mips32 platforms is going to hand out memory at the top of the address space, but this seems worth guarding against more generally (and leaving the tests as they are).

@cherrymui
Copy link
Member

I don't know the likelihood that the OS on mips32 platforms is going to hand out memory at the top of the address space

Never. The top half is kernel space.

@mknyszek
Copy link
Contributor

Sorry, to be clear I meant the top of the bottom half (so memory whose exclusive upper-bound would be 0x80000000).

@cherrymui
Copy link
Member

Oh, that's possible.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/234457 mentions this issue: runtime: prune ranges with explicit removeGreaterEqual method

@dmitshur dmitshur mentioned this issue Jun 24, 2020
Closed
@golang golang locked and limited conversation to collaborators May 20, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Projects
None yet
Milestone
Go1.15
Development

No branches or pull requests
4 participants
@mknyszek @bcmills @gopherbot @cherrymui