Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: some tests are running out of memory in plan9-amd64-9front #35809

Closed
fhs opened this issue Nov 24, 2019 · 2 comments
Closed

runtime: some tests are running out of memory in plan9-amd64-9front #35809

fhs opened this issue Nov 24, 2019 · 2 comments
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Plan9

Comments

@fhs
Copy link
Contributor

fhs commented Nov 24, 2019

What version of Go are you using (go version)?

$ go version
go version devel +master Sat Nov 23 23:42:39 EST 2019 plan9/amd64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE='on'
GOARCH='amd64'
GOBIN=''
GOCACHE='/usr/fhs/lib/cache/go-build'
GOENV='/usr/fhs/lib/go/env'
GOEXE=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='plan9'
GOINSECURE=''
GONOPROXY=''
GONOSUMDB=''
GOOS='plan9'
GOPATH='/home/fhs/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/big/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLDIR='/home/big/go/pkg/tool/plan9_amd64'
GCCGO='gccgo'
AR='ar'
CC='6c'
CXX='g++'
CGO_ENABLED='0'
GOMOD='/dev/null'
CGO_CFLAGS='-g -O2'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-g -O2'
CGO_FFLAGS='-g -O2'
CGO_LDFLAGS='-g -O2'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build497232617=/tmp/go-build -gno-record-gcc-switches'
GOROOT/bin/go version: go version devel +master Sat Nov 23 23:42:39 EST 2019 plan9/amd64
GOROOT/bin/go tool compile -V: compile version devel +master Sat Nov 23 23:42:39 EST 2019

What did you do?

cpu% cd src/runtime
cpu% go test -v -short

What did you expect to see?

All tests pass

What did you see instead?

...
=== RUN   TestPageAllocScavenge
=== RUN   TestPageAllocScavenge/AllFreeUnscavExhaust
runtime.test 1018: Killed: Insufficient physical memory
runtime.test 1015: Killed: Insufficient physical memory
runtime.test 1017: Killed: Insufficient physical memory
runtime.test 1016: Killed: Insufficient physical memory
runtime.test 1014: Killed: Insufficient physical memory
wait: interrupted
FAIL	runtime	76.934s
cpu% 

The system has 8GB of memory, which I think should be enough? I tried skipping this particular test but then I saw some other tests also running out of memory, include: TestPageAllocAlloc, TestPageAllocExhaust, TestPageAllocFree, TestPageAllocAllocAndFree, TestPageCacheFlush, TestPageAllocAllocToCache.

I'm guessing the plan9-amd64-9front builder is experiencing the same issue because it stops responding after regexp/syntax tests and runtime tests comes right after that. See: https://build.golang.org/log/de03aa0def03f66ebb7f8bd99f71a89190f911dc

@gopherbot Please add labels OS-Plan9, NeedsInvestigation, Builders

@gopherbot gopherbot added Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Plan9 labels Nov 24, 2019
@fhs
Copy link
Contributor Author

fhs commented Nov 24, 2019

Git bisect points to commit 39e8cb0 as the first bad commit.

git bisect log Output
git bisect start
# bad: [6f7b96f6cb5f5c156cf4e69b1fa1ec0e57b01677] syscall: follow convention for generated code comments in linux/riscv64 files
git bisect bad 6f7b96f6cb5f5c156cf4e69b1fa1ec0e57b01677
# bad: [21445b091ec0a0625282603e2730d10b34396375] runtime: make the scavenger self-paced
git bisect bad 21445b091ec0a0625282603e2730d10b34396375
# good: [543c6d2e0dcd24886fce6c00e84b7238c30c1c0b] math, cmd/compile: rename Fma to FMA
git bisect good 543c6d2e0dcd24886fce6c00e84b7238c30c1c0b
# good: [05aa4a7b7447051d187e0a86e53eee99fe065851] runtime: set GODEBUG=asyncpreemptoff=1 in TestCrashDumpsAllThreads
git bisect good 05aa4a7b7447051d187e0a86e53eee99fe065851
# bad: [1b0b9809046c1862f8ea0240fe016e516c67676f] runtime: add async preemption support on ARM64
git bisect bad 1b0b9809046c1862f8ea0240fe016e516c67676f
# bad: [e1ddf0507c6d6dcbe1a2ebe54b8728498edf0995] runtime: count scavenged bits for new allocation for new page allocator
git bisect bad e1ddf0507c6d6dcbe1a2ebe54b8728498edf0995
# bad: [73317080e12234defb59f84e2b5b15f69650b5d5] runtime: add scavenging code for new page allocator
git bisect bad 73317080e12234defb59f84e2b5b15f69650b5d5
# bad: [39e8cb0faac7785f89b21246a45e8cf8d5bc7d95] runtime: add new page allocator core
git bisect bad 39e8cb0faac7785f89b21246a45e8cf8d5bc7d95
# first bad commit: [39e8cb0faac7785f89b21246a45e8cf8d5bc7d95] runtime: add new page allocator core

/CC @mknyszek @0intro

@fhs
Copy link
Contributor Author

fhs commented Dec 12, 2019

This got fixed by commit acf3ff2.

git bisect log Output
git bisect start '--term-new=fixed' '--term-old=unfixed'
# fixed: [9c8c27a318459e76970a8ac0f171a8fc1a6ef5b9] syscall: use SOCK_CLOEXEC when creating sockets
git bisect fixed 9c8c27a318459e76970a8ac0f171a8fc1a6ef5b9
# unfixed: [8054b13536fc71d6f8867f90d7614de4064191ba] runtime: on plan9 don't return substitute address for sysReserve
git bisect unfixed 8054b13536fc71d6f8867f90d7614de4064191ba
# fixed: [5fd75810060e72ba9da44be85c7767417c8bbee7] cmd/go: fix check for -Wp,-U in cgo CFLAGS
git bisect fixed 5fd75810060e72ba9da44be85c7767417c8bbee7
# fixed: [fa3a121a79f85a4c957f29372b5ebfde7211a980] runtime: add a simple version number parser
git bisect fixed fa3a121a79f85a4c957f29372b5ebfde7211a980
# fixed: [acf3ff2e8a0ee777a35b42879c90a1d5a130988f] runtime: convert page allocator bitmap to sparse array
git bisect fixed acf3ff2e8a0ee777a35b42879c90a1d5a130988f
# unfixed: [a18608a044afed86c789e541db65526b9ccacef9] os/exec: document that cmd.Start() sets the Process field
git bisect unfixed a18608a044afed86c789e541db65526b9ccacef9
# unfixed: [386b1a4280d4a32cd8b69a92ba91b6d98832b721] runtime: treat call from runtime as transient in TestDebugCall
git bisect unfixed 386b1a4280d4a32cd8b69a92ba91b6d98832b721
# unfixed: [2ac1ca9160f52907ce1cd04738c80b1c055b5ba6] cmd/vet: honor analyzer flags when running vet outside $GOROOT/src
git bisect unfixed 2ac1ca9160f52907ce1cd04738c80b1c055b5ba6
# first fixed commit: [acf3ff2e8a0ee777a35b42879c90a1d5a130988f] runtime: convert page allocator bitmap to sparse array

@fhs fhs closed this as completed Dec 12, 2019
@golang golang locked and limited conversation to collaborators Dec 11, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Plan9
Projects
None yet
Development

No branches or pull requests

2 participants