Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: go uses too many CPUs on OpenBSD #30127

Closed
tedu opened this issue Feb 8, 2019 · 4 comments
Closed

runtime: go uses too many CPUs on OpenBSD #30127

tedu opened this issue Feb 8, 2019 · 4 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. OS-OpenBSD
Milestone

Comments

@tedu
Copy link

tedu commented Feb 8, 2019

What version of Go are you using (go version)?

1.11

Does this issue reproduce with the latest release?

All.

What operating system and processor architecture are you using (go env)?

OpenBSD 6.4 and later.

What did you do?

By default, OpenBSD has hw.smt=0 (hyperthreading disabled) which causes hw.ncpu to be twice as high as the actual CPUs running, hw.cpuonline. Go tries to schedule this many threads and they end up fighting each other.

See also: https://marc.info/?l=openbsd-ports&m=154960265529993&w=2

The diff below, against go/src/runtime, changes to use the number of CPUs
online. It's possible for this number to change, and thus become stale, but
that's unlikely, and not the default. But at least it's less wrong.

(This sysctl was added in 6.4.)

Apologies for pasted patch. I think it conveys the idea. I'm not sure how you want to handle backwards compatibility.

--- os_openbsd.go.orig	Fri Feb  8 00:02:27 2019
+++ os_openbsd.go	Fri Feb  8 00:06:21 2019
@@ -85,8 +85,8 @@
 	_KERN_OSREV = 3
 
 	_CTL_HW      = 6
-	_HW_NCPU     = 3
 	_HW_PAGESIZE = 7
+	_HW_NCPUONLINE = 25
 )
 
 func sysctlInt(mib []uint32) (int32, bool) {
@@ -101,7 +101,7 @@
 
 func getncpu() int32 {
 	// Fetch hw.ncpu via sysctl.
-	if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPU}); ok {
+	if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPUONLINE}); ok {
 		return int32(ncpu)
 	}
 	return 1
@mikioh mikioh changed the title go uses too many CPUs on OpenBSD runtime: go uses too many CPUs on OpenBSD Feb 8, 2019
@mikioh
Copy link
Contributor

mikioh commented Feb 8, 2019

As far as I can see, the function hw_sysctl in sys/kern/kern_sysctl.c returns an error for unknown state. It`s fine to test the kernel states one after another, HW_NCPUONLINE then HW_NCPU.

@tklauser tklauser added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Feb 8, 2019
@gopherbot
Copy link

Change https://golang.org/cl/161757 mentions this issue: runtime: use hw.ncpuonline sysctl in getncpu on openbsd

@bradfitz bradfitz added this to the Go1.13 milestone Feb 9, 2019
@bradfitz
Copy link
Contributor

bradfitz commented Feb 9, 2019

/cc @mdempsky

@mdempsky
Copy link
Member

mdempsky commented Feb 9, 2019

Checking hw.ncpuonline and then falling back to hw.ncpu for 6.2 compat sgtm.

@golang golang locked and limited conversation to collaborators Feb 26, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. OS-OpenBSD
Projects
None yet
Development

No branches or pull requests

6 participants