Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/sys: SchedGetaffinity fails on Linux machines with enough (potential) CPUs #23639

Closed
siebenmann opened this issue Jan 31, 2018 · 4 comments
Closed
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@siebenmann
Copy link

What version of Go are you using (go version)?

go version devel +926f278 Tue Jan 30 15:22:28 2018 +0000 linux/amd64

And also the current version of golang.org/x/sys from git.

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOOS="linux"

What did you do?

Here is a trivial program to call unix.SchedGetaffinity() to count the number of currently usable CPUs in the system:

package main

import (
        "fmt"
        "os"

        "golang.org/x/sys/unix"
)

func main() {
        var cpuset unix.CPUSet
        err := unix.SchedGetaffinity(0, &cpuset)
        if err != nil {
                fmt.Printf("numcpus: cannot get affinity: %s\n", err)
                os.Exit(1)
        }
        fmt.Printf("%d\n", cpuset.Count())
}

On many of our amd64 machines this will work fine. However, on some of them (with large CPU counts) it will fail with:

$ numcpus
numcpus: cannot get affinity: invalid argument

Using strace says that sched_getaffinity() is being called as follows:

sched_getaffinity(0, 8, 0xc420075ed8)   = -1 EINVAL (Invalid argument)

The second argument here is the number of bytes in the cpuset. Eight bytes is not very many and looking at how unix.CPUSet is defined in the source code suggests that there is supposed to be room for 1024 CPUs. However, inspection of the actual code making the system call in affinity_linux.go suggests that it is making a size of pointer versus size of object pointed to confusion. The set argument is a pointer to a CPUSet not a CPUSet, so unsafe.Sizeof(set) is the size of the pointer (8 bytes on a 64-bit machine) instead of the much larger size of the CPUSet.

@gopherbot gopherbot added this to the Unreleased milestone Jan 31, 2018
@ianlancetaylor
Copy link
Contributor

CC @tklauser

@rasky rasky added the NeedsFix The path to resolution is known, but the work has not been done. label Jan 31, 2018
@gopherbot
Copy link

Change https://golang.org/cl/91375 mentions this issue: unix: fix cpuset size argument in sched_affinity syscall

@tklauser
Copy link
Member

tklauser commented Feb 1, 2018

@siebenmann Thanks for the detailed report an analysis. I sent golang.org/cl/91375 with a fix. Care to give it a try?

@siebenmann
Copy link
Author

I can confirm that this change/fix fixes the issue for me.

@golang golang locked and limited conversation to collaborators Feb 1, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

5 participants