New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
x/sys/unix: add sched_getaffinity/sched_setaffinity #11243
Comments
I think it would be a mistake to export these from the runtime for the reason that Ian mentioned on golang-nuts. You basically have to disable the Go scheduler in order to use these from Go, even for the purposes of writing a resource manager, and it would be strange to add something so incompatible with the runtime to the runtime API. Ian mentioned adding these to x/sys. I don't know what the policy for x/sys is, but I assume this would be a fine collection of preexisting foot-guns to add these to. :) Note that you can already get at these through syscall.RawSyscall using something like the following (untested):
|
Should you call UnlockOSThread at the end of the func (or defer)? |
@aclements thanks for the hint of RawSyscall usage, copied to google group. To be clear, change the code to make sure it's using in Go program to setup affinity mask for a outside/standalone process(no write in Go).
|
No. If it sets the affinity for the current OS thread and then unlocks the Goroutine from the OS thread, then the calling goroutine can be scheduled away from the affinitized OS thread and other goroutines can be scheduled on to it. This is one of the reasons I called this a foot gun. |
@wheelcomplex: Ah, I hadn't appreciated that you wanted to change the affinity of an existing other PID. In that case, you don't need the Lock/UnlockOSThread at all. I had assumed you meant you were setting your own thread's affinity and then fork/exec'ing a new process that would inherit the CPU mask. Out of curiosity, how do you deal with setting the affinity of all of the other process's threads while the other process may be concurrently creating or destroying its threads? |
@aclements We make sure outside/standalone process(write by another team and no in Go) can not creating or destroying its threads by administrator command(rule of work), no limit by code. In fact, we name outside/standalone process as "work load cell", there are many type of cells, they all code in single thread and launch or kill by resource manager(part of cluster manager) base on system load or business requirements changes. |
Here is a small patch base on tip commit 71859ef, only tested in ubuntu vivid amd64. It offer GetAffinity/SetAffinityAddAffinity for setup affinity of one pid(thread id in multi-thread process) and new field "Affinitys []int" in type SysProcAttr struct to setup affinity when using os/exec.Cmd. Hope this get Go better. http://play.golang.org/p/EUUhXsOudU
The patch.
|
The general rule for SysProcAttr is that it is only used for operations that must occur between fork and exec. That is not the case for sched_setaffinity. In particular, you can do the operation by simply changing your command to invoke the taskset program. So that change should not be made. As noted previously, the SetAffinity, etc., functions should not be added to the syscall package, because that package is frozen. They should be added to golang.org/x/sys/unix. |
sched_set/getaffinity are still not added to golang.org/x/sys/unix now? |
@caoruidong Correct. This issue is still open. Want to send a patch? |
@ianlancetaylor OK. I will try |
Change https://golang.org/cl/85915 mentions this issue: |
Edit: this comment only valid for go < 1.10 This is just a note for future readers of this thread. If you use the code provided by aclements in their Jun 17, 2015 comment, you will set the affinity for the thread which the goroutine has been locked to. But there's a problem - while the runtime has locked other goroutines from spawning on that thread - the scheduler might run on that thread and might decide to spawn other threads, which will inherit the affinity (the link talks about inheriting priority, but affinity masks are also inherited by threads). So you are then obliged to add lines at the top of every goroutine you spawn in your program to try to set the affinity mask to match all CPUs (AKA "unset"). You won't be able to add this for any libraries you import, so it's a bigger footgun than it seems, and basically not usable. We would have to add a call parallel to |
You're correct that this was an issue before Go 1.10, but as of Go 1.10 the
runtime specifically does not spawn new threads from locked threads
precisely because of issues like this, so this shouldn't be a problem.
|
I edited my comment to make sure nobody stops reading there. How awesome!!!!!!! I'm gonna start using this trick then hahaha |
In Linux system resource manager programing(write in Go), we need sched_setaffinity to limit running CPU list of managed service process(no write in Go, C/C++/Java etc.) and use sched_getaffinity to verify it's corrected limited.
Export sched_getaffinity/sched_setaffinity func will be good for such purpose.
more in google group: https://groups.google.com/forum/#!topic/golang-nuts/99F1rtvcSSI
The text was updated successfully, but these errors were encountered: