Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/sys/unix: Getrlimit should support RLIMIT_NPROC (number of process limits) #14854

Closed
mithro opened this issue Mar 18, 2016 · 2 comments
Closed

Comments

@mithro
Copy link

mithro commented Mar 18, 2016

Under Linux a process can be restricted to the number of threads and processes it can launch at any one time. When hitting the process limit, pthread_create will start failing with EAGAIN which causes the go runtime to abort (see bugs #4056 and #14835).

In #4056 it is mentioned that a user should be managing the number of threads (see Comment 33 #4056 (comment) from @dvyukov)

Users must limit number of threads in sys/cgo calls.

However, different systems can vary widely in the number of threads that you are allowed to create. It is very common on server systems to have a thread limit of 10k, while on desktop and embedded systems it could be as little as 512. Thus the user need to know about the number of threads/processes that an operating system will allow.

Linux provides the ability to read this information using the getrlimit syscall, calling with RLIMIT_NPROC (see http://man7.org/linux/man-pages/man2/getrlimit.2.html). The go runtime already provides access to getrlimit in the syscall.Getrlimit (see https://golang.org/pkg/syscall/#Getrlimit) but this module doesn't currently have the RLIMIT_NPROC resource constant (maybe because it isn't POSIX?).

From the getrlimit man page;

RLIMIT_MEMLOCK and RLIMIT_NPROC derive from BSD and are not specified in POSIX.1-2001; they are present on the BSDs and Linux, but on few other implementations.

I believe this can probably be very easily fixed by changing the regex in mkerrors.sh like shown in this patch here -> https://go-review.googlesource.com/#/c/20751/1/src/syscall/mkerrors.sh

@ianlancetaylor ianlancetaylor changed the title runtime: syscall.Getrlimit should support RLIMIT_NPROC (number of process limits) syscall: Getrlimit should support RLIMIT_NPROC (number of process limits) Mar 18, 2016
@ianlancetaylor ianlancetaylor added this to the Go1.7 milestone Mar 18, 2016
@minux minux changed the title syscall: Getrlimit should support RLIMIT_NPROC (number of process limits) x/sys/unix: Getrlimit should support RLIMIT_NPROC (number of process limits) Mar 18, 2016
@minux minux modified the milestones: Unreleased, Go1.7 Mar 18, 2016
@minux
Copy link
Member

minux commented Mar 18, 2016

the syscall package is frozen, please add the missing
RLIMIT_* constants to x/sys/unix instead.

Note that it's very hard for the user to manage runtime
OS threads. Not only the user must limit concurrent cgo
calls, the user also must limit concurrent blocking syscalls
(file open/read/write, etc.)

Even it's possible for the user to carefully limit the
concurrency of sysclal/cgo in his/here own packages,
external packages might still pose a problem.

@gopherbot
Copy link

CL https://golang.org/cl/45052 mentions this issue.

micanzhang pushed a commit to micanzhang/sys that referenced this issue Sep 12, 2017
Add the following additional Getrlimit/Setrlimit resource constants (see
http://man7.org/linux/man-pages/man2/getrlimit.2.html):

  RLIMIT_LOCKS
  RLIMIT_MEMLOCK
  RLIMIT_MSGQUEUE
  RLIMIT_NICE
  RLIMIT_NPROC
  RLIMIT_RSS
  RLIMIT_RTPRIO
  RLIMIT_RTTIME
  RLIMIT_SIGPENDING

Fixes golang/go#14854

Change-Id: I302ff67d6d1b11d94f18614490fbd634a8caec19
Reviewed-on: https://go-review.googlesource.com/45052
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@golang golang locked and limited conversation to collaborators Jun 7, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants