-
Notifications
You must be signed in to change notification settings - Fork 18k
cmd/go: build fails in tests with "too many open files" #40498
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
FYI, this is MacOS Catalina plum:~ mtj$ uname -a Darwin plum.local 19.6.0 Darwin Kernel Version 19.6.0: Sun Jul 5 00:43:10 PDT 2020; root:xnu-6153.141.1~9/RELEASE_X86_64 x86_64 |
In general we expect available resources to scale more-or-less linearly with core count. (That's why the @MichaelTJones, could you supply the output of Have you tried using |
$ ulimit -n runtime.GOMAXPROCS(0) = 36 did not think to try, but have done so now do you want me to do bisectional analysis to find the least value that passes? |
I have seen this with Python too. @MichaelTJones, you might want to try 2048. |
@MichaelTJones, we're trying to figure out whether 256 is the hard limit or the soft limit. Could you run |
As a datapoint, with a MBP on macOS 10.15.5:
|
plum:~ mtj$ ulimit -n -S 256 plum:~ mtj$ ulimit -n -H unlimited plum:~ mtj$ sw_vers ProductName: Mac OS X ProductVersion: 10.15.6 BuildVersion: 19G73 |
Ok, so it sounds like maybe we can use That's probably worth pursuing before we try reducing test concurrency, since ideally we really do want the test to make use of more resources when on a multi-core machine. |
Are there any high-core build machines? My 18-core Xeon is not close to the
latest ideas of big. 32 and 64 core CPUs are readily available in the AMD
arena.
|
We have plenty of high-core builders — just not for macOS, which seems to have an unusually low limit on open files. (For comparison, the default soft limit on my 6-core Xeon workstation running a Debian Linux variant is 32,768 — or ~5k files per core — compared to the ~7 files per core observed on your machine.) |
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
no, an in-develpment build issue
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
cd $GOROOT/src; ./all.bash
What did you expect to see?
build run to completion and all tests pass.
What did you see instead?
build worked but one test fails.
this is our old nemesis "too many open files" on high core count (18 +18smt) apple machines.
The text was updated successfully, but these errors were encountered: