Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/compile: "too many open files in system" on macos Ventura #54833

Closed
jamesli2021 opened this issue Sep 2, 2022 · 13 comments
Closed

cmd/compile: "too many open files in system" on macos Ventura #54833

jamesli2021 opened this issue Sep 2, 2022 · 13 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Go1.20

Comments

@jamesli2021
Copy link

I'm running macOS Ventura and update to the latest beta build to test with Go 1.19. There could be a bug in Go reading files which might be affected when macOS Ventura is release to the public. Go 1.18.5 is working fine.

What version of Go are you using (go version)?

go version go1.19 darwin/arm64

Does this issue reproduce with the latest release?

Yes, 1.19

What operating system and processor architecture are you using (go env)?

macOS Ventura, M1 (arm)

What did you do?

Compiling a simple Go project and I get "too many open files in system".

What did you expect to see?

Able to compile as Go 1.18.5 which is working fine without error.

What did you see instead?

"... too many open files in system" appear in the terminal when ran "go build ." command.
@heschi heschi changed the title affected/package: Go 1.19 on macos Ventura cmd/compile: "too many open files in system" on macos Ventura Sep 2, 2022
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Sep 2, 2022
@heschi heschi added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 2, 2022
@heschi heschi added this to the Go1.20 milestone Sep 2, 2022
@heschi heschi added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Sep 2, 2022
@heschi
Copy link
Contributor

heschi commented Sep 2, 2022

cc @golang/compiler

Please provide a full reproducer, log, and the output of go env.

@jamesli2021
Copy link
Author

Here go env output:

go env                   
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/s/Library/Caches/go-build"
GOENV="/Users/s/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/s/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/s/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.19"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/s/Documents/aaa/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/q6/vgzzlsv56j5_jk6_ysj8ld0c0000gn/T/go-build4248313789=/tmp/go-build -gno-record-gcc-switches -fno-common"

@seankhliao seankhliao added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Sep 9, 2022
@gopherbot
Copy link

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

@yyuyitian
Copy link

i meet the same problem with go version 1.19.2

@sunznx
Copy link

sunznx commented Dec 15, 2022

I solved by remove $GOCACHE dir

go env get GOCACHE

@tstarbow-sfdc
Copy link

tstarbow-sfdc commented Feb 22, 2023 •
edited

go clean -cache

Solved a similar issue for me.

@alecthomas
Copy link

Same issue here, and same fix:

🐚 ~/dev/ftl $ find /Users/alec/Library/Caches/go-build
🐚 ~/dev/ftl $ go build ./cmd/ftl-controller
../../Library/Caches/hermit/pkg/go-1.20.5/src/crypto/crypto.go:9:2: open /Users/alec/Library/Caches/hermit/pkg/go-1.20.5/src/hash: too many open files in system
🐚 ~/dev/ftl $ go env get GOCACHE
/Users/alec/Library/Caches/go-build
🐚 ~/dev/ftl $ rm -rf ~/Library/Caches/go-build aat/ingress-directive
🐚 ~/dev/ftl $ go build ./cmd/ftl-controller
🐚 ~/dev/ftl $

This sounds like a classic case of not closing a file somewhere?

@ianlancetaylor
Copy link
Contributor

@alecthomas Can you provide the go env output on your system? Thanks.

@alecthomas
Copy link

For sure. I'm a bit annoyed that I didn't take a backup of the GOCACHE directory.

GO111MODULE=""
GOARCH="arm64"
GOBIN="/Users/alec/dev/ftl/.hermit/go/bin"
GOCACHE="/Users/alec/Library/Caches/go-build"
GOENV="/Users/alec/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/alec/go/pkg/mod"
GONOPROXY="*.sqcorp.co,github.com/squareup"
GONOSUMDB="*.sqcorp.co,github.com/squareup"
GOOS="darwin"
GOPATH="/Users/alec/go"
GOPRIVATE="*.sqcorp.co,github.com/squareup"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/alec/Library/Caches/hermit/pkg/go-1.20.5"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/alec/Library/Caches/hermit/pkg/go-1.20.5/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.20.5"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/alec/dev/ftl/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/h2/_9k30gds1dbdjsv1m3yw834w0000gn/T/go-build2041056486=/tmp/go-build -gno-record-gcc-switches -fno-common"

@alecthomas
Copy link

An extra bit of potentially useful information is that it would always report this error on the exact same directory (open /Users/alec/Library/Caches/hermit/pkg/go-1.20.5/src/hash). I also did a bunch of lsofing, and checked the ulimits on my machine and couldn't see anything obviously misconfigured.

@ianlancetaylor
Copy link
Contributor

I note that the reports that provide the information are all darwin/arm64. Is anybody seeing this problem on a different system?

@ncdc
Copy link

ncdc commented Jul 24, 2023

I just ran into it on darwin/amd64. go clean -cache fixed it for me.

@xlanor
Copy link

xlanor commented Jul 25, 2023 •
edited

ran into this on darwin/amd64 too, when my autogeneration failed midway. Likewise, go clean -cache fixed it.

My ulimit was originally at 256 but the problem persisted even after raising it to 10,000.

Am also on macos ventura.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Go1.20
Development

No branches or pull requests