Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: too many open files #28272

Closed
ecavalcanti opened this issue Oct 18, 2018 · 5 comments
Closed

net/http: too many open files #28272

ecavalcanti opened this issue Oct 18, 2018 · 5 comments
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.

Comments

@ecavalcanti
Copy link

What version of Go are you using (go version)?

go1.11.1

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

  • amd64
  • darwin
  • MacOS High Sierra (10.3.6)

What did you do?

The error too many open files is being thrown by submitting 20 requests per second in the code below.
The limit of open files on my system is 256 ( ulimit -n ).
Using lsof -p the FD is growing and not released.
If the OS open files limit is incremented the problem just is postponed. And I think that increment is not the solution.

package main

import (
	"log"
	"net/http"

	"golang.org/x/crypto/bcrypt"
)

func main() {

	srv := &http.Server{
		Addr: ":9000",
	}

	http.HandleFunc("/auth", func(w http.ResponseWriter, r *http.Request) {
		log.Printf("/auth")
		if err := bcrypt.CompareHashAndPassword([]byte("$2y$10$MWTmFsLIM3jHVNiDDZH/U.qkpebJ/z02phdclx9rEcz5B7/cbujcy"), []byte("test")); err != nil {
			log.Printf("Error on check key and secret. (%v)", err.Error())
			w.WriteHeader(http.StatusUnauthorized)
			return
		}

		w.WriteHeader(http.StatusOK)
	})
	log.Printf("Listening on port 9000")
	log.Fatal(srv.ListenAndServe())

}

What did you expect to see?

The FD be released and not throw too many open files error.

What did you see instead?

http: Accept error: accept tcp [::]:9000: accept: too many open files.

@ianlancetaylor ianlancetaylor changed the title Too many open files net/http: too many open files Oct 18, 2018
@bradfitz
Copy link
Contributor

If your server can't do 20 qps because its CPU is busy with, say, bcrypt but your load testing tool (which? you don't say.) keeps generating new connections, I would expect it to fail like this.

I'm not sure there's anything to do here.

@bradfitz bradfitz added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Oct 18, 2018
@ecavalcanti
Copy link
Author

ecavalcanti commented Oct 18, 2018

I am using the artillery to run the tests:

artillery quick -d 60 -r 20 http://localhost:9000/auth

@bradfitz
Copy link
Contributor

Well, the load testing tool has taught you something: you now know what your server can't do. Binary search on the -r value a bit and find the server's limit.

What do you want me to do with this bug?

@wsc1
Copy link

wsc1 commented Oct 18, 2018

caddy suggests to up the ulimit on fd's for production use. It seems to me like an OS config issue and not a bug in Go or it's std lib

@agnivade agnivade added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Mar 8, 2019
@henriquemenezes
Copy link

henriquemenezes commented Apr 1, 2019

Hi @ecavalcanti .

You should:

  • Increase ulimit of fd's in production environment. You can find fd limit running cat /proc/sys/fs/file-max.
  • Disable the keep alive of the net/http server.
  • Set Read and Write deadline timeouts.
  • Make sure that you closed all fd's in request handler.

The solution should look like this:

package main

import (
	"log"
	"net/http"
	"time"

	"golang.org/x/crypto/bcrypt"
)

func main() {
	srv := &http.Server{
		Addr:         ":9000",
		ReadTimeout:  5 * time.Second,
		WriteTimeout: 10 * time.Second,
	}

	srv.SetKeepAlivesEnabled(false)

	http.HandleFunc("/auth", func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Connection", "close")

		defer r.Body.Close()

		log.Printf("/auth")
		if err := bcrypt.CompareHashAndPassword([]byte("$2y$10$MWTmFsLIM3jHVNiDDZH/U.qkpebJ/z02phdclx9rEcz5B7/cbujcy"), []byte("test")); err != nil {
			log.Printf("Error on check key and secret. (%v)", err.Error())
			w.WriteHeader(http.StatusUnauthorized)
			return
		}

		w.WriteHeader(http.StatusOK)
	})
	log.Printf("Listening on port 9000")
	log.Fatal(srv.ListenAndServe())
}

@bradfitz bradfitz closed this as completed Apr 2, 2019
@golang golang locked and limited conversation to collaborators Apr 1, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

6 participants