If I write the benchmark without concurrency:

package benchhttp

import (
    "io"
    "io/ioutil"
    "net/http"
    "net/http/httptest"
    "testing"
)

func Benchmark(b *testing.B) {
    data := []byte("Foobar")
    srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
        w.Write(data)
    }))
    defer srv.Close()
    for i := 0; i < b.N; i++ {
        resp, err := http.Get(srv.URL)
        if err != nil {
            b.Fatal(err)
        }
        io.Copy(ioutil.Discard, resp.Body)
        resp.Body.Close()
    }
}

It works (no crash).

And the value displayed by watch "ss -a | wc -l" is low and stable (around 700).

I suspect you are exceeding the number of local socket connections permitted by your OS.

I suspect you are exceeding the number of local socket connections permitted by your OS.

You are probably right, but I can't explain why.

I've performed the test again, with differents GOMAXPROCS values, and I displayed the number of network connection:

➜  ss -a | wc -l                                         
758
➜  GOMAXPROCS=1 go test -bench=. -benchmem -benchtime=10s
testing: warning: no tests to run
Benchmark     500000         39875 ns/op        3980 B/op         56 allocs/op
PASS
ok      _test/benchhttp 20.326s
➜  ss -a | wc -l                                         
783
➜  GOMAXPROCS=2 go test -bench=. -benchmem -benchtime=10s
testing: warning: no tests to run
Benchmark-2       500000         23642 ns/op        3994 B/op         56 allocs/op
PASS
ok      _test/benchhttp 12.074s
➜  ss -a | wc -l                                         
789
➜  GOMAXPROCS=3 go test -bench=. -benchmem -benchtime=10s
testing: warning: no tests to run
Benchmark-3     --- FAIL: Benchmark-3
    benchhttp_test.go:21: Get http://127.0.0.1:38675: dial tcp 127.0.0.1:38675: connect: cannot assign requested address
FAIL
exit status 1
FAIL    _test/benchhttp 34.238s
➜  ss -a | wc -l                                         
29863

Most connections are in the TIME-WAIT state.

Should I configure something on my system or fix my code?

The point of a parallel benchmark is to run as many iterations of the function, in parallel, as will complete in 1 second. The benchmark will keep ramping up the number of iterations until it finds the answer. On your system, it seems that the answer is: more than the system can handle.

I would suggest that you put a limit in your code on the number of simultaneous open connections.

I'm going to close this issue because at this point I don't see anything to be fixed in Go.

I would suggest that you put a limit in your code on the number of simultaneous open connections.

As far as I know, there are only 8 simultaneous HTTP requests in my code.
My GOMAXPROCS value is 8 (default value) and RunParallel() runs GOMAXPROCS concurrent benchmarks by default.

Does net/http.Get() opens a new connection for each request?

http.Get keeps a cache of TCP connections, but when all are in use it opens another one. It also sets a limit on the number of idle connections to a given host; the default is 2. If your GOMAXPROCS is more than 2 the benchmark is going to be regularly discarding connections and opening new ones. Each closed connection will be in TIME_WAIT state for two minutes, tying up that connection.

Try adding this line to your benchmark:

    http.DefaultTransport.(*http.Transport).MaxIdleConnsPerHost = 100

It works!
Your explanation makes sense.
Thank you very much ! 👍

ss -s should show high number of tcp connections with much of them in TIMEWAIT .
Allow the kernel to recycle and reuse these connections if not your limit of sockets will be exhausted

Add these 2 lines to your sysctl.conf .

net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1

The sysctl way causes issues:
dial tcp 127.0.0.1:5001: i/o timeout

The MaxIdleConnsPerHost helped.