Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: HTTP requests with InsecureSkipVerify transport don't close TCP connections #16267

Closed
knadh opened this issue Jul 5, 2016 · 1 comment

Comments

@knadh
Copy link

knadh commented Jul 5, 2016

  1. What version of Go are you using (go version)?
    go version go1.6.2 linux/amd64
  2. What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/user/code/go"
GORACE=""
GOROOT="/usr/lib/go-1.6"
GOTOOLDIR="/usr/lib/go-1.6/pkg/tool/linux_amd64"
GO15VENDOREXPERIMENT="1"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"
  1. What did you do?

Run the following program. It creates an HTTP server on :8000 and spawns a goroutine that sends GET requests to the server every 1 second. The server writes nothing in the response body; body-length = 0. The client makes GET requests and does a body.Close() without reading the body. Note that the transport has InsecureSkipVerify.

package main

import (
    "crypto/tls"
    "fmt"
    "log"
    "net/http"
    "time"
)

func Server(w http.ResponseWriter, req *http.Request) {
    // If a response is written, the issue isn't triggered.
    // io.WriteString(w, "hello, world!\n")
}

func Client(url string) {
    req, err := http.NewRequest("GET", url, nil)
    if err != nil {
        log.Fatal(err)
    }

    // This transport is what's causing unclosed connections.
    tr := &http.Transport{
        TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
    }
    hc := &http.Client{Timeout: 2 * time.Second, Transport: tr}

    resp, err := hc.Do(req)
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()
}

func main() {
    // Wait a bit and launch one client every second.
    // On the terminal, run `ss | grep :8000` to see
    // ESTABLISHED connections grow with every request.

    go func() {
        time.Sleep(1 * time.Second)

        n := 0
        for {
            Client("http://localhost:8000")
            fmt.Println("Request", n)
            n += 1

            time.Sleep(1 * time.Second)
        }
    }()

    http.HandleFunc("/", Server)
    log.Fatal(http.ListenAndServe(":8000", nil))
}
  1. What did you expect to see?
    Every request the client makes should get a 200 response from the server and immediately close the connection.
  2. What did you see instead?
    The client keeps making requests and gets 200 back, but the underlying TCP connection is never closed. It's stuck in the ESTABLISHED state perpetually.

Check the open connections by running ss | grep :8000 every few seconds. They keep growing. netstat n | grep :8000 also shows TIME_WAIT connections.

  • Writing a response in Server() solves the issue.
  • Removing InsecureSkipVerify solves the issue.
@knadh knadh changed the title net/http: HTTP requests with InsecureSkipVerify transport doesn't close connections net/http: HTTP requests with InsecureSkipVerify transport don't close TCP connections Jul 5, 2016
@bradfitz
Copy link
Contributor

bradfitz commented Jul 5, 2016

This has nothing to do with InsecureSkipVerify.

You're leaking Transports which are holding TCP connections.

See the documentation under https://golang.org/pkg/net/http/#Transport :

By default, Transport caches connections for future re-use. This may leave many open connections when accessing many hosts. This behavior can be managed using Transport's CloseIdleConnections method and the MaxIdleConnsPerHost and DisableKeepAlives fields.

Transports should be reused instead of created as needed. Transports are safe for concurrent use by multiple goroutines.

Don't create a Transport for every request.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants