Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: HTTP client hangs indefinitely if the request handler does not return #65526

Open
tkashem opened this issue Feb 5, 2024 · 1 comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@tkashem
Copy link

tkashem commented Feb 5, 2024

Go version

go version go1.21.6 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/akashem/.cache/go-build'
GOENV='/home/akashem/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/akashem/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/akashem/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/lib/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/lib/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.6'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/akashem/go/src/k8s.io/kubernetes/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3786731428=/tmp/go-build -gno-record-gcc-switches'

What did you do?

I have a test where the client hangs indefinitely waiting for a response/error from the server, this is the setup of the test:

  • protocol: HTTP/1x
  • the client does not set any timeout on its side, the client request context does not have a deadline
  • on the server, per-request read (100ms) and write (200ms) deadline is set for the request
  • the request handler blocks indefinitely on a channel that is closed by the client when it receives a response from the server
func TestClientHangingForeverWithHTTP1(t *testing.T) {
	clientDoneCh, handlerDoneCh := make(chan struct{}), make(chan struct{})
	server := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
		defer close(handlerDoneCh)

		ctrl := http.NewResponseController(w)
		if err := ctrl.SetWriteDeadline(time.Now().Add(200 * time.Millisecond)); err != nil {
			t.Errorf("expected no error from SetWriteDeadline, but got: %v", err)
			return
		}
		if err := ctrl.SetReadDeadline(time.Now().Add(100 * time.Millisecond)); err != nil {
			t.Errorf("expected no error from SetReadDeadline, but got: %v", err)
			return
		}

		<-clientDoneCh
	}))

	defer server.Close()
	server.StartTLS()

	client := server.Client()
	func() {
		defer close(clientDoneCh)
		client.Get(server.URL + "/foo")
	}()

	select {
	case <-handlerDoneCh:
	case <-time.After(time.Minute):
		t.Errorf("expected the request handler to have terminated")
	}
}

What did you see happen?

The client waits indefinitely for a response from the server. SetWriteDeadline does not result in the connection being reset or closed after the deadline exceeds.

It seems the request handler in http/1x is not executed in a new goroutine by design

go/src/net/http/server.go

Lines 2031 to 2039 in b8ac61e

// HTTP cannot have multiple simultaneous active requests.[*]
// Until the server replies to this request, it can't read another,
// so we might as well run the handler in this goroutine.
// [*] Not strictly true: HTTP pipelining. We could let them all process
// in parallel even if their responses need to be serialized.
// But we're not going to implement HTTP pipelining because it
// was never deployed in the wild and the answer is HTTP/2.
inFlightResponse = w
serverHandler{c.server}.ServeHTTP(w, w.req)

What did you expect to see?

For http2, SetWriteDeadline will result in the stream being reset and thus unblocking the client after write timeout, even though the handler is blocked. The same version of the test does not fail with http2

Is this working as expected for http/1x?

There are clients that do not configure any timeout, these clients will hang indefinitely, or until the kernel TCP timeout takes effect. We are trying to replace the use of TimeoutHandler in the kubernetes apiserver with the ResponseController.
Please let us know if you have any suggestions, thanks!

@seankhliao
Copy link
Member

cc @neild

I think timeout with a 503 status and writedeadline are different operations

@seankhliao seankhliao added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

2 participants