Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net/http: Post http://example.com: net/http: HTTP/1 transport connection broken: readLoopPeekFailLocked: EOF #15446

Closed
erikdubbelboer opened this issue Apr 26, 2016 · 5 comments
Closed

net/http: Post http://example.com: net/http: HTTP/1 transport connection broken: readLoopPeekFailLocked: EOF #15446

erikdubbelboer opened this issue Apr 26, 2016 · 5 comments
Labels
FrozenDueToAge
Milestone
Go1.7

Comments

@erikdubbelboer
Copy link
Contributor

Please answer these questions before submitting your issue. Thanks!

  1. What version of Go are you using (go version)?
go version devel +d78c84c Mon Apr 25 23:22:56 2016 +0000 linux/amd64
  1. What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/erik"
GORACE=""
GOROOT="/home/erik/src/github.com/golang/go"
GOTOOLDIR="/home/erik/src/github.com/golang/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build769409056=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
  1. What did you do?

Send HTTP POST requests to a host that seems to close the connection some specific time after the response. We see the error Post http://example.com: net/http: HTTP/1 transport connection broken: readLoopPeekFailLocked: EOF multiple times per second when doing around 100 requests per second.

It's hard to debug on live traffic and I haven't been able to reproduce this in a clean example.

The error seems to come from roundTrip() which means the request is already in flight. But it also comes from readLoopPeekFailLocked() which is called from readLoop() in the case of no expected responses.

I'm guessing the connection is closed after a request. Because pc.readLoopPeekFailLocked(err) is called after err is wrapped in a beforeRespHeaderError{err} the peekErr == io.EOF in readLoopPeekFailLocked() is false and the error gets assigned to pc.closed while the connection is already in use. After that I'm not sure, the connection must have somehow already been chosen for the next request. Otherwise it wouldn't make sense that pc.numExpectedResponses == 0 while we are inside the loop to wait for a response.

Is there any debugging (logging things) I can do on the live traffic to help find this bug?
@bradfitz bradfitz self-assigned this Apr 28, 2016
@bradfitz bradfitz added this to the Go1.7 milestone Apr 28, 2016
@bradfitz
Copy link
Contributor

Thanks for the report. I think this should be fixable with the information provided.

@bradfitz bradfitz mentioned this issue May 13, 2016
Closed
@owenthereal
Copy link

@bradfitz Is this fixed on master?

@bradfitz
Copy link
Contributor

Not yet. This bug is still open. Commits will be referenced from here.

@gopherbot
Copy link

CL https://golang.org/cl/23160 mentions this issue.

@bradfitz bradfitz mentioned this issue May 17, 2016
Closed
@woria woria mentioned this issue Jun 17, 2016
Closed
@gopherbot
Copy link

CL https://golang.org/cl/25153 mentions this issue.

gopherbot pushed a commit that referenced this issue Jul 26, 2016
@bradfitz
net/http: make Transport.RoundTrip return raw Conn.Read error on peek…
ea2376f 
… failure

From at least Go 1.4 to Go 1.6, Transport.RoundTrip would return the
error value from net.Conn.Read directly when the initial Read (1 byte
Peek) failed while reading the HTTP response, if a request was
outstanding. While never a documented or tested promise, Go 1.7 changed the
behavior (starting at https://golang.org/cl/23160).

This restores the old behavior and adds a test (but no documentation
promises yet) while keeping the fix for spammy logging reported in #15446.

This looks larger than it is: it just changes errServerClosedConn from
a variable to a type, where the type preserves the underlying
net.Conn.Read error, for unwrapping later in Transport.RoundTrip.

Fixes #16465

Change-Id: I6fa018991221e93c0cfe3e4129cb168fbd98bd27
Reviewed-on: https://go-review.googlesource.com/25153
Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@andersbetner andersbetner mentioned this issue Aug 15, 2016
Closed
@golang golang locked and limited conversation to collaborators Jul 23, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
Go1.7
Development

No branches or pull requests
4 participants
@bradfitz @owenthereal @erikdubbelboer @gopherbot