net/http: Server rejects CONNECT requests without a Host header, per the spec #18215
Comments
bradfitz
changed the title
|
Does iOS itself have this bug, or certain app(s)? Did an older version of various specs permit CONNECT without a host header? I'm inclined to say that you can use http.ReadRequest and handle this yourself if you're a proxy supporting buggy clients, unless you can convince me that this is widespread. /cc @tombergan |
bradfitz
added
Thinking
WaitingForInfo
|
I am going to need some more time to look at some proper tcpdumps but we see this issue with iOS (set to use a HTTP proxy written in Go) and the native iOS Facebook app. I have heard rumours that Facebook may bypass the default iOS networking stack but I don't have any concrete evidence. The original proxy spec written by Ari Luotonen in 1998 did not require a I do have a branch that I can submit with a sample 1-line fix. |
|
|
|
I understand that Ari's draft uses HTTP 1.0 but the spec for CONNECT (https://tools.ietf.org/html/rfc2817, 2000) was only superseded by the HTTP 1.1 (https://tools.ietf.org/html/rfc7230, 2014). At the time of RFC2817 CONNECT was a reserved method but not official. My fix looks like this: |
|
Yes, the change itself is trivial. (adding a test will likely be much more code) This bug isn't about figuring out the change, though, but about making a decision on whether we should, understanding how wide-spread this is or if it was ever allowed. If you want a change to go in, convince me of either:
Related: #14952 is about adding opt-in knobs for accepting malformed HTTP. |
|
Understood. We will have more data in the upcoming weeks as we start more in-depth testing across the entire app ecosystem. Breaking the native Facebook app is a doozy though. |
|
Facebook might be big enough on its own to make this worth changing. Let's still figure out whose networking stack it is first. Then, if Facebook, we can ask Facebook to fix it, or at least see what they say. Maybe they'll just fix it quickly. If it's Apple, that's a bit harder, and almost certainly worth adding a special case at that point. |
|
I can take care of gathering the data for this issue in the next few days and will update here. |
|
@johnmah, any update? |
|
Just came here to add that while I'm sure this won't count as widespread, I'm encountering this exact problem with wget-1.13.4-3+deb7u2 (see https://bugs.launchpad.net/ubuntu/+source/wget/+bug/994097). This is in a corporate environment where I can't just go an upgrade to a working version. |
|
@walkert, what did you just link me to? I see nothing about Go or CONNECT or HTTP/1.0 or HTTP/1.1 there. Are you on the wrong bug? |
|
@bradfitz, I thought you were looking for more cases of clients that were not sending 'Host' headers when making a CONNECT request. The linked bug relates to version of wget which exhibit this behaviour though I can see reading through it now that it isn't detailed. I'll see if I can dig up something more official. |
|
Yeah, that bug doesn't mention "Host" or "header" or "version" or "CONNECT". Is it tracking an upstream bug or something? Hah, and the bottom comment from "Admin (ilw10)" is such an internet comment. Yet somehow that angry dude can understand what's happening in that bug more than me. |
|
Hah - yeah that's for sure. OK - so it would seem that wget was updated way back when - (http://git.savannah.gnu.org/cgit/wget.git/commit/src?id=aed7d4163a9e2083d294a9471e1347ab13d6f2ab) but alas the problem I have in my environment is dealing with old sources where I don't have the luxury of upgrading everyone (the version we have available pre-dates that update). I realize you're doing the right thing here and clients should be adhering to the specs, but it would certainly be nice if we had the option to determine how vehemently net/http enforces the rule(s). Incidentally, I've tried the modification above followed by a 'go build -a' and various other permutations and haven't been able to get the updates into my newly built executable. Would appreciate any pointers (this is on go 1.8). Cheers. |
|
+1 here. Working with legacy embedded devices, many of these also leave out the Host in requests. |
|
@thomasberger, any details on those devices or software/versions? |
|
@bradfitz sorry for not updating earlier but i got pulled into other projects. not sure what the best way to gather information but i did grab output from tcpdump between a proxied iPhone and the proxy server. 96.45.199.251.54971 = iPhone 10.3.2 handset |
|
@johnmah, thanks. That's unfortunate. I guess this warrants fixing, then. |
bradfitz
added
NeedsFix
|
CL https://golang.org/cl/44004 mentions this issue. |
Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (
go version)?1.7.4 (problem exists on master branch as well)
What operating system and processor architecture are you using (
go env)?linux-amd64
What did you do?
Sample code here: https://play.golang.org/p/dpLc5rnHJZ
We have created a proxy server using the net/http package. If a HTTP client using the proxy server issues a
CONNECTrequest without aHost:header, the golang HTTP server implementation will close the connection with400 Bad Request: missing required Host headererror.as opposed to:
Now this could be a contentious issue as the HTTP 1.1 spec (see https://tools.ietf.org/html/rfc7230#section-5.4) specifies that a Host header is mandatory, but we see some VERY popular devices (i.e.: iOS devices) implement proxy clients that do not send the
Host:header when configured to use proxies (and as a result issueCONNECTrequests). This could be an artifact of interop with older HTTP 1.0 proxies.It appears that the HTTP request code in
src/net/http/request.gowill synthesize a Host attribute based on theCONNECTparameters (authority-form) and ignores anyHost:header, so relaxing the Host header restriction would allow for better compatibility. As it stands now, any HTTP server implementing some form of CONNECT handler might not work with a large range of devices.What did you expect to see?
Client
CONNECTrequest completes as expected and leaves connection open for further requests.What did you see instead?
net/http Server implementation returns a
400 Bad Request: missing required Host headerand closes the connection.The text was updated successfully, but these errors were encountered: