New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
net/http: rejects requests where headers have whitespace #14392
Comments
Tangentially, is Chef aware that their HTTP API is not standards-compliant? If not, I'd suggest filing an issue with them too. |
@mdempsky |
There is also a fork of the Chef Server written in Go that is currently only compliant with the So while I am personally only working with the API, that project will no longer function on Go 1.6+. |
I really don't want to accept spaces, and I really don't want to add new API surface to whitelist broken things. http2 is explicitly against this too. Would Chef 11.x ever use http2 in any sort of deployment? What if we just explicitly whitelist Alternatively we do nothing and say if you don't want to upgrade from Chef 11 to Chef 12, don't upgrade from Go 1.5 to Go 1.6 either and keep your old software stack old. (kinda joking, but kinda not) |
I agree. I guess the question I was posing here is: are we OK in saying we do not wish to support older specifications that do not implement this HTTP restriction? I'm thinking about cases where these specs were written when this wasn't explicitly forbidden. My example of the Chef server being one of them.
Unfortunately, I can't speak on behalf of the Chef community. But to spitball, I don't believe they would, no.
We could whitelist it, I suppose. It feels absolutely gross, and I'd probably shy away from it. I would not want to litter the stdlib with things like that. If we did decide to go the whitelist route, that still leaves Go in a state that may be broken for other projects. If we're going to whitelist one non-compliant header, we should give the ability for others to do the same. But then there goes more in to the
Maybe. We use the Goiardi project listed above, and it works well for us in specific cases where we don't want to stand up a Chef server with all of its dependencies. We do want to upgrade to Chef 12, there are just a lot of yaks in the way. This may be bigger than just the Chef project, but it just so happened to be the case where I ran in to it. |
It's been there since at least 1999, if not earlier: https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2 That's not really new.
You're talking about API surface again, though. I don't think this problem is actually very widespread. We'll see, though. |
In RFC 2616 from 1999:
Note that SP (space) was always forbidden. |
Yeah, seen. I was reading that specific section of the RFC, and didn't have the token definition in front of me. Using the full-form document made it clear. Unfortunately too clear. Well then, I guess it would appear that I'm SOL. 😄 |
I'm still fine with putting a whitelist of shame with a single entry in the code. |
I've been thinking about this since your initial response, and I still am in agreement that requests with headers like this should be rejected. But I wonder if it was necessary to make this change in a way that cannot be managed by the person consuming In the document, Go 1 and the Future of Go Programs, there is the following statement made in the introduction:
Of course, it is intended that they do not change. In no way is it stating this is a binding statement. That's also confirmed based on a later section:
I think it was necessary to update Any favorite whiskies I could bribe you with to allow one small API addition to give consumers control of the whitelist? 😄 Honestly, the more I think about having the one item added to a whitelist the more gross it feels. I'd rather not go down that path. Is there a way it could be implemented that feels natural and doesn't add much weight? I'd be happy to offer my time to implement it and go through the CR process. |
Are you sure this is the problem. We had a long chat about this on slack and it appears that the https://docs.chef.io/release/oec_11-2/api_chef_server.html I might be reading this wrong. If so, please point me to the correct documentation for the version of the server you have to talk to. |
This is how I interpret it as well. v11.0 of the Chef Server docs: https://docs.chef.io/release/oec_11-0/auth.html#header-format (similar to what Dave has linked).
|
Thanks, @davecheney and @elithrar. Yeah, reading those docs, it sounds like whatever problem @theckman is having is not about spaces. What do you actually seeing happening on the wire? |
Yeah. I found my issue late last night I believe. I plan on looking at the workaround today. I just didn't get a chance to comment here. It is very possible there are now zero examples of this issue, so we may end up closing it. Let me look right now. |
Confirmed. This ended up being a bug in the The implementation appears to be accidentally including the header with the request. Normally, there isn't any harm in doing so as I'm not sure of any other servers that outright reject requests if there is a space in the header. So we're down to zero cases. I think that may be the sign that this issue isn't important. The Chef issue was just my example. I still think that this failure should not be abstracted away from consumers, as they should have the final say on the request (as well as the error page that gets returned by the server). This just stems from two desires:
Seeing as I was the only reporter, and my issue was a problem that was actually an incidental bug, maybe we should just ignore this pending any further reports. |
If you want to change errors you can implement your own ResponseWriter and
|
@elithrar I've been using |
@theckman Glad to hear this ended up working out. I like that these validations are now in place. It prevents people from building APIs that use invalid headers. At least if they are using Go. 😄 |
@nathany So based on that should |
I haven't verified it, but I believe it does, and personally I think it should. This is based on a 17-year old RFC and you've already found that the whitespace in your case is related to signing a request and not actual HTTP headers. We have no evidence that any server anywhere expects whitespace in header keys (and if any do, they are clearly broken). Happy to discuss further on Slack, but I'd like to be respectful of Brad's time. He has 56 other open issues to deal with. |
Which error? Was any error mentioned in this bug? Too many topics have been discussed here. Let's close this one and issue separate focused bugs from this. Please include details in the new bugs though with code and output. |
Congratulations on the Go 1.6 release! With the new release of Go, a behavior has been introduced that cannot be disabled and causes some of my HTTP requests to fail:
Researching further, I do see that there is an RFC dealing with whitespace in headers. Specifically section 3.2.3 of this document: https://tools.ietf.org/html/rfc7230.
However, this change has seemingly introduced backwards incompatibility with software that I have written against go1.5.3. Specifically, software that is using the same authentication mechanism as the
11.x
release of the Chef server.In their authentication mechanism, they have a header called
Hashed Path
which is used as part of the authentication and verification of the request. Without this header, the requests fail. I'm not only writing a client that must use this authentication mechanism, but also a server that consumes requests that match this spec.While I agree that enforcement of the RFC is important, there is no way to guarantee people are not in violation of the RFC due to its fairly recent inception. I'm not asking for the feature to be removed, but wonder if there is a way we could toggle this option for an
*http.Server
.The text was updated successfully, but these errors were encountered: