Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: option to allow Trailer header without chunked transfer encoding in HTTP response #27197

Closed
VictoriaRaymond opened this issue Aug 24, 2018 · 9 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@VictoriaRaymond
Copy link

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.10

Does this issue reproduce with the latest release?

Yes, as the code is not changed.

What operating system and processor architecture are you using (go env)?

Mac OS 10.13 (don't matter actually)

What did you do?

Read HTTP response via http.ReadResponse(). The HTTP response contains a "Trailer" header but no "Transfer-Encoding" head. This leads to an error during header parsing.

It is not possible to reproduce using the original URL for http request/response, as the URL requires authentication in advance. Here is a simplified code using static real response.

https://play.golang.org/p/nuL0CnYF1tu

What did you expect to see?

Since response body is not chunked in such case, the "Trailer" header can be considered (as least optionally) as a "no-op". It would be nice that net/http can ignore such error (by default, or with an option), and let caller to decide whether the response is valid or not.

RFC 7230 doesn't say anything regarding such case. It only mentions positive case when Transfer-Encoding is used.

Reference:
https://tools.ietf.org/html/rfc7230#section-4.4
https://tools.ietf.org/html/rfc7230#section-4.1.2

What did you see instead?

ReadResponse() returns ErrUnexpectedTrailer, which is originated from here: https://github.com/golang/go/blob/master/src/net/http/transfer.go#L761

For the moment, the caller has no choice but to discard the HTTP response.

@meirf
Copy link
Contributor

meirf commented Aug 25, 2018

@VictoriaRaymond, if we removed this error, at the very least we would want to know some more detail in terms of prioritizing the change. Can you give more details on why the server is setting trailer without chunked encoding?

@meirf meirf added this to the Unplanned milestone Aug 25, 2018
@Dreamacro
Copy link

@meirf This error occurs when downloading application on App Store with macOS. Although this request may be non-standard, there is no way to get response (return nil) when it's error. So i hope golang can provide a solution that can still access Response when ReadResponse get error

@meirf
Copy link
Contributor

meirf commented Aug 25, 2018

@Dreamacro, thanks. do you have a url we can look at?

@Dreamacro
Copy link

@meirf

The error was found in the App Store, such as download https://itunes.apple.com/cn/app/autodesk-sketchbook/id863486266?l=en&mt=12
Host is osxapps.itunes.apple.com, which looks like an API for verifying signatures
The returned headers can be found at https://play.golang.org/p/nuL0CnYF1tu

@meirf
Copy link
Contributor

meirf commented Aug 25, 2018

If you have a momemt, can you include an executable example of the code you're running against the url? (I've tried a few different ways to get the response, but I'm not seeing the exact headers you are seeing. Perhaps the response is different due to geography; I'm in the US, your profile shows China. There could be totally different server infrastructure. For example, my requests are being served by an akamai cdn.)

@bradfitz
Copy link
Contributor

Thanks for the https://play.golang.org/p/nuL0CnYF1tu demo.

I don't want to add more configuration option complexity. And I agree it's a little lame that users can't read these responses. This is easy enough to fix once it's decided what to do. I suppose we can just keep the Trailer value in the Response.Header map but not populate Response.Trailer.

@bradfitz bradfitz modified the milestones: Unplanned, Go1.12 Aug 26, 2018
@bradfitz bradfitz added the NeedsFix The path to resolution is known, but the work has not been done. label Aug 26, 2018
@gmichelo
Copy link
Contributor

I would love to fix this. Can I try?

@bradfitz
Copy link
Contributor

Sure

@gopherbot
Copy link

Change https://golang.org/cl/145398 mentions this issue: net/http: not to return error with Trailer header when no chunked

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

6 participants