-
Notifications
You must be signed in to change notification settings - Fork 18k
net/http/httputil: proxied streaming responses appear to lose some data on the way to the client #51646
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I don't see this is a Go project's issue. |
Hi. Well, the issue only happens in browser clients. So data sent by the Go server is somehow corrupted along the way. I wouldn't be able to reproduce in a play env. I'm not certain if this is a browser bug or something in the reverse proxy or net lib. But I can reproduce in different browsers, which led me to believe that the issue must be on the server side. Is there something wrong with the details I've shared in that repo? Are you not able to reproduce? |
Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only. For questions please refer to https://github.com/golang/go/wiki/Questions for {
line, err := reader.ReadBytes('\n')
if err != nil {
log.Fatal(err)
} It appears that your client can't split messages correctly when your data/content contains \n. |
@mengzhuo there may be an actual bug within one of the net libs. I would assume someone would help look into it. I'm not asking a question as to how to stream responses or read them as that's already done and working. I'm experiencing an actual issue where the browser based clients do not receive the full data. I can look into it myself, I just need some guidance so that I know what to look for or where I should be looking. I've already made some effort in trying to explain as best as I could how to reproduce. I would at least expect the same effort on your end to check it out and tell me with certainty that the issue is probably on the client side. I'm definitely aware this is an open source project and you or other contributors have no obligation to help, but if this is a bug, I'm sure you'd like to fix it at some point. I would appreciate it if you didn't close this issue as I think it's relevant. For context, it's very similar to #27816. The referenced client actually works just fine. I was unable to replicate the issue with it. So the issue seems to happen strictly when using a browser. So I wanted to verify that there's no issue on the server side and then report the bug in the appropriate channels for each browser where it happens. |
Sure, This triage is based on my understanding of your code. cc @neild net/http owner |
I appreciate that! Thank you! As mentioned, I'm more than happy to look into it if someone points me in the right direction. |
我也遇到类似的情况。使用 rust 反向代理测试正常,但是使用 httputil 就会出错〜,模拟发送http请求也不行。像是go的 http 实现有问题或有默认的参数设置问题,目前找不到原因。 |
I've reported the issue at https://bugs.chromium.org/p/chromium/issues/detail?id=1305928 as well. I've also ran more tests just using net/http and net/http/httputil to see if the issue could be in echo, but I could still get failures. So that leaves either the browsers or net/http (since I can get failures from net/http w/o using a proxy). |
Given that this doesn't need a reverse proxy to reproduce, it doesn't look like an issue with
and
spread over 2 reads Closing as this doesn't appear to be a bug with Go (or Chrome for that matter, just improper use of streaming APIs) |
@seankhliao ok, got it. I was under the impression that the browser would read line by line at every occurrence of a But I guess you're right since I found a handling text line by line example which illustrates how to handle chunks not ending on line boundaries. It was just not very clear from the docs that the readable stream doesn't necessarily split chunks at line endings. Thanks for taking the time to check the code I shared and giving more insight into the issue! |
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
I'm about to test that today.
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
I've setup a simple echo server and I'm using streaming responses to stream data to clients through another proxy server.
Everything seemed to work fine, but I later noticed that after some time (25 - 30 mins), browser clients would get a:
I initially thought the issue was in labstack/echo#2125. But then I also tried to use the net/http/httputil reverse proxy directly and I could still reproduce the error. So I think the issue could either be in the reverse proxy, the net/http server or could be in the browser.
I've tried my best to describe all the steps to reproduce the error at transcelestial/echo-streaming-bug.
I'll try to remove echo and use a simple server to check if the issue still persists.
What did you expect to see?
I don't expect to see any errors or data loss on the client end.
What did you see instead?
I get
after ~ 25 mins.
The text was updated successfully, but these errors were encountered: