New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
net/http/httputil: io.Copy in function of ReverseProxy is not high efficiency #10277
Comments
The problem with your code is that it has to read all the data into memory before writing it out. That's not something we should do unconditionally. To make it safe you'd have to find a way to do it only when the reader size is limited. Wrap in a LimitedReader, perhaps? I don't understand why your code is faster. Perhaps io.Copy could be improved. It allocates 32KB chunks, maybe that's not the right size for your test case. (too large? too small? not sure.) More info about your queries would help. |
@randall77 there is only 50 or 128 or 1024 bytes in my http request's body. in my http service, the http handle is like this:
Don't consider the safe, I just want to show which way is better. I do not modify the http ReverseProxy code, and I find in
the problem is the same, so I write the code to prove. |
@randall77 the go version is 1.4, and I use LimitedReader, the result is the same with the Reader. and I see when I use (io.Copy or io.CopyN), the server's cpu is more than 500%(12 cpus), the QPS is 24000, but when I use (ReadAll + Write), the server's cpu is only 300%, and the QPS is 28000. I my really server, I see the same phenomenon. in my ServeHTTP, I replace the io.Copy to ReadAll+Write, the effect is very good. but I my web router server which use ReverseProxy, the code is fficial code, i do not modify it, but it cost more cpus. |
-> bradfitz to decide whether to close |
Perhaps we can add an optional hook to |
@bradfitz, now I see when I test in Ali cloud host or Microsoft azure, the Performance of io.Copy is the same with the (ReadAll + Write), but in baidu cloud host, the Performance of io.Copy is bad, so maybe it is the system problem, not a code problem, i do not know |
CL https://golang.org/cl/9399 mentions this issue. |
in funtion
in boom test the QPS is 160000, but use another way like this:
in boom test the QPS is 190000.
The text was updated successfully, but these errors were encountered: