net/http: slow io.Copy(http.ResponseWriter, *os.File) with Content-Length #41513
Comments
ianlancetaylor
changed the title
ianlancetaylor
added
help wanted
NeedsInvestigation
|
Interesting test case. I'm not sure what is going on here. I can see that If I use a much larger file, rather than the Perhaps |
|
The problem is in The fix can be enable chunking even in case |
|
On macos I am getting both benchmarks close to each other. |
|
@darkLord19 That's because macOS doesn't have @cuonglm Thanks for spotting where the behavior changes. I don't think it would be right to always enable chunking because as noted above using I think we need to understand why CC @acln0 for any thoughts. |
|
When I tried to reproduce on my machine, I got this: ... which looks quite reasonable. I don't know what is going on here. I have a theory: I notice that the benchmark calls Lines 2962 to 2976 in 617f2c3 The server sleeping for extended periods of time may account for the sendfile code path looking really really slow. I cannot imagine sendfile needing ~50ms to send a small file otherwise. I have not been able to confirm my hypothesis, but I would kindly ask @Ryan-A-B to change the inner loop of the benchmark to this: and try again. Perhaps with this change, the benchmark would have more sensible results. |
I don't mean we always enable chunking, but still enable chunking when we detect there's Line 1448 in 23573d0 Which happens if Line 1464 in 23573d0 which seems useless, since when chunking is only set if Note The slow performance come from this line: Line 616 in 23573d0 |
|
@Ryan-A-B Just to make sure we are on the same page, I want to clarify that what I see is that larger files are faster in |
|
Running Note the huge jump from |
|
My Linux kernel version is 5.7.17. |
|
strace client and server separately, show there is a 200ms delay between sendfile and recvfrom. |
|
FYI, I got this on my Windows PC. |
|
@ianlancetaylor up to 512 bytes they're equal then the timing from 513 up 256 byte file BenchmarkFileWithoutContentLength-8 20000 104532 ns/op BenchmarkFileWithContentLength-8 20000 90245 ns/op 512 byte file BenchmarkFileWithoutContentLength-8 20000 94685 ns/op BenchmarkFileWithContentLength-8 20000 86804 ns/op 513 byte file BenchmarkFileWithoutContentLength-8 20000 96399 ns/op BenchmarkFileWithContentLength-8 20 52140028 ns/op 1024 byte file BenchmarkFileWithoutContentLength-8 20000 97876 ns/op BenchmarkFileWithContentLength-8 20 52145628 ns/op 1.2MB file BenchmarkFileWithoutContentLength-8 1000 1242620 ns/op BenchmarkFileWithContentLength-8 2000 1389453 ns/op 3.6MB file BenchmarkFileWithoutContentLength-8 500 2606823 ns/op BenchmarkFileWithContentLength-8 1000 1653781 ns/op |
Since the Content-Type detection sniff 512 bytes, so less 512 bytes will not invoke sendfile. |
|
my mac os benchmark： |
if the count param of sendfile larger than the file size to be sent, the TCP will cause a 200ms delay somehow. Fixes golang#41513 Fixes golang#45256
|
Change https://golang.org/cl/305229 mentions this issue: |
I tried this - this breaks HTTP server for range GETs etc - I have seen similar behavior with Regularly gets blocked, which seems to block and delay subsequent calls as well. This increases overall response times for calls - with unexpected latencies. My OS is CentOS Linux release 8.3.2011 |
|
cc @neild |
|
hey guys , what is going on here? Problem solved? |
|
Rediscovered as #61530 |
What version of Go are you using (
go version)?Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env)?go envOutputWhat did you do?
package http_test import ( "io" "io/ioutil" "log" "net/http" "net/http/httptest" "os" "strconv" "testing" ) func runHandlerBenchmark(b *testing.B, handler http.Handler) { server := httptest.NewServer(handler) defer server.Close() client := server.Client() for i := 0; i < b.N; i++ { response, err := client.Get(server.URL) if err != nil { log.Fatal(err.Error()) } _, err = ioutil.ReadAll(response.Body) if err != nil { log.Fatal(err.Error()) } } } type HandlerFile struct { callback func(response http.ResponseWriter, file *os.File) } func (handler *HandlerFile) ServeHTTP(response http.ResponseWriter, request *http.Request) { file, err := os.Open("./main_test.go") if err != nil { log.Fatal(err.Error()) } defer file.Close() handler.callback(response, file) _, err = io.Copy(response, file) if err != nil { log.Fatal(err.Error()) } } func BenchmarkFileWithoutContentLength(b *testing.B) { doNothing := func(response http.ResponseWriter, file *os.File) {} handler := HandlerFile{ callback: doNothing, } runHandlerBenchmark(b, &handler) } func BenchmarkFileWithContentLength(b *testing.B) { addContentLength := func(response http.ResponseWriter, file *os.File) { stat, err := file.Stat() if err != nil { log.Fatal(err.Error()) } response.Header().Set("Content-Length", strconv.FormatInt(stat.Size(), 10)) } handler := HandlerFile{ callback: addContentLength, } runHandlerBenchmark(b, &handler) }What did you expect to see?
Both benchmarks to take the same amount of time
What did you see instead?
The text was updated successfully, but these errors were encountered: