Issue golang#40888 describes my motivation for this patch:

. Rather than probe and guess if sendfile will work, fix the underlying
  issue of staring to respond before src is readable

. Do not send a status OK if header has not yet been written and reading
  from src is destined to fail

. Implicitly takes care of needsSniff() case

This allows splice to work on linux when src is a socket or other
spliceable non regular file.

The extra read of 512 bytes may raise an objection, and that's fair, but
we're already swapping some syscall prep work for another and a read of
512 probably will not impact the overall performance. For shorter
bodies, there's likely less setup time. A little initial slop is not that
unusual in zero copy network code, and sometimes actually helps.

I personally think this way is slightly less surprising and does not try
to peer into special knowledge implemented by readers and writers.

Issue golang#40888 describes my motivation for this patch:

. Rather than probe and guess if sendfile will work, fix the underlying
  issue of staring to respond before src is readable

. Do not send a status OK if header has not yet been written and reading
  from src is destined to fail

. Implicitly takes care of needsSniff() case

This allows splice to work on linux when src is a socket or other
spliceable non regular file.

The extra read of 512 bytes may raise an objection, and that's fair, but
we're already swapping some syscall prep work for another and a read of
512 probably will not impact the overall performance. For shorter
bodies, there's likely less setup time. A little initial slop is not that
unusual in zero copy network code, and sometimes actually helps.

I personally think this way is slightly less surprising and does not try
to peer into special knowledge implemented by readers and writers.

Issue golang#40888 describes my motivation for this patch:

. Rather than probe and guess if sendfile will work, fix the underlying
  issue of staring to respond before src is readable

. Do not send a status OK if header has not yet been written and reading
  from src is destined to fail

. Implicitly takes care of needsSniff() case

This allows splice to work on linux when src is a socket or other
spliceable non regular file.

The extra read of 512 bytes may raise an objection, and that's fair, but
we're already swapping some syscall prep work for another and a read of
512 probably will not impact the overall performance. For shorter
bodies, there's likely less setup time. A little initial slop is not that
unusual in zero copy network code, and sometimes actually helps.

I personally think this way is slightly less surprising and does not try
to peer into special knowledge implemented by readers and writers.

Issue golang#40888 describes my motivation for this patch:

. Rather than probe and guess if sendfile will work, fix the underlying
  issue of staring to respond before src is readable

. Do not send a status OK if header has not yet been written and reading
  from src is destined to fail

. Implicitly takes care of needsSniff() case

This allows splice to work on linux when src is a socket or other
spliceable non regular file.

The extra read of 512 bytes may raise an objection, and that's fair, but
we're already swapping some syscall prep work for another and a read of
512 probably will not impact the overall performance. For shorter
bodies, there's likely less setup time. A little initial slop is not that
unusual in zero copy network code, and sometimes actually helps.

I personally think this way is slightly less surprising and does not try
to peer into special knowledge implemented by readers and writers.

Issue golang#40888 describes my motivation for this patch:

. Rather than probe and guess if sendfile will work, fix the underlying
  issue of starting to respond before src is readable

. Do not send a status OK if header has not yet been written and reading
  from src is destined to fail

. Implicitly takes care of needsSniff() case

This allows splice to work on linux when src is a socket or other
spliceable non regular file.

The extra read of 512 bytes may raise an objection, and that's fair, but
we're already swapping some syscall prep work for another and a read of
512 probably will not impact the overall performance. For shorter
bodies, there's likely less setup time. A little initial slop is not that
unusual in zero copy network code, and sometimes actually helps.

Issue golang#40888 describes my motivation for this patch:

. Rather than probe and guess if sendfile will work, fix the underlying
  issue of starting to respond before src is readable

. Do not send a status OK if header has not yet been written and reading
  from src is destined to fail

. Implicitly takes care of needsSniff() case

This allows splice to work on linux when src is a socket or other
spliceable non regular file.

The extra read of 512 bytes may raise an objection, and that's fair, but
we're already swapping some syscall prep work for another and a read of
512 probably will not impact the overall performance. For shorter
bodies, there's likely less setup time. A little initial slop is not that
unusual in zero copy network code, and sometimes actually helps.

Issue golang#40888 describes my motivation for this patch:

. Rather than probe and guess if sendfile will work, fix the underlying
  issue of starting to respond before src is readable

. Do not send a status OK if header has not yet been written and reading
  from src is destined to fail

. Implicitly takes care of needsSniff() case

This allows splice to work on linux when src is a socket or other
spliceable non regular file.

The extra read of 512 bytes may raise an objection, and that's fair, but
we're already swapping some syscall prep work for another and a read of
512 probably will not impact the overall performance. For shorter
bodies, there's likely less setup time. A little initial slop is not that
unusual in zero copy network code, and sometimes actually helps.

Issue golang#40888 describes my motivation for this patch:

. Rather than probe and guess if sendfile will work, fix the underlying
  issue of starting to respond before src is readable

. Do not send a status OK if header has not yet been written and reading
  from src is destined to fail

. Implicitly takes care of needsSniff() case

This allows splice to work on linux when src is a socket or other
spliceable non regular file.

The extra read of 512 bytes may raise an objection, and that's fair, but
we're already swapping some syscall prep work for another and a read of
512 probably will not impact the overall performance. For shorter
bodies, there's likely less setup time. A little initial slop is not that
unusual in zero copy network code, and sometimes actually helps.

review feedback: use copyBufPool in both cases

Use the same pool for the normal copy case as we do for the short read.

Clean up and clarify comments.

clean up code addressing feedback

pass sendfile test

the sendfile path will require >512 bytes to kick in, so send a 1024
byte file for this test.

obey copy semantics

pattern the initial read and write after io.Copy(), paying attention to
either side's error and writing what has been read regardless.

This could be done with an io.LimitedReader with io.Copy() except that
returns the total bytes written, while a ReaderFrom needs to return the
total bytes read. While we could adjust for this by looking at the
limited reader's state, being explicit this way is probably more clear.

create sendfile test data in temp dir

Remove accidentally added 1024k file which was meant to be a 1024b file.
Generate the test data in a temporary directory regardless.

Issue golang#40888 describes my motivation for this patch:

. Rather than probe and guess if sendfile will work, fix the underlying
  issue of starting to respond before src is readable

. Do not send a status OK if header has not yet been written and reading
  from src is destined to fail

. Implicitly takes care of needsSniff() case

This allows splice to work on linux when src is a socket or other
spliceable non regular file.

The extra read of 512 bytes may raise an objection, and that's fair, but
we're already swapping some syscall prep work for another and a read of
512 probably will not impact the overall performance. For shorter
bodies, there's likely less setup time. A little initial slop is not that
unusual in zero copy network code, and sometimes actually helps.

Rather than probe and guess if sendfile will work inside ResponseWriter.ReadFrom(src),
this change fixes the underlying issue of starting to respond before src is readable
We'll no longer send a status OK if a header has not yet been written and reading
from src is destined to fail. This small change implicitly takes care of the need for
the server to sniff the response body to determine the Content-Type.

This allows splice to work on Linux when src is a socket or any non-regular file that's spliceable.

The extra read of 512 bytes may raise an objection, and that's fair, but
we're already swapping some syscall prep work for another and a read of
512 probably will not impact the overall performance. For shorter
bodies, there's likely less setup time. A little initial slop is not too
unusual in zero copy network code, and sometimes actually helps.

Fixes golang#40888

Change-Id: I4a8e2ad0ace1318bae66dae5671d06ea6d4838ed
GitHub-Last-Rev: 097364e
GitHub-Pull-Request: golang#40903
Reviewed-on: https://go-review.googlesource.com/c/go/+/249238
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>