I suspect the sendfile implementation for Solaris is broken, then.

Can you modify src/net/sendfile_solaris.go and add a return statement on the first line of func sendfile?

func sendFile(c *netFD, r io.Reader) (written int64, err error, handled bool) {
    return

And see if that fixes it? (it would revert to Read+Write)

/cc @4ad

@bradfitz You nailed it. I added a log line and the return statement, and I can now see the file being served correctly. Really appreciate the fast diagnosis.

Could you please try with the latest SmartOS image? Thanks.

@4ad sure thing, tested both versions on the latest base-64 image: SunOS 5.11 joyent_20151002T091442Z i86pc i386 i86pc Solaris

The current version fails as on the earlier SmartOS image, and the patched version suggested by @bradfitz works.

Just for confirmation, you tested on a recent SmartOS image, a.i. a recent kernel, not an older SmartOS image with a recent base-64 image?

@4ad the tests were all run on instances in the Joyent Public Cloud, base-64 is the name of their default basic 64-bit SmartOS image. joyent_20151002T091442Z is SmartOS Base Image v15.3.0. I can donate an instance for testing, if that helps.

EDIT: FWIW, I just rebuilt Consul with net/sendfile patched, and it now too works as expected.

Perhaps we should just disable sendfile on Solaris for Go 1.6 as a safe option. We can re-enable in Go 1.7 once we understand and fix the problem.

I think that is a good idea

On Tue, 12 Jan 2016, 05:57 Brad Fitzpatrick notifications@github.com
wrote:

Perhaps we should just disable sendfile on Solaris for Go 1.6 as a safe
option. We can re-enable in Go 1.7 once we understand and fix the problem.

—
Reply to this email directly or view it on GitHub
#13892 (comment).

Moved milestone to Go1.7. In Go 1.6, sendfile will just be disabled for Solaris and it will fall back to the path that works.

Any news here?

+cc @binarycrusader

I'll look into this.

Have reproduced on Solaris development build using decompressed Mark.Twain-Tom.Sawyer.txt.bz2 from src/compress/bzip2/testdata.

It consistently fails after the first 262,144 characters (256KiB), where it starts to the repeat the input all over again.

So the root cause of this issue is that the implementation of sendfile() for Solaris is not correct. In particular, for sizes beyond the size of the socket buffer, the same data will be sent (data_len / socket_buffer_size) + (data_len % socket_buffer_size) times.

This is only vaguely referenced in the Solaris sendfile(3c) man page. The FreeBSD sendfile(2) man page however makes this quite clear:

When using a socket marked for non-blocking I/O, sendfile() may send
fewer bytes than requested.  In this case, the number of bytes success-
fully written is returned in *sbytes (if specified), and the error    EAGAIN
is returned.

As a result, I believe the implementations for the following platforms may not actually be correct:

• dragonfly
• freebsd
• solaris

I am currently testing to determine whether any other platforms need a similar fix.

So my new test works on Solaris, but did not fail as expected on DragonFly/FreeBSD, despite the fact that they clearly do not account for the EAGAIN scenario. I would welcome any insight others might have to as to why it did not fail. The only guess I have is that the file is not big enough to trigger the failure on FreeBSD.

CL https://golang.org/cl/21769 mentions this issue.