Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os: (*os.File).SetDeadline() doesn't work after io.Copy(*net.TCPConn, *os.File) #28330

Closed
crvv opened this issue Oct 23, 2018 · 4 comments
Closed
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@crvv
Copy link
Contributor

crvv commented Oct 23, 2018

What version of Go are you using (go version)?

go version go1.11.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Linux AMD64

What did you do?

This is the same as #22934
io.Copy() uses net.sendFile to do the work, which calls (*os.File).Fd().

written, err = poll.SendFile(&c.pfd, int(f.Fd()), remain)

To reproduce, run the following code.

package main

import (
	"fmt"
	"io"
	"net"
	"os"
	"time"
)

func main() {
	r, w, err := os.Pipe()
	if err != nil {
		panic(err)
	}
	go func() {
		_, err := w.Write([]byte("123"))
		if err != nil {
			panic(err)
		}
		time.Sleep(time.Second * 4)
		_, err = w.Write([]byte("456"))
		if err != nil {
			panic(err)
		}
	}()
	go func() {
		l, err := net.Listen("tcp", "localhost:1234")
		if err != nil {
			panic(err)
		}
		defer l.Close()
		conn, err := l.Accept()
		if err != nil {
			panic(err)
		}
		defer conn.Close()
		_, err = io.CopyN(conn, r, 3)
		if err != nil {
			panic(err)
		}
	}()
	go func() {
		time.Sleep(time.Second / 5)
		c, err := net.Dial("tcp", "localhost:1234")
		if err != nil {
			panic(err)
		}
		defer c.Close()
		_, err = io.Copy(os.Stdout, c)
		if err != nil {
			panic(err)
		}
		fmt.Println()
	}()
	time.Sleep(time.Second / 2)

	deadline := time.Now().Add(time.Second)
	err = r.SetDeadline(deadline)
	if err != nil {
		panic(err)
	}
	buffer := make([]byte, 4)
	n, err := r.Read(buffer)
	if err != nil {
		panic(err)
	}
	fmt.Printf("got %v, deadline: %v, now: %v\n", string(buffer[:n]), deadline.Unix(), time.Now().Unix())
}

What did you expect to see?

panic: read |0: i/o timeout

What did you see instead?

got 456, deadline: 1540286817, now: 1540286820

I never encountered this in real applications.
I guess there are more similar issues around Fd() and the runtime poller.
Maybe it's better to just leave this to Go 2 and make Fd not touch the blocking mode.

@flyingtang
Copy link

i get this:

123
panic: read |0: i/o timeout

goroutine 1 [running]:
main.main()
        /Users/xg/scan/src/sssss/main.go:66 +0x355exit status 2

@ianlancetaylor ianlancetaylor added the NeedsFix The path to resolution is known, but the work has not been done. label Oct 23, 2018
@ianlancetaylor ianlancetaylor added this to the Go1.12 milestone Oct 23, 2018
@crvv
Copy link
Contributor Author

crvv commented Oct 24, 2018

@txg5214
I guess you tested this on macOS.
The bug only occurs on platforms where sendfile is used for copying *os.File to *net.TCPConn.
Those are Linux, DragonflyBSD, FreeBSD, Solaris and Windows.

https://github.com/golang/go/blob/master/src/net/sendfile_linux.go
https://github.com/golang/go/blob/master/src/net/sendfile_unix_alt.go
https://github.com/golang/go/blob/master/src/net/sendfile_windows.go

@acln0
Copy link
Contributor

acln0 commented Dec 18, 2018

I don't think much can be done here (without adding pointless new API) except to wait for Go 2.

To do sendfile, the net package needs to get at the file descriptor of the os.File, so that it can pass it down into poll.Sendfile. It can't get at its underlying poll.FD, so it has to make due with the integer file descriptor returned by Fd. Calls to Fd have to set the description in blocking mode for historical reasons.

In any case, I doubt this is a problem for any real program. Callers would have to both use a memory mappable descriptor in non-blocking mode, and want to call sendfile on it. And if it really is a problem, it can be worked around using unix.Setnonblock after io.Copy, since the blocking mode is a property of the file description rather than the file descriptor.

@gopherbot
Copy link

Change https://golang.org/cl/155137 mentions this issue: os: don't let sendFile put a pipe into blocking mode

@golang golang locked and limited conversation to collaborators Dec 28, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

5 participants