Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: os: API to detect that the read end of a pipe was closed #26049

Open
ibukanov opened this issue Jun 25, 2018 · 13 comments
Open

proposal: os: API to detect that the read end of a pipe was closed #26049

ibukanov opened this issue Jun 25, 2018 · 13 comments

Comments

@ibukanov
Copy link

Currently there is no API in Go to detect that the read end of a pipe was closed without writing to the pipe a non-empty slice. This prevents, for example, to write in a safe Go a version of GNU tail utility that exits immediately when it detects that the read end of its stdout was closed even when it waits for more input like when it is called via tail -f. For example, given the following case:

tail -f file-to-follow | grep -q foo

GNU tail exits immediately after the grep finds the word foo and terminates without waiting that the file-to-follow will be extended and trying to write those extra bytes to stdout.

Yet to write such functionality in Go one needs to use unsafe code to call sys.Select or similar and wait for an syscall.EPIPE from the stdout descriptor.

What version of Go are you using (go version)?

go1.10.3

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

amd64 linux fedora-28

@ianlancetaylor ianlancetaylor changed the title API to detect that the read end of a pipe was closed os: API to detect that the read end of a pipe was closed Jun 25, 2018
@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 25, 2018
@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Jun 25, 2018
@ianlancetaylor
Copy link
Contributor

Do you have a proposal for how to address this?

@ibukanov
Copy link
Author

An API that is sufficient to implement that tail-like functionality is to add to os.signal something like

func ClosedPipe(pipe *os.File) (<-chan struct{}, error)

that returns a channel that becomes ready when the read end of the pipe closes. It is very OK if this only works with pipes where SetWriteDeadline works, i.e. the pipes that the poller supports.

Alternatively the API can take a callback that is called when the pipe is closed, i.e. like

func ClosedPipe(pipe *os.File, callback func()) error

The nice thing of this form is that one can pass there CancelFunc from context.WithCancel() as a typical reaction to the closed read end of the pipe is to stop current activities that in future result in writes to the pipe.

@robpike
Copy link
Contributor

robpike commented Jun 25, 2018

How do I detect that the read end has been closed?

@ibukanov
Copy link
Author

@robpike On Linux when the read end is closed poll/epoll syscalls for the write end of the pipe return POLLERR/EPOLLERR.

@robpike robpike closed this as completed Jun 26, 2018
@cespare
Copy link
Contributor

cespare commented Jun 26, 2018

@robpike was this close intentional?

@robpike robpike reopened this Jun 26, 2018
@ibukanov
Copy link
Author

ibukanov commented Jun 26, 2018

On systems with kqueue the interface also provides notifications about the closed read end of the pipe. But on Windows it seems with anonymous pipes the only way to support this is via periodic calls to the write function with zero-length buffer. At least the question on StackOverflow has not got a better answer, https://stackoverflow.com/questions/11564166/detecting-if-anonymous-pipe-is-writable-to-detect-when-to-end-process

@FiloSottile FiloSottile added Proposal and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jun 26, 2018
@FiloSottile FiloSottile modified the milestones: Unplanned, Proposal Jun 26, 2018
@robpike
Copy link
Contributor

robpike commented Jun 26, 2018

Almost 50 years on and we still don't understand how to handle a closed pipe. It's remarkable how much energy has been consumed on this topic, how many crashes caused, how many solutions proposed.

I would be more comfortable about this if there was a non-epoll, non-signal-catching way to learn about a closed pipe. Is there?

@ibukanov
Copy link
Author

@robpike there is no such API besides listening for epoll/poll/kqueue events.

@robpike
Copy link
Contributor

robpike commented Jun 27, 2018

@ibukanov I am not surprised. As I said, 50 years on....

@ianlancetaylor
Copy link
Contributor

Can some API along these lines work for streaming network connections as well?

I don't see why this should be a function, it seems like it ought to be a method on the *os.File or net.TCPConn. I don't know if sending a value on a channel is right, but I don't have any better suggestion. I don't think a callback is a good idea.

If this really can't be implemented on Windows, that sounds like a real problem for the whole idea.

@ianlancetaylor ianlancetaylor changed the title os: API to detect that the read end of a pipe was closed proposal: os: API to detect that the read end of a pipe was closed Oct 14, 2020
@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) Oct 14, 2020
@ibukanov
Copy link
Author

In principle this can be implemented on Windows via periodic zero-length writes to the pipe. This is wasteful, but at least it is doable if one needs it.

@ianlancetaylor
Copy link
Contributor

Zero-length writes to the pipe should also work on Unix systems.

@ianlancetaylor
Copy link
Contributor

That is, if the application wants to find this out with some work, it can do zero-length writes. I don't think the Go runtime should be doing zero-length writes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Incoming
Development

No branches or pull requests

5 participants