New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
io: Pipe reader loses last line if writer exits as soon as write is finished #33297
Comments
Try adding the following to the end of the program: if err := cmd.Wait(); err != nil {
fmt.Print(err)
} |
If add And I also tries some other ways to avoid this:
The 2 ways above will stop the pipe reader losting the last line, I don't know why. |
This got nothing to do with If your goal is to have a process monitor the output of another program, it is easier to just make it the parent process. |
Please try the test case, the parent program do not terminate a long running child process if it just // parent.go
package main
import (
"fmt"
"io"
"os/exec"
)
func main() {
cmd := exec.Command("./child")
pr, pw := io.Pipe()
cmd.Stdin = pr
err := cmd.Start()
if err != nil {
fmt.Print(err)
return
}
n := 100
for i := 1; i <= n; i++ {
str := fmt.Sprintf("line %d\n", i)
pw.Write([]byte(str))
}
pw.Close()
} // child.go
package main
import (
"bufio"
"fmt"
"os"
"time"
)
func main() {
f, _ := os.Create("./child.log")
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
time.Sleep(time.Second)
fmt.Fprintln(f, scanner.Text())
}
f.Close()
} I try this steps: $ go build child.go
$ go run parent.go
$ tail -f child.log and will see the child process is still running, and it's PPID is 1.
And my user case is like this:
|
Ahh, sorry about that. |
So I found that if you use https://golang.org/pkg/os/exec/#Cmd.StdinPipe instead of creating your own pipes, the child process gets all 100 lines. Can you verify that works on your end too? |
I don't know what is the different between the |
@sunchao9 - has @AlexRouSg's suggestion addressed this issue? |
@julieqiu It's not entirely clear why there is or should there be a difference in behavior. I think at a minimum there should be a doc change if the difference is intentional. |
The suggestion could resolve my problem, thanks a lot. And I also agree with @AlexRouSg said, it would be nice if any one who want to use |
@sunchao9 Could you confirm with an strace log that the last I'd be surprised if there's a bug here, since this functionality is pretty close to the metal, and it's been stable for a long time. There's no error handling in this program, and I suspect at least one of the write calls is returning an error which is being ignored. |
Sorry about that, I have updated the issue using the template. And I add some trace log to check if the n := 100
for i := 1; i <= n; i++ {
str := fmt.Sprintf("line %d\n", i)
fmt.Print("writer: " + str)
n, err := pw.Write([]byte(str))
fmt.Printf("result = %d, %v\n", n, err)
}
pw.Close() and the returns just fine, there is no error happened, but the I also tried this morning under MacOS, the |
@sunchao9 Thanks for adding that info. I looked into this a little closer, and I think I understand what's going wrong. If you look at the documentation for io.Pipe, it's an in-memory pipe which basically passes byte slices from a writer to a reader on a channel. It's not a pipe in the UNIX sense; that's os.Pipe, which returns *os.File. The documentation for os/exec.Cmd explains the difference in behavior.
Because Closing because this is working as designed. Process synchronization is always a little tricky, but there are some useful examples in the |
I think this is a bug of
io.Pipe
, I want to use pipe files to write strings to the detached child process, but I find that the pipe reader always lost the last input line, here is the test case:What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
yes, go1.12.7 is the latest release version
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
I have a
writer.go
and areader.sh
, thewriter.go
writes 100 lines toreader.sh
which is started as a detached process. And I find that thereader.sh
always lost the last line unless thewriter.go
sleep a while before exit.What did you expect to see?
I expect to see the
reader.sh
will read all the 100 lines from it's Stdin, because it's Stdin is a pipe reader, and the pipe writer received 100 lines from the parent process.What did you see instead?
I run the
writer.go
to check if thereader.sh
could print all the 100 lines, but thereader.sh
lost the last line, and I tried lots of times, mostly it happens.and the
reader.log
's content is like this:So, I think there is a BUG under some OS platform (I tried the MacOS,
io.Pipe
works find) , if the pipe writer exit as soon as possible, the pipe reader will lost the last line. Please check if it is true, thansk a lot.The text was updated successfully, but these errors were encountered: