os/exec: "broken pipe" on testlog.txt while running test #35469
Comments
bcmills
added
Testing
I don't recall what that is. os/exec seems to only run subprocesses with a fresh argv, though, not inheriting that value from cmd/go. |
|
This likely means that the (The testlog file is used to log what the test does in order to see whether the test results can be cached.) |
|
I tried to do something like, diff --git a/src/os/exec/exec_test.go b/src/os/exec/exec_test.go
index 19d2111743..aa8bb5c97b 100644
--- a/src/os/exec/exec_test.go
+++ b/src/os/exec/exec_test.go
@@ -470,9 +470,14 @@ func basefds() uintptr {
return n
}
+func isTestLogFD(fd uintptr) bool {
+ target, _ := os.Readlink(fmt.Sprintf("/proc/self/fd/%v", fd))
+ return strings.HasSuffix(target, "testlog.txt")
+}
+
func closeUnexpectedFds(t *testing.T, m string) {
for fd := basefds(); fd <= 101; fd++ {
- if poll.IsPollDescriptor(fd) {
+ if poll.IsPollDescriptor(fd) || isTestLogFD(fd) {
continue
}
err := os.NewFile(fd, "").Close()But it never fires, because I then see we already have: // basefds returns the number of expected file descriptors
// to be present in a process at start.
// stdin, stdout, stderr, epoll/kqueue, epoll/kqueue pipe, maybe testlog
func basefds() uintptr {
n := os.Stderr.Fd() + 1
// The poll (epoll/kqueue) descriptor can be numerically
// either between stderr and the testlog-fd, or after
// testlog-fd.
for poll.IsPollDescriptor(n) {
n++
}
for _, arg := range os.Args {
if strings.HasPrefix(arg, "-test.testlogfile=") {
n++
}
}
return n
}So is I'd snapshot Perhaps we can do that conditional on being on the builders for now? |
|
It's interesting that the |
|
More examples on the MIPS builders: |
|
Change https://golang.org/cl/206939 mentions this issue: |
Our attempts to close existing open files are flaky. They will fail if, for example, file descriptor 3 is open when the test binary starts. Instead, report any such cases, and skip TestExtraFiles. Updates #35469 Change-Id: I7caec083f3f4a31579bf28fc9c82ae89b1bde49a Reviewed-on: https://go-review.googlesource.com/c/go/+/206939 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
Change https://golang.org/cl/207078 mentions this issue: |
It turns out that there is a path that initializes netpoll and opens file descriptors before running the os/exec init function: on some systems, the uses of NewFile when setting os.Stdin and friends can initialize netpoll which can open file descriptors. This in itself is not a problem, but when we check whether the new files are open using os.NewFile, a side-effect is to put them into non-blocking mode. This can then break future uses of netpoll. Updates #35469 Fixes #35566 Change-Id: I1b2e2c943695d1c2d29496b050abbce9ee710a00 Reviewed-on: https://go-review.googlesource.com/c/go/+/207078 Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
This is rare and has been happening for a long time. The CLs should give us more information if it happens again. Kicking to 1.15. |
|
We haven't heard much in the past 6 months from bug reports. Moving this Go1.16 and perhaps then we can move it to Unreleased or decide how to proceed from then on. |
|
Punting to Go1.17. |
|
This hasn't happened since November, 2019. Let's say that it's fixed. |
Several examples with this failure mode:
I suspect that this has something to do with
exec'ing the test itself as a subprocess.CC @bradfitz @ianlancetaylor
2019-11-08T20:05:25-7a5e0fe/linux-386-sid
2019-11-08T19:28:49-f6ff806/linux-386-sid
2019-11-07T08:25:32-7a2baa9/linux-mips-rtrk
2019-11-01T03:40:20-4a43a50/linux-386-sid
The text was updated successfully, but these errors were encountered: