Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: widespread deadlocks involving subprocesses #35375

Closed
bcmills opened this issue Nov 5, 2019 · 6 comments
Closed

runtime: widespread deadlocks involving subprocesses #35375

bcmills opened this issue Nov 5, 2019 · 6 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker Soon This needs to be done soon. (regressions, serious bugs, outages)
Milestone
Go1.14

Comments

@bcmills
Copy link
Contributor

bcmills commented Nov 5, 2019

Ever since CL 204800, which may or may not be directly related, the linux-arm64-packet builder has been failing consistently.

The failures affect various tests; the only consistent pattern I can see among them is that the tests are blocked waiting for some child process to exit, generally in os.(*Process).blockUntilWaitable.

CC @ianlancetaylor @mknyszek
@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker Soon This needs to be done soon. (regressions, serious bugs, outages) labels Nov 5, 2019
@bcmills bcmills added this to the Go1.14 milestone Nov 5, 2019
@bcmills
Copy link
Contributor Author

bcmills commented Nov 5, 2019

linux-386: https://build.golang.org/log/ca4e429817cd924771991d08cb82d0540708b93b
linux-amd64: https://build.golang.org/log/07a9ef5a59aa3a726def455678d1b5e3ed61d4c7

@bcmills
Copy link
Contributor Author

bcmills commented Nov 5, 2019

os.(*Process).wait on darwin is suspiciously similar.

darwin-amd64-10_14: https://build.golang.org/log/7917b648a72c359e8e63d8bbfefcb23d5c514516
darwin-amd64-10_12: https://build.golang.org/log/3eefa6817bbc7f42c3551955ad29075ee7773447

@bcmills
Copy link
Contributor Author

bcmills commented Nov 5, 2019

And even windows-amd64-2016: https://build.golang.org/log/3f64d9f90c37c55dce95f426a8ec7fdcf7dde9ed

goroutine 41 [syscall, 5 minutes, locked to thread]:
syscall.Syscall(0x7ffe15cc42c0, 0x2, 0x224, 0xffffffff, 0x0, 0x0, 0x0, 0x0)
	C:/workdir/go/src/runtime/syscall_windows.go:188 +0xe9
syscall.WaitForSingleObject(0x224, 0xc0ffffffff, 0x0, 0x0, 0x5)
	C:/workdir/go/src/syscall/zsyscall_windows.go:738 +0x6b
os.(*Process).wait(0xc0002e7a70, 0x0, 0x0, 0x0)
	C:/workdir/go/src/os/exec_windows.go:18 +0x87
os.(*Process).Wait(...)
	C:/workdir/go/src/os/exec.go:125

@bcmills bcmills changed the title runtime: linux-arm64-packet consistently deadlocking runtime: deadlock in os.(*Process).blockUntilWaitable Nov 5, 2019
@bcmills bcmills changed the title runtime: deadlock in os.(*Process).blockUntilWaitable os: deadlock in (*Process).Wait Nov 5, 2019
@ianlancetaylor
Copy link
Contributor

I don't see any particular reason to think this is a deadlock in (*Process).Wait itself. It seems rather more likely to be a problem causing the child process to deadlock or otherwise fail to exit in time.

@bcmills bcmills changed the title os: deadlock in (*Process).Wait runtime: widespread deadlocks involving subprocesses Nov 5, 2019
@bcmills
Copy link
Contributor Author

bcmills commented Nov 5, 2019

Fair enough. From the variety of tests I had thought that some of the subprocesses were non-Go programs, but now I don't think I actually had any evidence to support that assumption.

@gopherbot
Copy link

Change https://golang.org/cl/205558 mentions this issue: runtime: don't hold scheduler lock when calling timeSleepUntil

@bcmills bcmills mentioned this issue Nov 6, 2019
Closed
@bcmills bcmills mentioned this issue Nov 12, 2019
Closed
@bcmills bcmills mentioned this issue Dec 18, 2019
Closed
@golang golang locked and limited conversation to collaborators Nov 5, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker Soon This needs to be done soon. (regressions, serious bugs, outages)
Projects
None yet
Milestone
Go1.14
Development

No branches or pull requests
3 participants
@ianlancetaylor @bcmills @gopherbot