Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: netpoll failed on android/386 #29423

Closed
eliasnaur opened this issue Dec 26, 2018 · 9 comments
Closed

runtime: netpoll failed on android/386 #29423

eliasnaur opened this issue Dec 26, 2018 · 9 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.13

Comments

@eliasnaur
Copy link
Contributor

From the android/386 builder:

https://build.golang.org/log/e11446723639eb0b41a7268c1f19c0de5f8da7cc

runtime: epollwait on fd 6 failed with 22
fatal error: runtime: netpoll failed

runtime stack:
runtime.throw(0xea8e5d96, 0x17)
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/runtime/panic.go:617 +0x69
runtime.netpoll(0xeab24a00, 0x1a5c7cf0)
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/runtime/netpoll_epoll.go:75 +0x1f9
runtime.findrunnable(0xb9c23300, 0x0)
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/runtime/proc.go:2226 +0x8bb
runtime.schedule()
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/runtime/proc.go:2525 +0x207
runtime.park_m(0xb9cc0000)
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/runtime/proc.go:2605 +0x84
runtime.mcall(0xea78ab32)
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/runtime/asm_386.s:345 +0x49

goroutine 1 [chan receive]:
testing.tRunner.func1(0xb9c98000)
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/testing/testing.go:838 +0x1a2
testing.tRunner(0xb9c98000, 0xb9c3ff1c)
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/testing/testing.go:866 +0xac
testing.runTests(0xb9c0e0a0, 0xeab10180, 0x23, 0x23, 0x0)
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/testing/testing.go:1152 +0x286
testing.(*M).Run(0xb9c52380, 0x0)
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/testing/testing.go:1069 +0x141
main.main()
	_testmain.go:112 +0x14d

goroutine 41 [runnable]:
testing.tRunner.func1(0xb9cd2dc0)
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/testing/testing.go:857 +0x1fa
testing.tRunner(0xb9cd2dc0, 0xea9c93d8)
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/testing/testing.go:866 +0xac
created by testing.(*T).Run
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/testing/testing.go:913 +0x2d1

goroutine 58 [runnable]:
testing.runTests.func1.1(0xb9c98000)
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/testing/testing.go:1159
created by testing.runTests.func1
	/private/var/folders/f6/d2bhfqss2716nxm8gkv1fmb80000gn/T/workdir-host-darwin-amd64-eliasnaur-android/go/src/testing/testing.go:1159 +0x84
exitcode=2go_android_exec: adb -e shell rm -rf /data/local/tmp/unix.test-17686
FAIL	cmd/vendor/golang.org/x/sys/unix	0.532s
@odeke-em
Copy link
Member

Thank you for this report @eliasnaur!

Kindly paging @ianlancetaylor.

@ianlancetaylor ianlancetaylor added this to the Go1.13 milestone Dec 27, 2018
@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 27, 2018
@ianlancetaylor
Copy link
Contributor

That is the only instance I can find in the builder logs. It should be impossible.

@eliasnaur
Copy link
Contributor Author

Another, again on android/386:

https://build.golang.org/log/11cd9309ba16cce696dc4763a7be1a433a590a7d

@eliasnaur
Copy link
Contributor Author

eliasnaur commented Feb 10, 2019 •
edited

Another, this time on android/arm64:

https://build.golang.org/log/ec6550708e7d674f80b19c3a987d0be24eb7dbe4

@eliasnaur
Copy link
Contributor Author

FWIW, I can reproduce this crash locally by running the cmd/vendor/golang.org/x/sys/unix tests in a loop. I don't know how to debug it though.

@eliasnaur
Copy link
Contributor Author

It could be a coincidence, but I once got the crash just as I was ctrl-C'ing the test loop that had run for minutes. If so, there might be a racy interaction between epoll_pwait and signals or during process exit.

@eliasnaur
Copy link
Contributor Author

eliasnaur commented Feb 23, 2019 •
edited

Edit: deleted confused analysis. runtime.netpollclose only deregisters a filedescriptor, it doesn't close the poll fd itself.

@gopherbot
Copy link

Change https://golang.org/cl/163638 mentions this issue: unix: don't overwrite unrelated file descriptors in TestDup

gopherbot pushed a commit to golang/sys that referenced this issue Feb 25, 2019
@eliasnaur
unix: don't overwrite unrelated file descriptors in TestDup
cc5685c 
TestDup used a file descriptor without ensuring it was free,
leading to rare crashes in the runtime netpoller when the victim fd
was the polling descriptor.

Updates golang/go#29423

Change-Id: Idc8b6b47f7e966e045f57f2028e7b6b79e0fb3f3
Reviewed-on: https://go-review.googlesource.com/c/163638
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
@gopherbot
Copy link

Change https://golang.org/cl/163623 mentions this issue: cmd/vendor/golang.org/x/sys: re-vendor

@golang golang locked and limited conversation to collaborators Feb 26, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.13
Development

No branches or pull requests
4 participants
@eliasnaur @ianlancetaylor @odeke-em @gopherbot