Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net: UDPConn.ReadFromUDP blocked in syscall when underlying socket option is set #28165

Closed
hujun-open opened this issue Oct 11, 2018 · 3 comments
Closed

net: UDPConn.ReadFromUDP blocked in syscall when underlying socket option is set #28165

hujun-open opened this issue Oct 11, 2018 · 3 comments
Labels
FrozenDueToAge

Comments

@hujun-open
Copy link

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.11.1

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

linux, centos 7, amd4

What did you do?

I have a program need to create large number of listening UDPConn, basically around 30k, each UDPConn will run in its own go routine waiting incoming packets, so that's 30k go routines, however I hit the wall of 10000 system threads limitation; after search around, I found it is because I set socket option to use ESPinUDP encap: "syscall.SetsockoptInt(conn_fd, syscall.IPPROTO_UDP, 100, 2)" , with the socket option set, each go routine is blocked in syscall, thus consuming a lots of system threads:

101 @ 0x47b915 0x47b163 0x47841f 0x49039f 0x4f735b 0x516dba 0x514bad 0x6fd1bf 0x45c741
#	0x47b914	syscall.Syscall6+0x4			/usr/local/go/src/syscall/asm_linux_amd64.s:44
#	0x47b162	syscall.recvfrom+0xa2			/usr/local/go/src/syscall/zsyscall_linux_amd64.go:1648
#	0x47841e	syscall.Recvfrom+0xae			/usr/local/go/src/syscall/syscall_unix.go:262
#	0x49039e	internal/poll.(*FD).ReadFrom+0xfe	/usr/local/go/src/internal/poll/fd_unix.go:215
#	0x4f735a	net.(*netFD).readFrom+0x5a		/usr/local/go/src/net/fd_unix.go:208
#	0x516db9	net.(*UDPConn).readFrom+0x69		/usr/local/go/src/net/udpsock_posix.go:47
#	0x514bac	net.(*UDPConn).ReadFromUDP+0x6c		/usr/local/go/src/net/udpsock.go:109
#	0x6fd1be	main.listensocket+0x3ae			/root/gowork/src/test/test.go:55

if I don't set the socket option, then number of thread is far less, and go routine no longer use syscall:

101 @ 0x42f3cb 0x42a709 0x429db6 0x48f1aa 0x48f2bd 0x490408 0x4f735b 0x516dba 0x514bad 0x6fd0b7 0x45c741
#	0x429db5	internal/poll.runtime_pollWait+0x65	/usr/local/go/src/runtime/netpoll.go:173
#	0x48f1a9	internal/poll.(*pollDesc).wait+0x99	/usr/local/go/src/internal/poll/fd_poll_runtime.go:85
#	0x48f2bc	internal/poll.(*pollDesc).waitRead+0x3c	/usr/local/go/src/internal/poll/fd_poll_runtime.go:90
#	0x490407	internal/poll.(*FD).ReadFrom+0x167	/usr/local/go/src/internal/poll/fd_unix.go:219
#	0x4f735a	net.(*netFD).readFrom+0x5a		/usr/local/go/src/net/fd_unix.go:208
#	0x516db9	net.(*UDPConn).readFrom+0x69		/usr/local/go/src/net/udpsock_posix.go:47
#	0x514bac	net.(*UDPConn).ReadFromUDP+0x6c		/usr/local/go/src/net/udpsock.go:109
#	0x6fd0b6	main.listensocket+0x2a6			/root/gowork/src/test/test.go:55

so is there way to use socket option without consume so many system threads?

What did you expect to see?

consume less system threads

What did you see instead?

hit 10000 system threads limitation
@ianlancetaylor
Copy link
Contributor

How are you getting the value conn_fd that you mention? My first guess would be that you are doing something that causes the file descriptor to go into blocking mode. If so, the fix will be to call the SyscallConn method and call the Control method of the returned syscall.RawConn value.

Closing on the assumption that that is what you are looking for. Please comment if you disagree. In general, please ask questions on a forum, not the issue tracker; see https://golang.org/wiki/Questions . Thanks.

@hujun-open
Copy link
Author

Thanks, what you suggested works.
I was using File().Fd() to get the fd, and using syscall.SetsockoptInt

btw: I knew this is not right place to ask questions, but I have posted this question on go-nuts a few days ago, did't get any reply ...

@mikioh mikioh changed the title UDPConn.ReadFromUDP blocked in syscall when underlying socket option is set net: UDPConn.ReadFromUDP blocked in syscall when underlying socket option is set Oct 12, 2018
@mikioh
Copy link
Contributor

mikioh commented Oct 12, 2018

File().Fd() to get the fd, and using syscall.SetsockoptInt

And forgot to call FilePacketConn for turning the socket descriptor into the runtime-integrated network poller again.

but I have posted this question on go-nuts ... did't get any reply

Probably the provided information is not sufficient, as @iant had to "guess" for your "question." Having a look at https://github.com/golang/go/wiki/HowToAsk and https://github.com/golang/go/wiki/How-to-ask-for-help would help you next time.

@golang golang locked and limited conversation to collaborators Oct 12, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@mikioh @ianlancetaylor @gopherbot @hujun-open