Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: SO_REUSEADDDR vs UDP vs timeouts #1692

Closed
gopherbot opened this issue Apr 13, 2011 · 13 comments
Closed

net: SO_REUSEADDDR vs UDP vs timeouts #1692

gopherbot opened this issue Apr 13, 2011 · 13 comments
Milestone

Comments

@gopherbot
Copy link

by jimenezrick:

Looking into src/pkg/net/sock.go (func socket) I see that SO_REUSEADDR 
is set unconditionally to all types of sockets.

AFAIK SO_REUSEADDR makes sense with TCP which it allows to reuse a 
recently used port but the bind fails when other sockets is listening. 
With UDP sockets it is not so clear to me if SO_REUSEADDR has any 
advantage because it allows you to have two different process with the 
same UDP port open, which could not what you want.

The important part of the issue is that timeouts with ReadFrom() and WriteTo() 
don't occur when SO_REUSEADDR is disabled. The ReadFrom() and WriteTo() calls
block forever even when I previously used SetTimeout(). 


Using Ubuntu 10.04 (amd64) with 6g
revision c5c62aeb6267 release.r56/weekly.2011-03-07.1/release.


Simple test program to see the issue, comment the call to `setNoReuseAddress()'
to let the timeout occur:

package main 
import ( 
        "fmt" 
        "net" 
        "syscall" 
) 
func setNoReuseAddress(conn net.PacketConn) { 
        file, _ := conn.(*net.UDPConn).File() 
        fd := file.Fd() 
        syscall.SetsockoptInt(fd, syscall.SOL_SOCKET, syscall.SO_REUSEADDR, 0); 
} 

func main() { 
        buff := make([]byte, 1024) 
        conn, _ := net.ListenPacket("udp", "localhost:10000") 
        setNoReuseAddress(conn) 
        conn.SetTimeout(500 * 1000 * 1000) 
        _, _, err := conn.ReadFrom(buff) 
        if netErr, ok := err.(net.Error); ok && netErr.Timeout() { 
                fmt.Println("Timeout reached") 
        } 
}
@rsc
Copy link
Contributor

rsc commented May 3, 2011

Comment 1:

How could SO_REUSEADDR affect timeouts?

Owner changed to @rsc.

Status changed to Accepted.

@gopherbot
Copy link
Author

Comment 2 by jimenezrick:

I have used `strace' with both cases, when SO_REUSEADDR is enable by
default and when SO_REUSEADDR is disabled later by me inside my
program. Both traces are attached to this message. Next I describe
what could be an explanation of the problem.
Below there is a fragment of the output of `strace' when I *disable*
SO_REUSEADDR manually:
...
socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3   -- (*1)
fcntl(3, F_SETFD, FD_CLOEXEC)     = 0
setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
setsockopt(3, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
setsockopt(3, SOL_IPV6, IPV6_V6ONLY, [0], 4) = 0
bind(3, {sa_family=AF_INET6, sin6_port=htons(10000), ...) = 0
...
fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0   -- (*2)
dup(3)                            = 7            -- (*3)
fcntl(7, F_GETFL)                 = 0x802 (flags O_RDWR|O_NONBLOCK)
fcntl(7, F_SETFL, O_RDWR)         = 0     -- (*4)
setsockopt(7, SOL_SOCKET, SO_REUSEADDR, [0], 4) = 0   -- (*5)
gettimeofday({1304448918, 597094}, NULL) = 0
recvfrom(3,  <unfinished ...>    (*1)      -- (*6)
...
(*1) Ok, this is my socket and SO_REUSEADDR is set on it atomatically.
(*2) As I have seen O_NONBLOCK mode is used to handle the timeouts.
(*3) IMPORTANT: when SO_REUSEADDR is NOT disabled (manually), the
`dup()' call *does not* happend neither the next fcntl() call.
(*4) From the dup'ed file descriptor set the file descriptor status to
O_RDWR as before BUT WITHOUT O_NONBLOCK.
(*5) Here is when is disable SO_REUSEADDR manually with:
syscall.SetsockoptInt().
(*6) Here the recvfrom() stalls because O_NONBLOCK no longer is used.
Then I hit Control-C to interrupt de program.
So, that is what I have seen in case it is useful to analyze the
issue. In this occasion I used 8g (x86), as it presents the same
issue.
On Tue, May 3, 2011 at 12:38 PM,  <go@googlecode.com> wrote:

@gopherbot
Copy link
Author

Comment 3 by jimenezrick:

Sorry, the attachments were missing.

Attachments:

  1. timeout-disabled.trace (8979 bytes)
  2. timeout-enabled.trace (8315 bytes)

@rsc
Copy link
Contributor

rsc commented Oct 6, 2011

Comment 4:

Mikio or anyone else, any ideas?

Status changed to HelpWanted.

@rsc
Copy link
Contributor

rsc commented Oct 6, 2011

Comment 5:

// Called directly by OS as signal handler.
#pragma textflag 7
before the definition of runtime.sigignore in both freebsd/386/signal.c and
freebsd/amd64/signal.c.
This is a difference compared to the Darwin port because on Darwin the function gets
called via sigtramp.  On FreeBSD it is being given to the signal handler directly, so it
cannot be a typical Go stack-checking function.

@mikioh
Copy link
Contributor

mikioh commented Oct 7, 2011

Comment 6:

I guess we are talking of two distint issues here.
``Wild use of SO_REUSEADDR'' is really good for us?' and 
the snippet program described in comment #0 doesn't 
work well.
The former is a bit controversial thing, actually I have no 
concrete opinion on that. But I feel it's okay because apps 
work well even if we take some risks and advances on socket 
descriptor table search by using SO_REUSE(ADDR|PORT) options.
As you know the socket consists of 5-tuple, a five is enough 
to avoid end-to-end identification confusion under current 
IP networking environment.
The latter seems like a bug that kinda fate-sharing on socket 
descriptor. I just uploaded a patch to fix this, confirmed on 
darwin, freebsd and linux with the program in comment #0.
Please take a look at <http://golang.org/cl/5239041/>;

@rsc
Copy link
Contributor

rsc commented Dec 9, 2011

Comment 7:

Labels changed: added priority-later.

@rsc
Copy link
Contributor

rsc commented Dec 12, 2011

Comment 8:

Labels changed: added priority-go1.

@dhobsd
Copy link
Contributor

dhobsd commented Jan 6, 2012

Comment 9:

SO_REUSEADDR does have a place in UDP, for multicast messages. Multicast messages can be
delivered to multiple clients listening for packets on that UDP address/port
combination. With SO_REUSEADDR set, the OS will deliver to all of them. With
SO_REUSEADDR set on unicast UDP sockets, the OS will deliver to only one of the
listening sockets (and in my experience, it will deliver to the last socket bound to
that address/port, though I don't know how this is across OSes or if it has changed).
It is valid to want to turn off SO_REUSEADDR in both TCP and UDP, and there's no reason
we shouldn't export *netFD.setReuseAddr via TCPConn and UDPConn. Patch incoming.

@robpike
Copy link
Contributor

robpike commented Jan 13, 2012

Comment 10:

Owner changed to builder@golang.org.

@mikioh
Copy link
Contributor

mikioh commented Jan 13, 2012

Comment 11:

Owner changed to @mikioh.

@mikioh
Copy link
Contributor

mikioh commented Jan 13, 2012

Comment 12:

Status changed to Accepted.

@mikioh
Copy link
Contributor

mikioh commented Jan 15, 2012

Comment 13:

This issue was closed by revision 7419921.

Status changed to Fixed.

@rsc rsc added this to the Go1 milestone Apr 10, 2015
@rsc rsc removed the priority-go1 label Apr 10, 2015
@golang golang locked and limited conversation to collaborators Jun 24, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants