Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/net/{ipv4,ipv6}: MSG_DONTWAIT flag ignored #46891

Closed
matzf opened this issue Jun 23, 2021 · 3 comments
Closed

x/net/{ipv4,ipv6}: MSG_DONTWAIT flag ignored #46891

matzf opened this issue Jun 23, 2021 · 3 comments
Labels
FrozenDueToAge help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@matzf
Copy link

matzf commented Jun 23, 2021

What version of Go are you using (go version)?

$ go version
go version go1.16.4 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/matzf/.cache/go-build"
GOENV="/home/matzf/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/matzf/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/matzf/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.16"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.16/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.4"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/gofoo/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2959451986=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Setting the flag syscall.MSG_DONTWAIT when calling x/net/ipv4.(*PacketConn).WriteBatch.
Example program:

go.mod
module gofoo

go 1.16

require golang.org/x/net v0.0.0-20210610132358-84b48f89b13b

package main

import (
	"errors"
	"fmt"
	"log"
	"net"
	"syscall"

	"golang.org/x/net/ipv4"
)

func main() {
	c, err := net.ListenPacket("udp4", "0.0.0.0:12345")
	if err != nil {
		log.Fatal(err)
	}
	defer c.Close()
	cc := ipv4.NewPacketConn(c)

	remote, err := net.ResolveUDPAddr("udp4", "1.1.1.1:12345") // sorry, cloudflair
	if err != nil {
		log.Fatal(err)
	}

	b := make([]byte, 1300)
	msg := ipv4.Message{}
	msg.Buffers = make([][]byte, 1)
	msg.Buffers[0] = b
	msg.N = len(b)
	msg.Addr = remote
	msgs := []ipv4.Message{msg}

	for i := 0; i < 1000; i++ {
		_, err := cc.WriteBatch(msgs, syscall.MSG_DONTWAIT)
		var errno syscall.Errno
		if err != nil && errors.As(err, &errno) && errno.Timeout() {
			fmt.Println(err)
		} else if err != nil {
			log.Fatal(err)
		}
		fmt.Println(i)
	}
}

What did you expect to see?

If the MSG_DONTWAIT flag was honored, the program would print "resource temporary unavailable" as soon as the kernels UDP send buffer fills up.

What did you see instead?

The MSG_DONTWAIT flag is ignored; the program blocks on each WriteBatch call until there is room in the send buffer such that the send operation can be completed.
Note: depending on local network speeds and settings, it might be necessary to increase the number of packets sent in the program above.

Additional Notes

The MSG_DONTWAIT flag is also ignored for ReadBatch.

The effect of passing the MSG_DONTWAIT flag is that the system call should return EAGAIN /EWOULDBLOCK when the call would block. This is analogous to the O_NONBLOCK setting on the socket itself -- as far as I understand, this is always set for sockets in go, as it is part of the network polling. Therefore, the only way to disambiguate this would be to explicitly handle this MSG_DONTWAIT flag in the implementation of the x/net library code. In particular, my suggestion would be:

diff --git a/internal/socket/rawconn_mmsg.go b/internal/socket/rawconn_mmsg.go
index d80a15c..13ff208 100644
--- a/internal/socket/rawconn_mmsg.go
+++ b/internal/socket/rawconn_mmsg.go
@@ -60,7 +60,7 @@ func (c *Conn) sendMsgs(ms []Message, flags int) (int, error) {
        var n int
        fn := func(s uintptr) bool {
                n, operr = sendmmsg(s, hs, flags)
-               if operr == syscall.EAGAIN {
+               if operr == syscall.EAGAIN && (flags&syscall.MSG_DONTWAIT == 0) {
                        return false
                }
                return true
@gopherbot gopherbot added this to the Unreleased milestone Jun 23, 2021
@seankhliao seankhliao added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 25, 2021
@seankhliao
Copy link
Member

cc @ianlancetaylor

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/333469 mentions this issue: internal/socket: support MSG_DONTWAIT

@matzf
Copy link
Author

matzf commented Jul 27, 2021

Friendly ping @ianlancetaylor . Please let me know if I can help in any way with getting feedback on this issue or the proposed CL.

@golang golang locked and limited conversation to collaborators Aug 5, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants