net: listen tcp: errno -9 #23446

alllexx88 · 2018-01-15T14:07:42Z

What version of Go are you using (`go version`)?

go1.10beta2

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

GOARCH="mipsle"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOHOSTARCH="mipsle"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/root/go"
GORACE=""
GOROOT="/opt/lib/golang"
GOTMPDIR=""
GOTOOLDIR="/opt/lib/golang/pkg/tool/linux_mipsle"
GCCGO="/opt/bin/gccgo"
GOMIPS="softfloat"
CC="/opt/bin/gcc"
CXX="/opt/bin/g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -mabi=32 -march=mips32 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build074360272=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Ran this code

package main

import (
	"net"
	"fmt"
)

func main() {
	_, err := net.Listen("tcp", ":8080")
	if err != nil {
		fmt.Println(err)
	}
}

What did you expect to see?

No output

What did you see instead?

listen tcp :8080: errno -9

I'm trying to get Go programs to run on a softfloat mips32r2 router. The first issue I encountered was pipe2 syscall not being supported by the kernel, so I wrote a patch to use pipe syscall plus fcntl instead (see mips_no_pipe2.patch). Now I'm facing this issue with net.Listen always failing. Any help or hints on how to debug this are appreciated.

Thanks,
Alex

The text was updated successfully, but these errors were encountered:

davecheney · 2018-01-15T19:48:25Z

Which kernel is your system running?

alllexx88 · 2018-01-15T20:06:42Z

@davecheney The router is running tomatousb firmware. The kernel is a custom 2.6.22.19 kernel:

[root@unknown root]$ uname -a
Linux unknown 2.6.22.19 #33 Sun Jun 26 17:47:11 CEST 2016 mips GNU/Linux

If I'm not mistaken, kernel sources can be found here, and config is here

davecheney · 2018-01-15T20:14:22Z

I don’t know the exact minimum kernel version MIPS needs but I suspect the problem you have is your kernel doesnt support accept4. Again the details are sketchy, but it’s the version of accept that supports passing the close on accept flag. Look in the net package for calls to syscall.Accept and then patch as appropriate. There may be some fallback code from the days we didn’t support rhel5, but still tried anyway.

…

On 16 Jan 2018, at 07:06, alllexx88 ***@***.***> wrote: @davecheney The router is running tomatousb firmware. The kernel is a custom 2.6.22.19 kernel: ***@***.*** root]$ uname -a Linux unknown 2.6.22.19 #33 Sun Jun 26 17:47:11 CEST 2016 mips GNU/Linux If I'm not mistaken, kernel sources can be found here, and config is here — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

alllexx88 · 2018-01-15T21:17:20Z

@davecheney Looks like my kernel indeed doesn't support accept4 syscall. I'll try to workaround it using accept and fcntl tomorrow. Thanks!

mvdan · 2018-01-15T21:20:34Z

I take it that there is nothing to fix in Go about this, then? Or should the minimum requirements for this case be better specified somewhere?

In any case, https://github.com/golang/go/wiki/MinimumRequirements does mention 2.6.23, and this post refers to 2.6.22.

davecheney · 2018-01-15T21:55:49Z

The minimum kernel version of non intel platforms is higher than 2.6.23. Maybe we should document the sys calls we need, accept4(1), etc, rather than the kernel version.

…

On 16 January 2018 at 08:20, Daniel Martí ***@***.***> wrote: I take it that there is nothing to fix in Go about this, then? Or should the minimum requirements for this case be better specified somewhere? In any case, https://github.com/golang/go/wiki/MinimumRequirements does mention 2.6.23, and this post refers to 2.6.22. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#23446 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA09GU6v6-3j5BlQqMvwz8MMRupDeks5tK8ElgaJpZM4Redmg> .

bradfitz · 2018-01-15T22:13:23Z

Maybe we should document the sys calls we need, accept4(1), etc, rather
than the kernel version.

Please no. That will help a very certain type of user, but not most users. Most users know their kernel version but don't know what system calls their kernel supports.

If anything, any system calls assumed to be present by Go should perhaps throw on ENOSYS with "your kernel is too old".

alllexx88 · 2018-01-16T10:16:31Z

@davecheney Looks like it's not accept4 at fault in my case. I wrote this patch(hack) to avoid using accept4 syscall, but net.Listen() gives me the same error:

--- a/src/syscall/zsyscall_linux_mips.go
+++ b/src/syscall/zsyscall_linux_mips.go
@@ -1416,11 +1416,17 @@ func accept(s int, rsa *RawSockaddrAny, addrlen *_Socklen) (fd int, err error) {
 // THIS FILE IS GENERATED BY THE COMMAND AT THE TOP; DO NOT EDIT
 
 func accept4(s int, rsa *RawSockaddrAny, addrlen *_Socklen, flags int) (fd int, err error) {
-	r0, _, e1 := Syscall6(SYS_ACCEPT4, uintptr(s), uintptr(unsafe.Pointer(rsa)), uintptr(unsafe.Pointer(addrlen)), uintptr(flags), 0, 0)
+	r0, _, e1 := Syscall(SYS_ACCEPT, uintptr(s), uintptr(unsafe.Pointer(rsa)), uintptr(unsafe.Pointer(addrlen)))
 	fd = int(r0)
 	if e1 != 0 {
 		err = errnoErr(e1)
+		return
+	}
+	flags2, err := fcntl(fd, F_GETFL, 0)
+	if err != nil {
+		return
 	}
+	_, err = fcntl(fd, F_SETFL, flags2 | flags)
 	return
 }
 
--- a/src/syscall/zsyscall_linux_mipsle.go
+++ b/src/syscall/zsyscall_linux_mipsle.go
@@ -1416,11 +1416,17 @@ func accept(s int, rsa *RawSockaddrAny, addrlen *_Socklen) (fd int, err error) {
 // THIS FILE IS GENERATED BY THE COMMAND AT THE TOP; DO NOT EDIT
 
 func accept4(s int, rsa *RawSockaddrAny, addrlen *_Socklen, flags int) (fd int, err error) {
-	r0, _, e1 := Syscall6(SYS_ACCEPT4, uintptr(s), uintptr(unsafe.Pointer(rsa)), uintptr(unsafe.Pointer(addrlen)), uintptr(flags), 0, 0)
+	r0, _, e1 := Syscall(SYS_ACCEPT, uintptr(s), uintptr(unsafe.Pointer(rsa)), uintptr(unsafe.Pointer(addrlen)))
 	fd = int(r0)
 	if e1 != 0 {
 		err = errnoErr(e1)
+		return
+	}
+	flags2, err := fcntl(fd, F_GETFL, 0)
+	if err != nil {
+		return
 	}
+	_, err = fcntl(fd, F_SETFL, flags2 | flags)
 	return
 }

I grepped for ENOSYS in strace output and here's what I got (not sure if it matters):

$ strace ./test 2>&1 | grep ENOSYS
futex(0x7fee8938, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, 0) = -1 ENOSYS (Function not implemented)

@mvdan

I take it that there is nothing to fix in Go about this, then?

Probably you are correct, though I still don't know why net.Listen fails in my case (not due to missing accept4, as it seems), but Go works fine on my other systems with newer kernels (arm, i686). Is it OK for this discussion-attempt to get Go to work on 2.6.22.19 kernel to stay here, or should I better post on some mailing list?

Thanks

ianlancetaylor · 2018-01-16T14:38:12Z

According to https://github.com/golang/go/wiki/MinimumRequirements for little-endian mips64 you need a kernel newer than 4.1. No specific requirements are listed for little-endian 32-bit mips, but it seems very likely that 2.6.22 is too old.

Errno 9 is EBADF, but errno -9 is a bug in how the port interacts with the kernel. I have no idea whether this is due to using an old kernel or due to some other problem.

crvv · 2018-01-23T09:30:54Z

The errno -9 is similar to #14297. It is 9, EBADF.
The error EBADF is returned by epollctl.

go/src/runtime/netpoll_epoll.go

Line 47 in 99e6e48

return -epollctl(epfd, _EPOLL_CTL_ADD, int32(fd), &ev)

I can reproduce the issue on MIPS Linux 2.6.18.
It works fine on MIPS Linux 2.6.32 and 4.4.89.

gopherbot · 2018-01-23T09:40:54Z

Change https://golang.org/cl/89235 mentions this issue: runtime: fix errno sign for epollctl on mips, mips64 and ppc64

alllexx88 · 2018-01-23T10:10:54Z

@crvv Thank you for clarifying things! 👍 So it's a EBADF after all. Now it would be great if there was a workaround for this error for older MIPS Linux.

ianlancetaylor · 2018-01-23T13:57:11Z

@alllexx88 There's really nothing we can do if the epoll_ctl system call doesn't work. On GNU/Linux systems the Go runtime requires epoll.

alllexx88 · 2018-01-23T14:18:53Z

@ianlancetaylor epoll_ctl should work in Linux 2.6.22.19, if I understand it correctly. It just returns EBADF on an fd that is expected to be valid. I just don't understand where (and why) it is happening, since I'm not really familiar with Go internals. Patching for missing pipe2 and accept4 syscalls was relatively simple, but I can't figure out where it goes wrong with epoll_ctl. On a side note, this is not a Go issue, modern GNU/Linux MIPS platforms that're powerful enough to run heavy Go applications usually have newer kernels (at least 3.2+ I think).

I'm still interested in finding a workaround, but this issue can be closed, I think. Thank you all.

The caller of epollctl expects it to return a negative errno value, but it returns a positive errno value on mips, mips64 and ppc64. The change fixes this. Updates #23446 Change-Id: Ie6372eca6c23de21964caaaa433c9a45ef93531e Reviewed-on: https://go-review.googlesource.com/89235 Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>

gopherbot · 2018-07-25T18:44:26Z

Change https://golang.org/cl/125895 mentions this issue: runtime: fix syscall error returns on mips/mips64/ppc64

dwimmer · 2018-07-25T18:46:16Z

This error was caused by epollcreate1() returning ENOSYS instead of -ENOSYS. This was interpreted as success and the value of ENOSYS was later passed as a file descriptor to epollctl() leading to the (valid) EBADF.

dwimmer · 2018-08-01T19:06:18Z

@daenney, I saw your post about net.Listen() on the RT-AC66U and I thought you might be interested to know that it should work fine now that this is fixed (should be in Go 1.11 or try it out now on master).

The linux syscall functions used in runtime are designed around the calling convention of returning errors as negative numbers. On some other systems (like mips and ppc) the actual syscalls signal errors in other ways. This means that the assembly implementations of the syscall functions on these platforms need to transform the return values in the error cases to match the expected negative errno values. This was addressed for certain syscalls in https://golang.org/cl/19455 and https://golang.org/cl/89235. This patch handles the rest of the syscall functions in sys_linux_*.s that return any value for mips/mips64/ppc64. Fixes #23446 Change-Id: I302100261231f76d5850ab2c2ea080170d7dba72 GitHub-Last-Rev: e358e2b GitHub-Pull-Request: #26606 Reviewed-on: https://go-review.googlesource.com/125895 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>

mvdan added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 15, 2018

mvdan changed the title ~~listen tcp: errno -9~~ net: listen tcp: errno -9 Jan 15, 2018

alllexx88 mentioned this issue Jan 23, 2018

Go packages (rclone and syncthing) on older MIPS kernel Entware/Entware-ng#860

Closed

alllexx88 closed this as completed Jan 23, 2018

dwimmer mentioned this issue Jul 25, 2018

runtime: fix syscall error returns on mips/mips64/ppc64 #26606

Closed

dwimmer mentioned this issue Aug 1, 2018

dnscrypt-proxy wont start [FATAL] listen udp 127.0.0.1:65053: errno -9 thuantran/dnscrypt-asuswrt-installer#26

Closed

golang locked as resolved and limited conversation to collaborators Aug 2, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net: listen tcp: errno -9 #23446

net: listen tcp: errno -9 #23446

alllexx88 commented Jan 15, 2018 •

edited

davecheney commented Jan 15, 2018

alllexx88 commented Jan 15, 2018

davecheney commented Jan 15, 2018 via email

alllexx88 commented Jan 15, 2018

mvdan commented Jan 15, 2018

davecheney commented Jan 15, 2018 via email

bradfitz commented Jan 15, 2018

alllexx88 commented Jan 16, 2018

ianlancetaylor commented Jan 16, 2018

crvv commented Jan 23, 2018

gopherbot commented Jan 23, 2018

alllexx88 commented Jan 23, 2018

ianlancetaylor commented Jan 23, 2018

alllexx88 commented Jan 23, 2018

gopherbot commented Jul 25, 2018

dwimmer commented Jul 25, 2018 •

edited

dwimmer commented Aug 1, 2018

net: listen tcp: errno -9 #23446

net: listen tcp: errno -9 #23446

Comments

alllexx88 commented Jan 15, 2018 • edited

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

davecheney commented Jan 15, 2018

alllexx88 commented Jan 15, 2018

davecheney commented Jan 15, 2018 via email

alllexx88 commented Jan 15, 2018

mvdan commented Jan 15, 2018

davecheney commented Jan 15, 2018 via email

bradfitz commented Jan 15, 2018

alllexx88 commented Jan 16, 2018

ianlancetaylor commented Jan 16, 2018

crvv commented Jan 23, 2018

gopherbot commented Jan 23, 2018

alllexx88 commented Jan 23, 2018

ianlancetaylor commented Jan 23, 2018

alllexx88 commented Jan 23, 2018

gopherbot commented Jul 25, 2018

dwimmer commented Jul 25, 2018 • edited

dwimmer commented Aug 1, 2018

alllexx88 commented Jan 15, 2018 •

edited

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

dwimmer commented Jul 25, 2018 •

edited