Comment 1:

I would like to know what call websocket is making, but in general
this is correct behavior: if you are trying to run 1000 system calls
simultaneously, you need 1000 threads to do that.  If your OS won't
let you have that, there is nothing Go can do about it.
Can you include some of the stack traces from the crash?

Status changed to Accepted.

Comment 2 by vladimir.webdev:

Thanks for looking into this.
Goroutines are stated to be lightweight analog of threads. My code is trying to create
1000 lightweight goroutines, but instead I am getting > 1000 OS threads (as I understand
2-3k threads). I can't understand how this could be treated as correct behavior...
Probably you are looking for this (but I also attached complete output):
goroutine 203 [1]:
runtime.gosched+0x56 /archive/vladimir/Dropbox/workspace/go/src/pkg/runtime/proc.c:769
    runtime.gosched()
runtime.exitsyscall+0x7d
/archive/vladimir/Dropbox/workspace/go/src/pkg/runtime/proc.c:871
    runtime.exitsyscall()
runtime.cgocall+0x10a
/archive/vladimir/Dropbox/workspace/go/src/pkg/runtime/cgocall.c:136
    runtime.cgocall(0x8105ce2, 0x3ebc8c, 0x3)
runtime.cmalloc+0x4b /archive/vladimir/Dropbox/workspace/go/src/pkg/runtime/cgocall.c:180
    runtime.cmalloc(0xa, 0x804f084)
net._Cfunc_CString+0x2a
/archive/vladimir/Dropbox/workspace/go/src/pkg/net/_obj/_cgo_defun.c:31
    net._Cfunc_CString(0x81764b1, 0x9, 0x86643b0, 0x0)
net.cgoLookupIPCNAME+0x94
/archive/vladimir/Dropbox/workspace/go/src/pkg/net/_obj/cgo_linux.cgo1.go:76
    net.cgoLookupIPCNAME(0x81764b1, 0x9, 0x0, 0x0, 0x0, ...)
net.cgoLookupIP+0x3d
/archive/vladimir/Dropbox/workspace/go/src/pkg/net/_obj/cgo_linux.cgo1.go:120
    net.cgoLookupIP(0x81764b1, 0x9, 0x0, 0x0, 0x0, ...)
net.cgoLookupHost+0x40
/archive/vladimir/Dropbox/workspace/go/src/pkg/net/_obj/cgo_linux.cgo1.go:16
    net.cgoLookupHost(0x81764b1, 0x9, 0x0, 0x0, 0x0, ...)
net.LookupHost+0x3d /archive/vladimir/Dropbox/workspace/go/src/pkg/net/lookup_unix.go:14
    net.LookupHost(0x81764b1, 0x9, 0x0, 0x0, 0x0, ...)
net.hostPortToIP+0x14f /archive/vladimir/Dropbox/workspace/go/src/pkg/net/ipsock.go:128
    net.hostPortToIP(0x8162b9c, 0x3, 0x81764b1, 0xe, 0x0, ...)
net.ResolveTCPAddr+0x37 /archive/vladimir/Dropbox/workspace/go/src/pkg/net/tcpsock.go:35
    net.ResolveTCPAddr(0x8162b9c, 0x3, 0x81764b1, 0xe, 0x0, ...)
net.resolveNetAddr+0x32e /archive/vladimir/Dropbox/workspace/go/src/pkg/net/dial.go:15
    net.resolveNetAddr(0x81650cc, 0x4, 0x8162b9c, 0x3, 0x81764b1, ...)
net.Dial+0x54 /archive/vladimir/Dropbox/workspace/go/src/pkg/net/dial.go:48
    net.Dial(0x8162b9c, 0x3, 0x81764b1, 0xe, 0x0, ...)
websocket.Dial+0x130
/archive/vladimir/Dropbox/workspace/go/src/pkg/websocket/client.go:113
    websocket.Dial(0x81764ac, 0x14, 0x8161924, 0x0, 0x8171aa0, ...)
main.openConn+0x48 /home/vladimir/goplayground/testclient/test.go:9
    main.openConn(0x9774d8d0, 0x0)
runtime.goexit /archive/vladimir/Dropbox/workspace/go/src/pkg/runtime/proc.c:246
    runtime.goexit()
----- goroutine created by -----
main.main+0x5e /home/vladimir/goplayground/testclient/test.go:22

Attachments:

1. o.txt (874836 bytes)

Comment 3:

Goroutines expand to full threads if they get blocked in a system call
(otherwise the program as a whole would stop running),
and for reasons I do not yet understand, this code is getting blocked
in 1000 system calls simultaneously.
Russ

Comment 4 by vladimir.webdev:

Ah, it seems now I finally got it. But will runtime try to reduce the number of OS
threads when it gets the control from system call back? Because it is hard to imagine
the situation when so many threads are useful...

Comment 5:

It may be hard to imagine the situation, but the situation just happened!
If it happened once it is likely to happen again.
Russ

Comment 6:

The best solution seems to be to limit the maximum number of concurrent requests issued
by the "net" package (patch in respect to Go release.r60.3):
diff -r c1702f36df03 src/pkg/net/ipsock.go
--- a/src/pkg/net/ipsock.go     Tue Oct 18 10:55:12 2011 +1100
+++ b/src/pkg/net/ipsock.go     Mon Nov 28 18:09:33 2011 +0100
@@ -101,8 +101,13 @@
        return host + ":" + port
 }
 
+var concurrencyControl = make(chan byte, 10)
+
 // Convert "host:port" into IP address and port.
 func hostPortToIP(net, hostport string) (ip IP, iport int, err os.Error) {
+       concurrencyControl <- 0
+       defer func(){<-concurrencyControl}()
+
        var (
                addr IP
                p, i int

Comment 7:

Labels changed: added priority-later, removed priority-medium.

Comment 9:

Is this issue fixable in any way? is it about revisiting the way cgo is used to resolve
hostnames, or reducing the amount of OS threads used by the runtime?

Comment 11:

[The time for maybe has passed.]

Comment 12:

This is essentially fixed by revision e255db359405 (coalesce duplicate lookups) and
revision 565e2d46bb68 (limit concurrent cgo DNS lookups).

Status changed to Fixed.