net: TestLookup* failures with "i/o timeout" #50191

bcmills · 2021-12-15T16:38:11Z

greplogs --dashboard -md -l -e 'FAIL: TestLookup.*\n .*i/o timeout.*(?:.*\n)*FAIL\s+net' --since=2021-01-01

2021-12-15T00:33:55-9d0ca26/netbsd-arm64-bsiegert

--- FAIL: TestLookupHostCancel (14.66s)
    lookup_test.go:962: lookup www.google.com on 192.168.87.1:53: read udp 192.168.87.23:52715->192.168.87.1:53: i/o timeout
FAIL

2021-12-14T21:23:43-fc8ae98/freebsd-arm-paulzhol

--- FAIL: TestLookupCNAME (76.51s)
    lookup_test.go:370: backoff 1s after failure lookup www.iana.org on 192.168.1.253:53: read udp 192.168.5.15:27642->192.168.1.253:53: i/o timeout
    lookup_test.go:370: backoff 5s after failure lookup www.iana.org on 192.168.1.253:53: read udp 192.168.5.15:11002->192.168.1.253:53: i/o timeout
    lookup_test.go:370: backoff 30s after failure lookup www.iana.org on 192.168.1.253:53: read udp 192.168.5.15:49406->192.168.1.253:53: i/o timeout
    lookup_test.go:376: lookup www.iana.org on 192.168.1.253:53: read udp 192.168.5.15:58118->192.168.1.253:53: i/o timeout
--- FAIL: TestLookupDotsWithRemoteSource (31.02s)
    lookup_test.go:660: LookupAddr(8.8.8.8): lookup 8.8.8.8.in-addr.arpa. on 192.168.1.253:53: read udp 192.168.5.15:28987->192.168.1.253:53: i/o timeout (mode=go)
    lookup_test.go:673: LookupCNAME(www.mit.edu, mode=go): lookup www.mit.edu on 192.168.1.253:53: read udp 192.168.5.15:38636->192.168.1.253:53: i/o timeout
    lookup_test.go:660: LookupAddr(8.8.8.8): lookup 8.8.8.8.in-addr.arpa. on 192.168.1.253:53: read udp 192.168.5.15:17754->192.168.1.253:53: i/o timeout (mode=cgo)
FAIL
FAIL	net	172.152s

2021-11-20T01:00:16-be18cd5/darwin-amd64-nocgo

--- FAIL: TestLookupGoogleHost (20.04s)
    lookup_test.go:403: lookup google.com on 8.8.8.8:53: read udp 172.17.20.32:54917->8.8.8.8:53: i/o timeout
--- FAIL: TestLookupGoogleIP (20.03s)
    lookup_test.go:455: lookup google.com. on 8.8.8.8:53: read udp 172.17.20.32:58664->8.8.8.8:53: i/o timeout
--- FAIL: TestLookupHostCancel (21.01s)
    lookup_test.go:954: lookup www.google.com on 8.8.8.8:53: read udp 172.17.20.32:64884->8.8.8.8:53: i/o timeout

2021-11-17T17:04:16-54b9cb8/solaris-amd64-oraclerel

--- FAIL: TestLookupDotsWithRemoteSource (25.10s)
    lookup_test.go:707: LookupSRV(xmpp-server, tcp, google.com): lookup _xmpp-server._tcp.google.com on 129.70.161.2:53: read udp 129.70.161.63:58549->129.70.161.2:53: i/o timeout (mode=cgo)
FAIL
FAIL	net	50.714s

2021-08-23T21:22:58-8157960/openbsd-arm64-jsing

--- FAIL: TestLookupDotsWithRemoteSource (12.00s)
    lookup_test.go:674: LookupCNAME(www.mit.edu, mode=go): lookup www.mit.edu on 10.2.2.254:53: read udp 10.2.2.13:28276->10.2.2.254:53: i/o timeout
FAIL
FAIL	net	69.817s

2021-07-29T15:30:38-70fd4e4/linux-riscv64-jsing

--- FAIL: TestLookupDotsWithRemoteSource (21.38s)
    lookup_test.go:661: LookupAddr(8.8.8.8): lookup 8.8.8.8.in-addr.arpa. on 127.0.0.53:53: read udp 127.0.0.1:44851->127.0.0.53:53: i/o timeout (mode=go)
    lookup_test.go:661: LookupAddr(8.8.8.8): lookup 8.8.8.8: Temporary failure in name resolution (mode=cgo)
FAIL
FAIL	net	82.313s

2021-07-07T20:33:11-186a3bb/linux-s390x-ibm

--- FAIL: TestLookupGoogleHost (20.02s)
    lookup_test.go:404: lookup google.com. on 9.9.9.9:53: read udp 148.100.113.12:49071->9.9.9.9:53: i/o timeout
--- FAIL: TestLookupGoogleIP (20.02s)
    lookup_test.go:456: lookup google.com. on 9.9.9.9:53: read udp 148.100.113.12:36505->9.9.9.9:53: i/o timeout
panic: test timed out after 3m0s

2021-06-28T21:17:28-5385e23/linux-mips64-rtrk

--- FAIL: TestLookupGoogleHost (40.08s)
    lookup_test.go:404: lookup google.com on 10.11.104.32:53: read udp 10.81.110.62:47707->10.11.104.32:53: i/o timeout
--- FAIL: TestLookupGoogleIP (40.08s)
    lookup_test.go:456: lookup google.com on 10.11.104.32:53: read udp 10.81.110.62:52046->10.11.104.32:53: i/o timeout
FAIL
FAIL	net	320.061s

2021-06-22T14:07:22-666315b/openbsd-arm64-jsing

--- FAIL: TestLookupDotsWithRemoteSource (12.31s)
    lookup_test.go:661: LookupAddr(8.8.8.8): lookup 8.8.8.8.in-addr.arpa. on 10.2.2.254:53: read udp 10.2.2.13:31775->10.2.2.254:53: i/o timeout (mode=go)
FAIL
FAIL	net	52.869s

2021-05-03T16:31:10-472f519/linux-s390x-ibm
2021-05-03T16:25:05-d75fbac/linux-s390x-ibm
2021-05-03T16:23:09-7b768d4/linux-s390x-ibm
2021-04-10T01:46:41-52bf14e/linux-amd64-wsl

--- FAIL: TestLookupGoogleHost (10.01s)
    lookup_test.go:404: lookup google.com on 172.24.112.1:53: read udp 172.24.123.145:35967->172.24.112.1:53: i/o timeout
--- FAIL: TestLookupGoogleIP (10.01s)
    lookup_test.go:456: lookup google.com on 172.24.112.1:53: read udp 172.24.123.145:50137->172.24.112.1:53: i/o timeout
FAIL
FAIL	net	50.567s

2021-03-30T23:56:09-606e0ab/linux-s390x-ibm

--- FAIL: TestLookupDotsWithRemoteSource (107.85s)
    lookup_test.go:661: LookupAddr(8.8.8.8): lookup 8.8.8.8.in-addr.arpa. on 9.9.9.9:53: read udp 148.100.113.12:43600->9.9.9.9:53: i/o timeout (mode=go)
    lookup_test.go:682: LookupMX(google.com): lookup google.com on 9.9.9.9:53: no such host (mode=go)
    lookup_test.go:695: LookupNS(google.com): lookup google.com on 9.9.9.9:53: read udp 148.100.113.12:36256->9.9.9.9:53: i/o timeout (mode=go)
    lookup_test.go:708: LookupSRV(xmpp-server, tcp, google.com): lookup _xmpp-server._tcp.google.com on 9.9.9.9:53: no such host (mode=go)
    lookup_test.go:661: LookupAddr(8.8.8.8): lookup 8.8.8.8.in-addr.arpa. on 9.9.9.9:53: read udp 148.100.113.12:59168->9.9.9.9:53: i/o timeout (mode=cgo)
panic: test timed out after 3m0s

2021-03-30T21:20:51-e0ce0af/linux-s390x-ibm
2021-03-25T04:06:37-c69515c/darwin-amd64-nocgo

--- FAIL: TestLookupHostCancel (15.84s)
    lookup_test.go:955: lookup www.google.com on 8.8.8.8:53: read udp 10.50.0.227:60709->8.8.8.8:53: i/o timeout
FAIL
FAIL	net	36.017s

2021-01-26T00:02:15-ce8b318/netbsd-arm64-bsiegert

The text was updated successfully, but these errors were encountered:

bcmills · 2021-12-15T16:39:07Z

Marking as release-blocker for Go 1.18 via #11811: these failures occur frequently in the builders, and most of them can be addressed by suitable application of testenv.SkipFlaky.

gopherbot · 2021-12-15T16:47:56Z

Change https://golang.org/cl/372216 mentions this issue: net: skip Lookup tests of external hosts on builders with flaky networks

gopherbot · 2021-12-15T16:48:11Z

Change https://golang.org/cl/372294 mentions this issue: dashboard: mark builders with unreliable DNS as flaky

For #50191 Change-Id: Ic1059127e756d69ea9b75cf5805669ec43fedd5c Reviewed-on: https://go-review.googlesource.com/c/go/+/372216 Trust: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>

bcmills · 2022-01-10T17:15:36Z

This release-blocking issue is awaiting review of CL 372294 and perhaps a corresponding coordinator redeploy (CC @golang/release).

These are the builders identified in the logs in golang/go#50191, and associated builders with the same owners (which I assume are likely to employ the same DNS servers). For golang/go#50191 Change-Id: I5906782c6a63a001b0ba1194cffad08de2d8ce6a Reviewed-on: https://go-review.googlesource.com/c/build/+/372294 Trust: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

toothrot · 2022-01-19T17:20:06Z

The coordinator has been deployed. Closing (until proven otherwise!)

bcmills · 2022-05-31T15:02:41Z

Looks like we need to mark two more builders with FlakyNet (CC @bsiegert). I'll send a CL.

greplogs -l -e 'FAIL: TestLookup.*\n .*i/o timeout.*(?:.*\n)*FAIL\s+net' --since=2022-01-20
2022-05-30T21:23:29-af88fb6/netbsd-arm64-bsiegert
2022-05-28T20:07:28-52f68ef/netbsd-arm64-bsiegert
2022-05-27T21:36:06-70d499c/netbsd-arm64-bsiegert
2022-05-25T16:16:01-4e4db1e/netbsd-arm64-bsiegert
2022-05-20T19:01:40-69b412b/netbsd-arm-bsiegert
2022-04-29T01:00:55-e7b0559/netbsd-arm64-bsiegert
2022-04-06T17:10:45-9535031/netbsd-arm-bsiegert

gopherbot · 2022-05-31T15:05:12Z

Change https://go.dev/cl/409474 mentions this issue: dashboard: mark netbsd-arm-bsiegert and netbsd-arm64-bsiegert with FlakyNet

…akyNet These builders have both exhibited multiple 'i/o timeout' errors contacting www.google.com in net.TestLookupHostCancel. Updates golang/go#50191. Change-Id: I3d5ef2940f8c6df3487743a26f64d1e38fdcb231 Reviewed-on: https://go-review.googlesource.com/c/build/+/409474 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com>

bcmills added NeedsFix The path to resolution is known, but the work has not been done. release-blocker labels Dec 15, 2021

bcmills added this to the Go1.18 milestone Dec 15, 2021

bcmills added the Builders x/build issues (builders, bots, dashboards) label Dec 16, 2021

bcmills mentioned this issue Jan 10, 2022

net: TestLookupHostCancel failure with "too many open files" on netbsd-arm-bsiegert builder #50537

Closed

toothrot closed this as completed Jan 19, 2022

bcmills mentioned this issue May 31, 2022

build dashboard triage log #52653

Closed

cherrymui mentioned this issue Jan 5, 2023

net: TestLookupHostCancel failures with i/o timeout or temporary error #57617

Open

golang locked and limited conversation to collaborators May 31, 2023

gopherbot added the FrozenDueToAge label May 31, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net: TestLookup* failures with "i/o timeout" #50191

net: TestLookup* failures with "i/o timeout" #50191

bcmills commented Dec 15, 2021

bcmills commented Dec 15, 2021

gopherbot commented Dec 15, 2021

gopherbot commented Dec 15, 2021

bcmills commented Jan 10, 2022

toothrot commented Jan 19, 2022

bcmills commented May 31, 2022

gopherbot commented May 31, 2022

net: TestLookup* failures with "i/o timeout" #50191

net: TestLookup* failures with "i/o timeout" #50191

Comments

bcmills commented Dec 15, 2021

bcmills commented Dec 15, 2021

gopherbot commented Dec 15, 2021

gopherbot commented Dec 15, 2021

bcmills commented Jan 10, 2022

toothrot commented Jan 19, 2022

bcmills commented May 31, 2022

gopherbot commented May 31, 2022