Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: TestServerGracefulClose failures with connection reset by peer #57084

Closed
gopherbot opened this issue Dec 5, 2022 · 17 comments
Closed
Assignees
Labels
NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure.
Milestone

Comments

@gopherbot
Copy link

gopherbot commented Dec 5, 2022

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`

Issue created automatically to collect these failures.

Example (log):

--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (4.24s)
        serve_test.go:3153: ReadLine: read tcp 127.0.0.1:33546->127.0.0.1:33545: read: connection reset by peer

watchflakes

@gopherbot gopherbot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 5, 2022
@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose"
2022-12-02 20:16 freebsd-riscv64-unmatched go@5efa4dab net/http.TestServerGracefulClose (log)
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (4.24s)
        serve_test.go:3153: ReadLine: read tcp 127.0.0.1:33546->127.0.0.1:33545: read: connection reset by peer

watchflakes

@bcmills bcmills changed the title net/http: TestServerGracefulClose failures net/http: TestServerGracefulClose failures with connection reset by peer Dec 6, 2022
@bcmills
Copy link
Contributor

bcmills commented Dec 6, 2022

(attn @neild)

@bcmills
Copy link
Contributor

bcmills commented Dec 6, 2022

See also #36178.

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2022-12-05 16:41 freebsd-arm64-dmgk go@ad55b878 net/http.TestServerGracefulClose (log)
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (4.89s)
        serve_test.go:3153: ReadLine: read tcp 127.0.0.1:29763->127.0.0.1:29762: read: connection reset by peer
2022-12-19 21:44 darwin-amd64-12_0 go@7683c71a net/http.TestServerGracefulClose (log)
2022/12/19 20:58:18 httptest.Server blocked in Close after 5 seconds, waiting for connections:
  *net.TCPConn 0xc00053a008 127.0.0.1:50540 in state active
2022/12/19 20:58:18 httptest.Server blocked in Close after 5 seconds, waiting for connections:
  *net.TCPConn 0xc00053a118 127.0.0.1:50559 in state active
--- FAIL: TestServerGracefulClose (3.00s)
    serve_test.go:3151: ReadLine: read tcp 127.0.0.1:50835->127.0.0.1:50834: read: connection reset by peer
2022-12-21 19:08 darwin-amd64-nocgo go@856ec210 net/http.TestServerGracefulClose (log)
2022/12/21 19:27:32 httptest.Server blocked in Close after 5 seconds, waiting for connections:
  *net.TCPConn 0xc000096040 127.0.0.1:50524 in state active
2022/12/21 19:27:32 httptest.Server blocked in Close after 5 seconds, waiting for connections:
  *net.TCPConn 0xc000203880 127.0.0.1:50527 in state active
2022/12/21 19:27:32 httptest.Server blocked in Close after 5 seconds, waiting for connections:
  *net.TCPConn 0xc000174070 127.0.0.1:50540 in state active
--- FAIL: TestServerGracefulClose (2.41s)
    serve_test.go:3144: ReadLine: read tcp 127.0.0.1:50784->127.0.0.1:50783: read: connection reset by peer
2022-12-22 19:21 darwin-amd64-12_0 go@91bc4cd1 net/http.TestServerGracefulClose (log)
--- FAIL: TestServerGracefulClose (2.75s)
    serve_test.go:3151: ReadLine: read tcp 127.0.0.1:50514->127.0.0.1:50513: read: connection reset by peer

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-01-04 22:23 darwin-amd64-11_0 go@e04be8b2 net/http.TestServerGracefulClose (log)
--- FAIL: TestServerGracefulClose (4.77s)
    serve_test.go:3151: ReadLine: read tcp 127.0.0.1:50617->127.0.0.1:50616: read: connection reset by peer

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-02-09 15:21 dragonfly-amd64-622 go@3161081c net/http.TestServerGracefulClose (log)
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (1.11s)
        serve_test.go:3149: ReadLine: read tcp 127.0.0.1:50143->127.0.0.1:58048: read: connection reset by peer

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-03-28 17:34 netbsd-arm64-bsiegert go@c02e1bfb net/http.TestServerGracefulClose (log)
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (2.80s)
        serve_test.go:3099: ReadLine: read tcp 127.0.0.1:58793->127.0.0.1:58794: read: connection reset by peer

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-05-03 22:13 netbsd-arm64-bsiegert go@f742ddc3 net/http.TestServerGracefulClose (log)
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (2.05s)
        serve_test.go:3105: ReadLine: read tcp 127.0.0.1:63207->127.0.0.1:63208: read: connection reset by peer

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-05-10 21:43 dragonfly-amd64-622 go@8e8f4be3 net/http.TestServerGracefulClose (log)
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (1.01s)
        serve_test.go:3105: ReadLine: read tcp 127.0.0.1:56413->127.0.0.1:51576: read: connection reset by peer

watchflakes

@bcmills bcmills added this to the Backlog milestone May 17, 2023
@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-05-16 21:06 freebsd-arm64-dmgk go@298ff309 net/http.TestServerGracefulClose (log)
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (2.94s)
        serve_test.go:3105: ReadLine: read tcp 127.0.0.1:57637->127.0.0.1:57636: read: connection reset by peer

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-06-06 15:33 dragonfly-amd64-622 go@46249e56 net/http.TestServerGracefulClose (log)
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (1.02s)
        serve_test.go:3105: ReadLine: read tcp 127.0.0.1:50051->127.0.0.1:64824: read: connection reset by peer

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-07-18 05:23 netbsd-arm-bsiegert go@28ca813a net/http.TestServerGracefulClose (log)
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (2.30s)
        serve_test.go:3108: ReadLine: read tcp 127.0.0.1:54521->127.0.0.1:54522: read: connection reset by peer

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-08-17 16:33 freebsd-arm64-dmgk go@51fd0cb3 net/http.TestServerGracefulClose (log)
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (5.16s)
        serve_test.go:3108: ReadLine: read tcp 127.0.0.1:65279->127.0.0.1:65278: read: connection reset by peer

watchflakes

@gopherbot
Copy link
Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-09-06 14:19 freebsd-arm64-dmgk go@d25a9355 net/http.TestServerGracefulClose (log)
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (3.14s)
        serve_test.go:3108: ReadLine: read tcp 127.0.0.1:13595->127.0.0.1:13594: read: connection reset by peer

watchflakes

@bcmills
Copy link
Contributor

bcmills commented Sep 11, 2023

This seems to be closely related to #51104 / #3595.

@gopherbot
Copy link
Author

Change https://go.dev/cl/527196 mentions this issue: net/http: scale rstAvoidanceDelay to reduce test flakiness

@bcmills bcmills self-assigned this Sep 11, 2023
@bcmills bcmills added the Testing An issue that has been verified to require only test changes, not just a test failure. label Sep 11, 2023
@bcmills bcmills modified the milestones: Backlog, Go1.22 Sep 11, 2023
@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Sep 11, 2023
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 11, 2023
@gopherbot
Copy link
Author

Change https://go.dev/cl/528398 mentions this issue: net/http: avoid leaking goroutines when TestServerGracefulClose retries

gopherbot pushed a commit that referenced this issue Sep 15, 2023
If the call to ReadString returns an error, the closure in
testServerGracefulClose will return an error and retry the test with a
longer timeout. If that happens, we need to wait for the conn.Write
goroutine to complete so that we don't leak connections across tests.

Updates #57084.
Fixes #62643.

Change-Id: Ia86c1bbd0a5e5d0aeccf4dfeb994c19d1fb10b00
Reviewed-on: https://go-review.googlesource.com/c/go/+/528398
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure.
Projects
Status: Done
Development

No branches or pull requests

2 participants