Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net/http: TestClientWriteShutdown failures on darwin-arm64-11_0-toothrot #49860

Closed
bcmills opened this issue Nov 29, 2021 · 6 comments
Closed

net/http: TestClientWriteShutdown failures on darwin-arm64-11_0-toothrot #49860

bcmills opened this issue Nov 29, 2021 · 6 comments
Labels
arch-arm64 FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Darwin WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Go1.19

Comments

@bcmills
Copy link
Contributor

bcmills commented Nov 29, 2021 

--- FAIL: TestClientWriteShutdown (10.00s)
    serve_test.go:3093: timeout
FAIL
FAIL	net/http	15.573s

greplogs --dashboard -md -l -e 'FAIL: TestClientWriteShutdown .+\n .+: timeout'

2021-11-29T00:58:50-1ea4d3b/darwin-arm64-11_0-toothrot
2021-09-02T19:16:19-9633195/darwin-arm64-11_0-toothrot
2021-08-23T19:49:23-0a7f00a/darwin-arm64-11_0-toothrot
2021-05-28T18:36:54-6624771/darwin-arm64-11_0-toothrot
2021-05-09T00:07:34-ec4efa4/darwin-arm64-11_0-toothrot
2021-04-27T18:54:49-0ae9c3b/darwin-arm64-11_0-toothrot
2021-04-12T21:59:59-263e13d/darwin-arm64-11_0-toothrot
2021-03-18T00:23:25-51e4bb2/darwin-arm64-11_0-toothrot
2021-03-13T11:52:17-b3235b7/darwin-arm64-11_0-toothrot

(attn @neild)
@bcmills bcmills added arch-arm64 NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Darwin labels Nov 29, 2021
@bcmills bcmills added this to the Backlog milestone Nov 29, 2021
@bcmills
Copy link
Contributor Author

bcmills commented Nov 29, 2021

This looks to me like a deadlock, but (due to the use of an unnecessary and arbitrary timeout) the test is currently not producing useful goroutine dumps to diagnose it.

We need to fix the test before we can fix the underlying deadlock.

@bcmills bcmills modified the milestones: Backlog, Go1.19 Nov 29, 2021
@bcmills
Copy link
Contributor Author

bcmills commented Nov 29, 2021

Marking as release-blocker for Go 1.19, rather than 1.18, because it appears to affect a single platform and it isn't obvious to me that we well have a useful goroutine dump to diagnose it before the 1.18 release.

(I suspect that we will want to backport the fix once it is identified.)

@gopherbot
Copy link

Change https://golang.org/cl/367615 mentions this issue: net/http: eliminate arbitrary timeout in TestClientWriteShutdown

gopherbot pushed a commit that referenced this issue Nov 30, 2021
net/http: eliminate arbitrary timeout in TestClientWriteShutdown
18934e1 
This test occasionally hangs on the darwin-arm64-11_0-toothrot
builder. When it does, it fails with the unhelpful error message
"timeout" instead of a useful goroutine dump.

This change eliminates the use of an arbitrary timeout channel, so
that if (and probably when) the test hangs again we will get more
useful logs to diagnose the root cause.

For #49860

Change-Id: I23f6f1c81209f0b2dbe565e1dfb26b1b2eff0187
Reviewed-on: https://go-review.googlesource.com/c/go/+/367615
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
@bcmills
Copy link
Contributor Author

bcmills commented Mar 16, 2022

Curiously, no repros since the timeout was removed. Maybe it wasn't a deadlock after all. 🤔

greplogs --dashboard -md -l -e 'FAIL: TestClientWriteShutdown .+\n .+: timeout' --since=2021-11-30

@bcmills bcmills added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed release-blocker labels Mar 16, 2022
@gopherbot
Copy link

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

@cherrymui
Copy link
Member

Observed a recent failure on 1.17 release branch
2022-07-25T23:15:10-489c148/darwin-arm64-11_0-toothrot

CL https://golang.org/cl/367615 didn't make into 1.17. So this is still observing old behavior. Not new, then.

@cherrymui cherrymui mentioned this issue Jul 27, 2022
Closed
@golang golang locked and limited conversation to collaborators Jul 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
arch-arm64 FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Darwin WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Go1.19
Development

No branches or pull requests
3 participants
@bcmills @gopherbot @cherrymui