Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/net/http2: frequent failures in TestClientConnCloseAtBody #49335

Closed
bcmills opened this issue Nov 4, 2021 · 5 comments
Closed

x/net/http2: frequent failures in TestClientConnCloseAtBody #49335

bcmills opened this issue Nov 4, 2021 · 5 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker Testing An issue that has been verified to require only test changes, not just a test failure.
Milestone

Comments

@bcmills
Copy link
Contributor

bcmills commented Nov 4, 2021

This test has been failing typically multiple times per day since at least 2021-10-04.

--- FAIL: TestClientConnCloseAtBody (0.01s)
    transport_test.go:4572: got 1 active streams, want 0
FAIL
FAIL	golang.org/x/net/http2	22.488s

greplogs --dashboard -md -l -e 'FAIL: TestClientConnCloseAtBody'

2021-11-03T18:37:22-4a448f8-cfd016d/freebsd-amd64-race
2021-11-03T16:57:44-4a448f8-b212ba6/windows-amd64-race
2021-11-03T05:00:00-4a448f8-519c0a2/windows-amd64-longtest
2021-11-02T23:44:58-4a448f8-a0f373c/windows-amd64-race
2021-11-02T20:59:34-4a448f8-79024cf/windows-amd64-race
2021-11-02T20:26:55-4a448f8-7c9510e/linux-amd64-sid
2021-11-02T19:37:42-4a448f8-1011e26/freebsd-amd64-race
2021-11-02T18:31:53-4a448f8-79c5240/windows-amd64-race
2021-11-02T17:31:50-4a448f8-4e7dd9f/windows-amd64-race
2021-11-02T16:19:45-4a448f8-f2ff1c6/windows-amd64-2016
2021-11-02T03:55:19-4a448f8-6f1e9a9/freebsd-amd64-race
2021-11-01T21:27:46-4a448f8-f9cb33c/freebsd-amd64-race
2021-11-01T21:27:26-4a448f8-631b567/linux-amd64-race
2021-11-01T19:34:20-4a448f8-d19c5bd/freebsd-amd64-race
2021-11-01T19:34:20-4a448f8-4a84298/windows-amd64-race
2021-10-31T18:39:05-9967326-89c5270/windows-amd64-2016
2021-10-31T18:13:09-9967326-fd09e88/freebsd-amd64-race
2021-10-31T15:36:50-9967326-243c5ae/android-386-emu
2021-10-31T02:24:29-9967326-0bef30d/windows-amd64-race
2021-10-29T23:15:00-9967326-8dfb447/windows-amd64-longtest
2021-10-29T22:46:45-9967326-4a84298/freebsd-amd64-race
2021-10-29T22:01:56-540bb53-c76893d/windows-386-2008
2021-10-29T19:42:51-540bb53-4a84298/windows-amd64-race
2021-10-29T18:56:29-540bb53-6afdf01/windows-amd64-race
2021-10-29T02:12:05-d418f37-d3d8852/windows-amd64-2016
2021-10-28T18:17:57-d418f37-f229e70/freebsd-amd64-race
2021-10-28T15:55:26-d418f37-61536ec/linux-amd64-race
2021-10-28T15:35:25-d418f37-278b9b3/windows-amd64-2016
2021-10-28T14:25:03-d418f37-a3bb28e/linux-amd64-race
2021-10-28T03:44:12-d418f37-2ff1074/windows-amd64-race
2021-10-28T01:15:26-d418f37-b2fe2eb/windows-amd64-race
2021-10-28T01:03:38-d418f37-12dc48f/freebsd-amd64-race
2021-10-27T06:23:35-d418f37-5786a54/freebsd-amd64-12_2
2021-10-27T06:23:35-d418f37-5786a54/windows-amd64-race
2021-10-27T05:33:58-d418f37-ca5f65d/freebsd-amd64-race
2021-10-26T22:05:53-d418f37-80be4a4/windows-amd64-race
2021-10-26T21:17:38-d418f37-091db63/freebsd-amd64-race
2021-10-26T05:05:24-d418f37-23fdd7f/windows-amd64-race
2021-10-26T01:22:47-d418f37-adfb85b/windows-amd64-race
2021-10-26T01:18:46-d418f37-49fda9e/windows-amd64-race
2021-10-25T21:16:46-d418f37-11b64b4/windows-amd64-race
2021-10-25T21:03:33-d418f37-60c3069/windows-amd64-2016
2021-10-25T18:46:45-d418f37-f47335e/linux-amd64-race
2021-10-25T15:31:40-d418f37-89cc528/freebsd-amd64-race
2021-10-25T14:50:31-d418f37-1ed060c/windows-amd64-race
2021-10-23T20:23:35-d418f37-8dbf3e9/freebsd-amd64-race
2021-10-20T00:05:22-c6ed85c-4cc6a91/windows-amd64-2016
2021-10-19T23:23:29-c6ed85c-2be5b84/android-amd64-emu
2021-10-19T23:23:29-c6ed85c-2ac3bdf/freebsd-amd64-race
2021-10-19T18:22:59-4f30a5c-6c3cd5d/windows-amd64-race
2021-10-19T16:19:27-4f30a5c-f92a358/freebsd-amd64-race
2021-10-19T15:03:39-4f30a5c-6294207/freebsd-amd64-race
2021-10-19T09:49:38-4f30a5c-b0351bf/windows-amd64-2016
2021-10-18T22:56:07-4f30a5c-eba0e86/freebsd-amd64-race
2021-10-18T22:56:07-4f30a5c-eba0e86/linux-amd64-race
2021-10-18T21:57:36-4f30a5c-3befaf0/freebsd-amd64-race
2021-10-18T21:33:08-4f30a5c-33b3260/windows-386-2008
2021-10-18T15:38:40-4f30a5c-74acbaf/windows-amd64-2016
2021-10-16T16:27:40-4f30a5c-640a49b/windows-amd64-race
2021-10-15T21:04:44-4f30a5c-c580180/windows-amd64-race
2021-10-15T21:04:44-4f30a5c-2ac3bdf/windows-amd64-race
2021-10-15T20:34:15-db2dff3-1b072b3/freebsd-amd64-11_4
2021-10-15T20:34:15-db2dff3-1b072b3/freebsd-amd64-race
2021-10-15T20:29:48-db2dff3-85cbdda/windows-amd64-race
2021-10-15T19:54:44-db2dff3-22951fb/windows-amd64-2016
2021-10-15T05:20:52-fd004c5-81484cf/linux-amd64-race
2021-10-15T02:02:50-fd004c5-3da0ff8/freebsd-amd64-race
2021-10-14T23:32:09-fd004c5-9e8ed86/freebsd-amd64-race
2021-10-14T22:23:26-fd004c5-2ac3bdf/windows-amd64-race
2021-10-14T17:25:44-2b766c0-fbdf830/windows-amd64-race
2021-10-13T20:31:25-e13a265-b90d258/windows-amd64-race
2021-10-13T17:16:12-e13a265-4a3daee/linux-amd64-race
2021-10-13T17:16:12-e13a265-4a3daee/windows-amd64-race
2021-10-13T16:36:59-ee2e9a0-69041c7/linux-amd64-race
2021-10-13T15:56:29-ee2e9a0-5326094/linux-amd64-race
2021-10-12T18:31:29-caeb26a-f8bfdc9/windows-amd64-race
2021-10-12T18:00:16-caeb26a-6e0adde/freebsd-amd64-race
2021-10-12T11:00:47-caeb26a-9c1dbdf/linux-amd64-race
2021-10-12T06:55:50-caeb26a-d887d3b/windows-amd64-2016
2021-10-11T22:34:49-caeb26a-d90f0b9/windows-amd64-race
2021-10-11T22:17:47-caeb26a-b41030e/linux-amd64-clang
2021-10-11T17:04:08-caeb26a-2ac3bdf/windows-amd64-race
2021-10-08T19:30:58-59d4e92-f430bda/freebsd-amd64-race
2021-10-08T18:07:25-59d4e92-d480b5c/linux-amd64-race
2021-10-08T17:58:41-59d4e92-16a3cef/freebsd-amd64-12_2
2021-10-08T15:27:20-59d4e92-b3a4b2d/freebsd-amd64-race
2021-10-07T20:37:43-59d4e92-ef2ebbe/windows-amd64-race
2021-10-07T18:11:41-59d4e92-2ac3bdf/windows-amd64-2016
2021-10-07T18:11:41-59d4e92-2ac3bdf/windows-amd64-race
2021-10-07T16:38:24-59d4e92-be571a3/linux-amd64-race
2021-10-07T14:01:52-59d4e92-ecb2f23/freebsd-amd64-race
2021-10-07T14:01:52-59d4e92-ecb2f23/windows-amd64-race
2021-10-07T00:39:17-62292e8-812a33d/freebsd-amd64-race
2021-10-06T22:42:28-62292e8-4002616/freebsd-amd64-race
2021-10-06T22:42:28-62292e8-4002616/linux-amd64-race
2021-10-06T20:21:49-62292e8-f580b75/freebsd-amd64-race
2021-10-06T19:08:15-62292e8-6487b15/freebsd-amd64-race
2021-10-06T16:46:09-d2e5035-3160571/freebsd-amd64-race
2021-10-05T22:42:11-d2e5035-96fface/freebsd-amd64-race
2021-10-05T20:35:54-d4b1ae0-990c9c6/linux-amd64-race
2021-10-05T16:06:49-d4b1ae0-060cd73/windows-amd64-2016
2021-10-05T14:12:44-d4b1ae0-7983830/windows-amd64-race
2021-10-05T07:32:41-d4b1ae0-d55009c/windows-amd64-race
2021-10-04T22:59:44-69340ce-78c2529/linux-amd64-fedora
2021-10-04T22:05:34-69340ce-7d57324/windows-amd64-race
2021-10-04T22:05:34-69340ce-04242ac/windows-amd64-race
2021-10-04T20:57:50-b30845b-e8a85e9/freebsd-amd64-race
2021-10-04T19:50:52-b30845b-7d822af/freebsd-amd64-race
2021-10-04T17:17:11-cedda3a-cc5e3de/windows-amd64-race
2021-10-04T16:44:53-cedda3a-04242ac/windows-amd64-race

(CC @neild)

@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker labels Nov 4, 2021
@bcmills bcmills added this to the Go1.18 milestone Nov 4, 2021
@gopherbot
Copy link

Change https://golang.org/cl/361536 mentions this issue: http2: don't count aborted streams as active in tests

@neild
Copy link
Contributor

neild commented Dec 1, 2021

@gopherbot Please open backport issues for 1.16 and 1.17. Test-only fix for flaky failures.

@gopherbot
Copy link

Backport issue(s) opened: #49910 (for 1.16), #49911 (for 1.17).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@gopherbot
Copy link

Change https://golang.org/cl/368554 mentions this issue: [internal-branch.go1.16-vendor] http2: don't count aborted streams as active in tests

@gopherbot
Copy link

Change https://golang.org/cl/368555 mentions this issue: [internal-branch.go1.17-vendor] http2: don't count aborted streams as active in tests

gopherbot pushed a commit to golang/net that referenced this issue Dec 1, 2021
… active in tests

Deflake TestClientConnCloseAtBody, which assumes all streams
have been fully cleaned up after ClientConn.Close returns.
Only count active streams, not ones in the process of aborting.

Perhaps ClientConn.Close should block until all streams have
been cleaned up, although this could result in Close blocking
indefinitely if a stream is blocked reading from the request body.

For golang/go#49335.
For golang/go#49910.

Change-Id: I172e0d3f10875191d29b24598de0abbdeb35e434
Reviewed-on: https://go-review.googlesource.com/c/net/+/361536
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit b53810d)
Reviewed-on: https://go-review.googlesource.com/c/net/+/368554
Reviewed-by: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit to golang/net that referenced this issue Dec 1, 2021
… active in tests

Deflake TestClientConnCloseAtBody, which assumes all streams
have been fully cleaned up after ClientConn.Close returns.
Only count active streams, not ones in the process of aborting.

Perhaps ClientConn.Close should block until all streams have
been cleaned up, although this could result in Close blocking
indefinitely if a stream is blocked reading from the request body.

For golang/go#49335.
For golang/go#49911.

Change-Id: I172e0d3f10875191d29b24598de0abbdeb35e434
Reviewed-on: https://go-review.googlesource.com/c/net/+/361536
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit b53810d)
Reviewed-on: https://go-review.googlesource.com/c/net/+/368555
Reviewed-by: Michael Knyszek <mknyszek@google.com>
@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Dec 1, 2021
dteh pushed a commit to dteh/fhttp that referenced this issue Jun 22, 2022
Deflake TestClientConnCloseAtBody, which assumes all streams
have been fully cleaned up after ClientConn.Close returns.
Only count active streams, not ones in the process of aborting.

Perhaps ClientConn.Close should block until all streams have
been cleaned up, although this could result in Close blocking
indefinitely if a stream is blocked reading from the request body.

Fixes golang/go#49335.

Change-Id: I172e0d3f10875191d29b24598de0abbdeb35e434
Reviewed-on: https://go-review.googlesource.com/c/net/+/361536
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
fedosgad pushed a commit to fedosgad/oohttp that referenced this issue Jun 22, 2022
… active in tests

Deflake TestClientConnCloseAtBody, which assumes all streams
have been fully cleaned up after ClientConn.Close returns.
Only count active streams, not ones in the process of aborting.

Perhaps ClientConn.Close should block until all streams have
been cleaned up, although this could result in Close blocking
indefinitely if a stream is blocked reading from the request body.

For golang/go#49335.
For golang/go#49911.

Change-Id: I172e0d3f10875191d29b24598de0abbdeb35e434
Reviewed-on: https://go-review.googlesource.com/c/net/+/361536
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit b53810dc28af183c5c4adfbb6b6d982379845b06)
Reviewed-on: https://go-review.googlesource.com/c/net/+/368555
Reviewed-by: Michael Knyszek <mknyszek@google.com>
@golang golang locked and limited conversation to collaborators Dec 1, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker Testing An issue that has been verified to require only test changes, not just a test failure.
Projects
None yet
Development

No branches or pull requests

4 participants