Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/net/http2: TestServerDos_.* failures with "timeout waiting for frame" #50556

Closed
bcmills opened this issue Jan 11, 2022 · 1 comment
Closed
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker Testing An issue that has been verified to require only test changes, not just a test failure.
Milestone

Comments

@bcmills
Copy link
Contributor

bcmills commented Jan 11, 2022

(Pulled out from #34615 (comment), since the underlying cause may or may not be the same.)

--- FAIL: TestServerDoS_MaxHeaderListSize (2.27s)
    server_test.go:511: Error while expecting a HEADERS frame: timeout waiting for frame
    server_test.go:261: Framer read log:
        2022-01-11 11:07:02.018643035 Framer 0x40001300e0: read SETTINGS len=24, settings: MAX_FRAME_SIZE=1048576, MAX_CONCURRENT_STREAMS=250, MAX_HEADER_LIST_SIZE=1048896, INITIAL_WINDOW_SIZE=1048576
        2022-01-11 11:07:02.019169963 Framer 0x40001300e0: read SETTINGS flags=ACK len=0
        2022-01-11 11:07:02.019334467 Framer 0x40001300e0: read WINDOW_UPDATE len=4 (conn) incr=983041
    server_test.go:267: Framer write log:
        2022-01-11 11:07:02.012021141 Framer 0x40001300e0: wrote SETTINGS len=0
        2022-01-11 11:07:02.018771496 Framer 0x40001300e0: wrote SETTINGS flags=ACK len=0
        2022-01-11 11:07:02.019595556 Framer 0x40001300e0: wrote HEADERS flags=END_STREAM stream=1 len=4065
        2022-01-11 11:07:02.134889194 Framer 0x40001300e0: wrote CONTINUATION flags=END_HEADERS stream=1 len=1048576
FAIL
FAIL	golang.org/x/net/http2	73.247s

greplogs --dashboard -md -l -e '(?ms)FAIL: TestServerDoS[_ ].* timeout waiting for frame.*^FAIL\s+(?:golang.org/x/)net/http'

2022-01-11T09:31:09-d55c255-9de1ac6/netbsd-arm64-bsiegert
2022-01-07T19:22:09-8b8fc08-378766a/netbsd-arm-bsiegert
2022-01-07T06:34:04-5b0dc2d-11b28e7/linux-riscv64-jsing
2021-12-08T04:14:00-db4efeb-a19e72c/freebsd-arm-paulzhol
2021-12-03T18:27:20-0a0e4e1-c4a8550/linux-riscv64-jsing
2021-11-22T19:43:16-6a13c67-8f559bc/js-wasm
2021-11-20T00:32:49-6a13c67-57aba32/openbsd-arm64-jsing
2021-11-17T21:26:25-47ca1ff-0440fb8/linux-riscv64-jsing
2021-11-10T18:24:20-ef0fda0-23dd389/linux-riscv64-jsing
2021-10-29T23:25:18-9967326-5d6d9f5/linux-riscv64-jsing
2021-10-28T03:35:34-d418f37-b8f928b/linux-riscv64-jsing
2021-10-27T19:18:20-d418f37-514ebae/linux-riscv64-jsing
2021-10-23T06:31:02-d418f37-6c20001/linux-riscv64-jsing
2021-10-18T16:33:42-4f30a5c-4d55072/linux-riscv64-jsing
2021-10-15T20:34:15-db2dff3-1b072b3/freebsd-arm-paulzhol
2021-10-11T17:04:08-caeb26a-2ac3bdf/netbsd-arm64-bsiegert
2021-10-10T15:11:26-3b03d30-d2d21d9/netbsd-arm-bsiegert
2021-10-09T01:04:29-3b03d30-e1c294a/linux-riscv64-jsing
2021-10-01T23:51:16-e81a3d9-c129af9/linux-riscv64-jsing
2021-09-28T04:43:08-7d9f5e0-7d57324/netbsd-arm-bsiegert
2021-09-17T00:31:49-12bc252-14e812b/netbsd-arm-bsiegert
2021-09-06T06:52:46-ad29c8a-c7f09ee/netbsd-arm-bsiegert
2021-09-03T16:21:42-ad29c8a-8226cb0/netbsd-arm-bsiegert
2021-07-30T20:45:25-c6fcb2d-ae7943e/netbsd-arm64-bsiegert
2021-06-12T00:12:55-84b48f8-9d46ee5/linux-riscv64-jsing
2021-06-10T20:20:58-84b48f8-770f1de/linux-riscv64-jsing
2021-05-20T17:08:46-37e1c6a-9636878/netbsd-arm-bsiegert
2021-05-17T22:19:03-4163338-bfe3573/freebsd-arm-paulzhol
2021-05-09T00:08:05-16afe75-14c3d2a/freebsd-arm-paulzhol
2021-04-27T23:12:57-85d9c07-983dea9/netbsd-arm-bsiegert
2021-04-27T13:59:22-5f58ad6-0b9ca4d/netbsd-arm-bsiegert
2021-04-20T00:49:17-e915ea6-4f5aec4/netbsd-arm64-bsiegert
2021-04-14T05:49:15-afb366f-58fdac0/netbsd-arm-bsiegert
2021-04-14T04:02:01-afb366f-6d8ba77/netbsd-arm-bsiegert
2021-04-02T21:57:58-0fccb6f-f25d78f/linux-riscv64-jsing
2021-03-31T06:09:03-cb1fcc7-1d967ab/netbsd-arm64-bsiegert
2021-03-31T04:13:44-e572328-135c9f4/netbsd-arm-bsiegert
2021-03-26T22:08:43-6ef6e9b-9c7463c/netbsd-arm-bsiegert
2021-03-26T06:03:03-6b15177-7c88ae4/netbsd-arm-bsiegert
2021-03-23T11:14:58-d523dce-53dd0d7/linux-riscv64-jsing
2021-03-23T04:23:52-d523dce-0265b64/netbsd-arm-bsiegert
2021-03-14T19:58:14-e18ecbb-a8b59fe/netbsd-arm64-bsiegert
2021-03-14T17:56:50-e18ecbb-061a690/netbsd-arm-bsiegert
2020-10-29T05:50:24-942e2f4-068911e/netbsd-arm-bsiegert
2020-10-27T06:32:35-146b70c-1095dd6/linux-riscv64-jsing
2020-10-26T22:00:15-146b70c-76bce1d/linux-riscv64-jsing
2020-10-02T08:52:16-4acb6c0-15bf061/netbsd-arm64-bsiegert
2020-05-15T16:15:25-a91f071-2b70ffe/freebsd-arm-paulzhol
2020-05-09T23:17:17-7e3656a-c9442dc/freebsd-arm-paulzhol
2020-05-05T15:41:37-1ed2336-b4ecafc/freebsd-arm-paulzhol
2020-02-25T08:36:15-5a598a2-450d0b2/freebsd-arm-paulzhol
2020-02-21T02:29:11-4628272-6dd11bc/freebsd-arm-paulzhol
2020-02-18T20:21:35-1617124-1cd724a/freebsd-arm-paulzhol
2020-02-17T10:30:03-1617124-88e564e/netbsd-arm-bsiegert
2020-02-13T19:49:45-1617124-3eab754/freebsd-arm-paulzhol
2020-01-10T21:56:50-c0dbc17-9720aff/freebsd-arm-paulzhol
2019-12-07T15:57:30-e7e4b65-da4d585/netbsd-arm-bsiegert
2019-12-06T23:53:46-1ddd1de-33d2076/freebsd-arm-paulzhol
2019-12-06T23:45:07-1ddd1de-ea89ce1/freebsd-arm-paulzhol
2019-12-02T22:42:22-ef20fe5-94f4686/netbsd-arm-bsiegert
2019-12-02T20:12:54-ef20fe5-a18608a/freebsd-arm-paulzhol
2019-11-27T23:14:19-ef20fe5-8a5af79/freebsd-arm-paulzhol
2019-11-27T15:06:55-ef20fe5-4e3d580/freebsd-arm-paulzhol
2019-11-26T19:05:22-ffdde10-27e50a1/freebsd-arm-paulzhol
2019-11-24T23:31:50-8f7fa26-8d72096/freebsd-arm-paulzhol
2019-11-23T00:06:55-fc4aabc-688aa74/netbsd-arm-bsiegert
2019-11-22T21:09:43-fc4aabc-9f3c2b6/freebsd-arm-paulzhol
2019-11-22T19:03:50-fc4aabc-941ac9c/freebsd-arm-paulzhol
2019-11-19T06:10:03-d06c31c-8cf5293/netbsd-arm-bsiegert
2019-11-18T05:26:46-f9c8255-2bde3c1/freebsd-arm-paulzhol
2019-11-16T20:31:45-f9c8255-6ba0be1/freebsd-arm-paulzhol
2019-11-16T16:09:21-f9c8255-a852806/netbsd-arm-bsiegert
2019-11-16T16:09:21-f9c8255-6219b48/netbsd-arm-bsiegert
2019-11-15T02:31:58-2180aed-498eaee/freebsd-arm-paulzhol
2019-11-04T05:27:25-0deb692-fb29e22/netbsd-arm-bsiegert
2019-10-27T23:36:14-53de4c7-9e6d3ca/netbsd-arm-bsiegert
2019-10-27T23:36:14-53de4c7-7ad8def/netbsd-arm-bsiegert
2019-10-27T21:20:02-d64ee3f-9e6d3ca/netbsd-arm-bsiegert
2019-10-03T12:52:41-2ec1893-64785bf/solaris-amd64-smartosbuildlet
2019-09-30T13:41:27-c5a3c61-2af114a/solaris-amd64-smartosbuildlet
2019-09-23T16:28:16-aa69164-cf06b9a/freebsd-arm-paulzhol
2019-09-18T13:04:20-a8b05e9-cf06b9a/solaris-amd64-smartosbuildlet
2019-09-06T06:20:27-ba9fcec-9da9904/solaris-amd64-smartosbuildlet
2019-08-23T17:22:08-74dc4d7-a5a92aa/freebsd-arm-paulzhol
2019-06-19T01:48:44-b5b0513-4ce6a8e/freebsd-arm-paulzhol
2019-06-11T14:12:13-3f473d3-06a61ef/freebsd-arm-paulzhol
2019-06-07T20:50:08-461777f-d497d80/netbsd-arm-bsiegert
2019-06-07T10:09:15-1492cef-8969b05/netbsd-arm-bsiegert
2019-06-03T01:37:58-f3200d1-98100c5/solaris-amd64-smartosbuildlet
2019-05-22T15:58:17-f3200d1-efa061d/solaris-amd64-smartosbuildlet
2019-05-22T15:58:17-f3200d1-afcfe0d/freebsd-arm-paulzhol
2019-05-21T16:00:24-018c4d4-1d1ba85/js-wasm
2019-05-20T16:08:56-3ec1911-82ee4e7/js-wasm
2019-05-17T14:26:32-3ec1911-afcfe0d/freebsd-arm-paulzhol
2019-05-16T19:56:27-3ec1911-cb5c82b/solaris-amd64-smartosbuildlet
2019-05-09T21:13:56-f4e77d3-a4f5c9c/js-wasm
2019-05-09T16:23:09-f4e77d3-ff70494/js-wasm
2019-05-08T18:24:30-f4e77d3-19966e9/solaris-amd64-smartosbuildlet
2019-05-08T01:49:03-f4e77d3-3403ee5/solaris-amd64-smartosbuildlet
2019-05-07T17:43:46-f4e77d3-49ad7bc/solaris-amd64-smartosbuildlet
2019-05-06T21:12:37-f4e77d3-3a1b4e7/netbsd-arm-bsiegert
2019-05-02T21:46:14-7f726ca-bf35b7c/js-wasm
2019-05-01T00:44:15-9ce7a69-ba978f5/solaris-amd64-smartosbuildlet
2019-04-30T22:56:01-4829fb1-c706d42/js-wasm
2019-04-30T22:20:57-4829fb1-65b89c3/js-wasm
2019-04-30T16:59:13-4829fb1-f686a28/js-wasm
2019-04-29T17:42:27-4829fb1-08b956f/js-wasm
2019-04-24T02:42:50-574d568-97dfbc5/js-wasm
2019-04-24T00:56:36-afa5a82-ca0c449/js-wasm
2019-04-24T00:09:18-afa5a82-de05071/js-wasm
2019-04-23T00:58:08-afa5a82-f0e9754/js-wasm
2019-04-22T14:08:47-afa5a82-db42bb3/js-wasm
2019-04-20T15:02:01-afa5a82-d23cba6/js-wasm
2019-04-19T12:32:46-1f3472d-bdd7bb5/js-wasm
2019-04-18T17:27:12-1da14a5-24a6478/js-wasm
2019-04-17T00:05:41-1da14a5-33e5da4/js-wasm
2019-04-16T20:27:15-1da14a5-5781df4/solaris-amd64-smartosbuildlet
2019-04-16T18:50:43-1da14a5-2bdbc94/js-wasm
2019-04-15T04:12:52-eb5bcb5-60a8dbf/js-wasm
2019-04-13T02:39:29-eb5bcb5-ee64b35/solaris-amd64-smartosbuildlet
2019-04-11T20:17:26-eb5bcb5-d86c35e/js-wasm
2019-04-02T23:18:37-74de082-64f22e4/js-wasm
2019-04-02T20:29:58-74de082-2038955/js-wasm
2019-04-02T05:45:33-74de082-5651721/js-wasm
2019-03-29T23:05:45-74de082-576442b/js-wasm
2019-03-28T19:34:53-63eda1e-57bd577/js-wasm
2019-03-28T02:38:28-63eda1e-ea9eddb/js-wasm
2019-03-27T15:20:28-710a502-5ee2290/js-wasm
2019-03-27T13:52:52-710a502-f24e109/js-wasm
2019-03-26T09:47:28-15845e8-cd53093/js-wasm
2019-03-26T03:33:13-e3b2ff5-17f888c/js-wasm
2019-03-25T07:55:12-e3b2ff5-9eef964/js-wasm
2019-03-21T21:05:13-1272bf9-f23c601/js-wasm
2019-03-20T14:39:14-1272bf9-af7b757/js-wasm
2019-03-19T16:01:31-d196dff-834d229/js-wasm
2019-03-19T08:30:50-d196dff-451a2eb/js-wasm
2019-03-19T04:35:33-d196dff-d24d25b/js-wasm
2019-03-18T19:57:52-9f648a6-991c85a/js-wasm
2019-03-18T14:05:54-9f648a6-6ca51f7/js-wasm
2019-03-18T07:58:23-9f648a6-e5f6e2d/js-wasm
2019-03-17T17:00:42-9f648a6-746f405/js-wasm
2019-03-16T14:16:39-9f648a6-14c3692/js-wasm
2019-03-15T21:45:04-9f648a6-356f5a1/js-wasm
2019-03-15T21:25:01-9f648a6-37e4a61/js-wasm
2019-03-15T04:00:43-9f648a6-d06704a/js-wasm
2019-03-13T15:11:37-5c2c250-04f1b65/js-wasm
2019-03-13T10:39:58-5c2c250-5f40351/js-wasm
2019-03-13T08:27:26-c19f08b-8390781/solaris-amd64-smartosbuildlet
2019-03-13T04:17:56-d888771-a891f2e/solaris-amd64-smartosbuildlet
2019-03-12T07:27:56-d888771-14a58d6/js-wasm
2019-03-12T05:00:20-d888771-9fedec7/js-wasm
2019-03-12T02:07:34-d888771-8b2a2d0/js-wasm
2019-03-11T19:18:03-d888771-486ca37/js-wasm
2019-03-10T07:45:41-c10a055-8390781/js-wasm
2019-03-10T07:45:41-c10a055-4dd46a6/js-wasm
2018-11-12T19:10:48-03003ca-402eb45/freebsd-arm-paulzhol
2018-08-26T01:23:51-8a410e7-2191fce/freebsd-arm-paulzhol

CC @neild @tombergan

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 11, 2022
@bcmills bcmills added this to the Go1.18 milestone Jan 11, 2022
@bcmills bcmills self-assigned this Jan 11, 2022
@gopherbot
Copy link

Change https://golang.org/cl/377814 mentions this issue: http2: eliminate arbitrary timeouts in readFrame methods

@bcmills bcmills added release-blocker Testing An issue that has been verified to require only test changes, not just a test failure. labels Jan 12, 2022
dteh pushed a commit to dteh/fhttp that referenced this issue Jun 22, 2022
Certain slower builders empirically do not finish before these
timeouts, and if a real deadlock should occur it would be more useful
to let the test itself time out and get a goroutine dump anyway.

Fixes golang/go#50556
Fixes golang/go#34615

Change-Id: I53ea616faa34f4ccc73af8eb18b794e12271b883
Reviewed-on: https://go-review.googlesource.com/c/net/+/377814
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Trust: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
@rsc rsc unassigned bcmills Jun 22, 2022
WeiminShang added a commit to WeiminShang/net that referenced this issue Nov 16, 2022
Certain slower builders empirically do not finish before these
timeouts, and if a real deadlock should occur it would be more useful
to let the test itself time out and get a goroutine dump anyway.

Fixes golang/go#50556
Fixes golang/go#34615

Change-Id: I53ea616faa34f4ccc73af8eb18b794e12271b883
Reviewed-on: https://go-review.googlesource.com/c/net/+/377814
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Trust: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
@golang golang locked and limited conversation to collaborators Jun 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker Testing An issue that has been verified to require only test changes, not just a test failure.
Projects
None yet
Development

No branches or pull requests

2 participants