Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/net/http2: TestTransportGroupsPendingDials failures with "conns = …; want empty" #43176

Closed
bcmills opened this issue Dec 14, 2020 · 5 comments
Closed

x/net/http2: TestTransportGroupsPendingDials failures with "conns = …; want empty" #43176

bcmills opened this issue Dec 14, 2020 · 5 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.18

Comments

@bcmills
Copy link
Contributor

bcmills commented Dec 14, 2020 

--- FAIL: TestTransportGroupsPendingDials (1.00s)
    transport_test.go:321: State of pool after CloseIdleConnections: conns = map[127.0.0.1:57918:[0xc000290600]]; want empty
FAIL
FAIL	golang.org/x/net/http2	21.794s

2020-12-09T12:38:23-ac852fb/dragonfly-amd64-5_8
2020-12-02T16:19:06-c7110b5/dragonfly-amd64-5_8
2020-11-10T03:11:24-69a7880/dragonfly-amd64
2020-11-10T03:11:24-69a7880/dragonfly-amd64-5_8
2020-10-29T05:33:32-8adf50f/dragonfly-amd64-5_8
2020-10-22T23:12:55-08b3837/dragonfly-amd64-5_8
2020-10-08T22:28:04-59f7323/dragonfly-amd64
2020-10-06T15:34:59-a7d1128/dragonfly-amd64-5_8
2020-10-02T20:24:02-0a1ea39/dragonfly-amd64
2020-10-02T20:24:02-0a1ea39/dragonfly-amd64-5_8
2020-09-30T14:50:03-4acb6c0/dragonfly-amd64-5_8
2020-09-23T18:22:12-328152d/dragonfly-amd64-5_8
2020-09-04T19:48:48-62affa3/dragonfly-amd64
2020-05-20T18:23:14-0ba52f6/dragonfly-amd64-5_8
2020-05-19T11:38:04-d87ec0c/dragonfly-amd64
2020-05-19T11:38:04-d87ec0c/dragonfly-amd64-5_8

CC @bradfitz @ianlancetaylor (for CL 17134), @tombergan @empijei @fraenkel @neild for net/http2.
@bcmills bcmills added OS-Dragonfly NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Dec 14, 2020
@bcmills bcmills added this to the Backlog milestone Dec 14, 2020
@dmitshur dmitshur changed the title net/http2: TestTransportGroupsPendingDials failures on dragonfly builders x/net/http2: TestTransportGroupsPendingDials failures on dragonfly builders May 21, 2021
@bcmills bcmills changed the title x/net/http2: TestTransportGroupsPendingDials failures on dragonfly builders x/net/http2: TestTransportGroupsPendingDials failures with "conns = …; want empty" Dec 7, 2021
@bcmills
Copy link
Contributor Author

bcmills commented Dec 7, 2021

This does not appear to be Dragonfly-specific: it has at least also occurred on FreeBSD.

greplogs --dashboard -md -l -e 'FAIL: TestTransportGroupsPendingDials' --since=2020-12-15

2021-11-15T23:55:09-69e39ba-9e13a88/freebsd-arm-paulzhol
2021-10-25T03:00:02-d418f37-7b55457/freebsd-arm-paulzhol
2021-09-15T23:28:10-943fd67-abc4f09/freebsd-arm-paulzhol
2021-07-30T20:45:25-c6fcb2d-ae7943e/dragonfly-amd64
2021-06-10T13:23:58-84b48f8-7677616/dragonfly-amd64
2021-06-09T18:34:25-abc4532-182157c/dragonfly-amd64
2021-05-20T13:19:43-4163338-f07e4da/dragonfly-amd64
2021-05-08T03:27:59-0714010-b211fe0/dragonfly-amd64-5_8
2021-05-05T01:48:39-bbd867f-caf4c94/dragonfly-amd64
2021-05-01T14:20:56-aec3718-72ccabc/dragonfly-amd64-5_8
2021-04-30T18:46:51-89ef3d9-afa58dd/dragonfly-amd64-5_8
2021-04-30T00:08:50-89ef3d9-02ab8d1/dragonfly-amd64-5_8
2021-04-20T01:47:04-e915ea6-0ccdcb2/dragonfly-amd64
2021-03-30T08:39:13-22f4162-a81b5e4/dragonfly-amd64-5_8
2021-03-24T14:38:53-2c4c8ec-e8700f1/dragonfly-amd64-5_8
2021-03-24T05:16:36-2c4c8ec-f39c4de/dragonfly-amd64
2021-03-16T13:06:04-d523dce-26c32de/dragonfly-amd64-5_8
2021-03-16T01:40:41-34ac3e1-15f1670/dragonfly-amd64
2021-02-25T22:38:21-3d97a24-5f15af1/dragonfly-amd64
2021-02-25T19:58:05-3d97a24-7fcf989/freebsd-arm-paulzhol
2021-02-25T17:50:59-3d97a24-bcac57f/dragonfly-amd64-5_8
2021-02-25T02:22:12-3d97a24-d822ffe/dragonfly-amd64
2021-01-19T17:33:33-6772e93-ccb2e90/dragonfly-amd64-5_8
2021-01-19T12:49:13-6772e93-d047c91/dragonfly-amd64-5_8
2021-01-18T18:16:12-6772e93-dbab079/dragonfly-amd64
2021-01-15T18:29:43-6772e93-1deae0b/dragonfly-amd64-5_8
2021-01-15T15:33:02-6772e93-ff196c3/dragonfly-amd64-5_8

@bcmills
Copy link
Contributor Author

bcmills commented Dec 7, 2021

I think this failure indicates a real (but perhaps minor?) bug in (*Transport).CloseIdleConnections.

From its documentation, I would expect CloseIdleConnections to block until the idle connections have actually all been closed, in which case they should no longer be present in the pool.

@bcmills bcmills mentioned this issue Dec 7, 2021
Open
@bcmills bcmills modified the milestones: Backlog, Go1.18 Dec 7, 2021
@bcmills bcmills self-assigned this Dec 7, 2021
@bcmills
Copy link
Contributor Author

bcmills commented Dec 7, 2021

Filed #50027 for the suspected real bug, which is more-or-less orthogonal to the purpose of the test. I think we can make the test less flaky by reducing its expectations regarding connection idleness.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/369936 mentions this issue: http2: in TestTransportGroupsPendingDials, call CloseIdleConnections on every retry

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/370175 mentions this issue: http2: deflake TestTransportGroupsPendingDials

@bcmills bcmills mentioned this issue May 19, 2022
Open
dteh pushed a commit to dteh/fhttp that referenced this issue Jun 22, 2022
@neild
http2: deflake TestTransportGroupsPendingDials
310e6d4 
This test makes assumptions about the internal structure of *clientConnPool,
as well as assuming that a goroutine will schedule and run within one
second. The former assumption isn't necessary, and the latter causes
flakiness.

Refactor the test to count dial and close calls, which is all it needs
to test the desired behavior (one pending dial per destination).

Fixes golang/go#43176.

Change-Id: I125b110f196e29f303960c6851089118f8fb5d38
Reviewed-on: https://go-review.googlesource.com/c/net/+/370175
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
@rsc rsc unassigned bcmills Jun 23, 2022
WeiminShang added a commit to WeiminShang/net that referenced this issue Nov 16, 2022
@WeiminShang @neild
http2: deflake TestTransportGroupsPendingDials
bd59aac 
This test makes assumptions about the internal structure of *clientConnPool,
as well as assuming that a goroutine will schedule and run within one
second. The former assumption isn't necessary, and the latter causes
flakiness.

Refactor the test to count dial and close calls, which is all it needs
to test the desired behavior (one pending dial per destination).

Fixes golang/go#43176.

Change-Id: I125b110f196e29f303960c6851089118f8fb5d38
Reviewed-on: https://go-review.googlesource.com/c/net/+/370175
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
@golang golang locked and limited conversation to collaborators Jun 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.18
Development

No branches or pull requests
2 participants
@bcmills @gopherbot