Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: TestSelectFairness failures with "unfair select" #52465

Closed
bcmills opened this issue Apr 21, 2022 · 6 comments
Closed

runtime: TestSelectFairness failures with "unfair select" #52465

bcmills opened this issue Apr 21, 2022 · 6 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@bcmills
Copy link
Contributor

bcmills commented Apr 21, 2022 •
edited 

#!watchflakes
post <- pkg == "runtime" && test == "TestSelectFairness"

greplogs --dashboard -md -l -e 'FAIL: TestSelectFairness .*(?:\n .*)*unfair select'

2022-04-20T16:05:16-338a817/illumos-amd64
2022-03-23T19:31:43-2792a41/netbsd-386-9_0
2020-08-19T00:10:22-98a0071/js-wasm

@golang/runtime: the comment for this test suggests that it has a confidence level of “six nines”, which would be one spurious failure per ~1M runs. Is it still in line with that target?

See previously #34808, #22047.
@bcmills bcmills added this to the Backlog milestone Apr 21, 2022
@mengzhuo
Copy link
Contributor

The "six nines" requires less than 244 out of 10000 trials however the fail logs shows 5245,5253,5246.

@thanm thanm added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 22, 2022
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 7, 2022
@bcmills bcmills mentioned this issue Nov 21, 2022
Closed
@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime" && test == "TestSelectFairness"
2022-11-18 17:59 linux-386-buster go@b2faff18 runtime.TestSelectFairness (log) 
--- FAIL: TestSelectFairness (0.01s)
    chan_test.go:488: 5256 4744 0.5256 0.025599999999999956
    chan_test.go:490: unfair select: in 10000 trials, results were 5256, 4744

— watchflakes

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime" && test == "TestSelectFairness"
2023-07-31 17:13 netbsd-386-9_3 go@4a14d9c9 runtime.TestSelectFairness (log) 
--- FAIL: TestSelectFairness (0.02s)
    chan_test.go:488: 4742 5258 0.4742 0.02579999999999999
    chan_test.go:490: unfair select: in 10000 trials, results were 4742, 5258

— watchflakes

@randall77
Copy link
Contributor

This last one looks to be about a 1 in a million chance of being that unbalanced (258 extra heads above the expected 5000 out of 10000 trials).
Maybe there is an issue here? But I'm inclined to just soften the test. I don't think the world would end if selects weren't perfectly random (60/40, say).

@gopherbot
Copy link

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "runtime" && test == "TestSelectFairness"
2023-10-12 20:11 darwin-amd64-10_15 go@15a274b6 runtime.TestSelectFairness (log) 
--- FAIL: TestSelectFairness (0.01s)
    chan_test.go:488: 5247 4753 0.5247 0.024700000000000055
    chan_test.go:490: unfair select: in 10000 trials, results were 5247, 4753

— watchflakes

@gopherbot
Copy link

Change https://go.dev/cl/538815 mentions this issue: runtime: make select fairness test less picky

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
Test Flakes
Status: Done
Milestone
Backlog
Development

No branches or pull requests
5 participants
@mengzhuo @bcmills @randall77 @gopherbot @thanm