New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: cmd/vet: add check for sync.WaitGroup abuse #18022
Comments
I don't like the idea. It's IMHO perhaps a task for |
I think this has been proposed before (probably on the mailing lists)
and rejected.
|
A func main() {
var wg sync.WaitGroup
defer wg.Wait()
go func() {
wg.Add(1)
defer wg.Done()
}()
} and In terms of vet's requirements:
@dominikh , |
As far as the proposal goes, I don't like how it limits the func signature to |
@dsnet The check in staticcheck has no (known) false positives. It shouldn't have a significant number of false negatives, either. The implementation is a simple pattern-based check, detecting I'm -1 on the proposed |
@dominikh I don't see any extra level of nesting here, though (assuming any func signature is allowed). To be nitpicky, another thing that stands out from the proposal is how |
@mvdan The extra level of nesting would come from a predicted usage that looks something like this:
as opposed to
Admittedly the same level of indentation, but syntactically it's one extra level of nesting. |
Ah yes, I was thinking indentation there. |
The problematic case is that
|
@cznic if you mean without the extra |
@mvdan Do you mean by allowing something like the following?
IMHO that's way too much |
panic is recovered? |
@dominikh true; I was simply pointing at the issue without contemplating a solution :) |
The API change here has the problems identified above with argument evaluation. Also, in general the libraries do not typically phrase functionality in terms of callbacks. If we're going to start using callbacks broadly, that should be a separate decision (and not one to make today). For both these reasons, it seems like .Go is not a clear win. It would be nice to have a vet check that we trust (no false positives). Perhaps it is enough to look for two statements
back to back and reject that always. Thoughts about how to make vet catch this reliably? |
I agree that |
It sounds like we are deciding to make go vet check this and not add new API here. Any arguments against that? |
SGTM |
I've added this proposal to the proposal process bin, but it's blocked on someone figuring out how to implement a useful check. Is anyone interested in doing that? |
Staticcheck has a fairly trivial check: for a GoStmt of a FuncLit, if the first statement in the FuncLit is a call to The check could be trivially hardened by
Edit: which is pretty much what you have suggested in #18022 (comment) |
Thanks for the info @dominikh. |
Ping @dominikh. Thanks. |
Sorry @rsc, I missed your earlier comment. Sure, that's fine by me. I can send code if it helps, but I'd have to write it from scratch – the check in staticcheck makes use of staticcheck internals. |
@dominikh sure, if you feel like sending code please do. Otherwise anyone else who wants to is welcome. I just want to make sure we're not overstepping versus staticcheck (which I'm a big fan of). |
On hold for details of the eventual implemented check. |
The addition of WaitGroup.Go in the standard library has been repeatedly proposed and rejected. See golang/go#18022, golang/go#23538, and golang/go#39863 In summary, the argument for WaitGroup.Go is that it avoids bugs like: go func() { wg.Add(1) defer wg.Done() ... }() where the increment happens after execution (not before) and also (to a lesser degree) because: wg.Go(func() { ... }) is shorter and more readble. The argument against WaitGroup.Go is that the provided function takes no arguments and so inputs and outputs must closed over by the provided function. The most common race bug for goroutines is that the caller forgot to capture the loop iteration variable, so this pattern may make it easier to be accidentally racy. However, that is changing with golang/go#57969. In my experience the probability of race bugs due to the former still outwighs the latter, but I have no concrete evidence to prove it. The existence of errgroup.Group.Go and frequent utility of the method at least proves that this is a workable pattern and the possibility of accidental races do not appear to manifest as frequently as feared. A reason *not* to use errgroup.Group everywhere is that there are many situations where it doesn't make sense for the goroutine to return an error since the error is handled in a different mechanism (e.g., logged and ignored, formatted and printed to the frontend, etc.). While you can use errgroup.Group by always returning nil, the fact that you *can* return nil makes it easy to accidentally return an error when nothing is checking the return of group.Wait. This is not a hypothetical problem, but something that has bitten us in usages that was only using errgroup.Group without intending to use the error reporting part of it. Thus, add a (yet another) variant of WaitGroup here that is identical to sync.WaitGroup, but with an extra method. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
The addition of WaitGroup.Go in the standard library has been repeatedly proposed and rejected. See golang/go#18022, golang/go#23538, and golang/go#39863 In summary, the argument for WaitGroup.Go is that it avoids bugs like: go func() { wg.Add(1) defer wg.Done() ... }() where the increment happens after execution (not before) and also (to a lesser degree) because: wg.Go(func() { ... }) is shorter and more readble. The argument against WaitGroup.Go is that the provided function takes no arguments and so inputs and outputs must closed over by the provided function. The most common race bug for goroutines is that the caller forgot to capture the loop iteration variable, so this pattern may make it easier to be accidentally racy. However, that is changing with golang/go#57969. In my experience the probability of race bugs due to the former still outwighs the latter, but I have no concrete evidence to prove it. The existence of errgroup.Group.Go and frequent utility of the method at least proves that this is a workable pattern and the possibility of accidental races do not appear to manifest as frequently as feared. A reason *not* to use errgroup.Group everywhere is that there are many situations where it doesn't make sense for the goroutine to return an error since the error is handled in a different mechanism (e.g., logged and ignored, formatted and printed to the frontend, etc.). While you can use errgroup.Group by always returning nil, the fact that you *can* return nil makes it easy to accidentally return an error when nothing is checking the return of group.Wait. This is not a hypothetical problem, but something that has bitten us in usages that was only using errgroup.Group without intending to use the error reporting part of it. Thus, add a (yet another) variant of WaitGroup here that is identical to sync.WaitGroup, but with an extra method. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
The addition of WaitGroup.Go in the standard library has been repeatedly proposed and rejected. See golang/go#18022, golang/go#23538, and golang/go#39863 In summary, the argument for WaitGroup.Go is that it avoids bugs like: go func() { wg.Add(1) defer wg.Done() ... }() where the increment happens after execution (not before) and also (to a lesser degree) because: wg.Go(func() { ... }) is shorter and more readble. The argument against WaitGroup.Go is that the provided function takes no arguments and so inputs and outputs must closed over by the provided function. The most common race bug for goroutines is that the caller forgot to capture the loop iteration variable, so this pattern may make it easier to be accidentally racy. However, that is changing with golang/go#57969. In my experience the probability of race bugs due to the former still outwighs the latter, but I have no concrete evidence to prove it. The existence of errgroup.Group.Go and frequent utility of the method at least proves that this is a workable pattern and the possibility of accidental races do not appear to manifest as frequently as feared. A reason *not* to use errgroup.Group everywhere is that there are many situations where it doesn't make sense for the goroutine to return an error since the error is handled in a different mechanism (e.g., logged and ignored, formatted and printed to the frontend, etc.). While you can use errgroup.Group by always returning nil, the fact that you *can* return nil makes it easy to accidentally return an error when nothing is checking the return of group.Wait. This is not a hypothetical problem, but something that has bitten us in usages that was only using errgroup.Group without intending to use the error reporting part of it. Thus, add a (yet another) variant of WaitGroup here that is identical to sync.WaitGroup, but with an extra method. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
The addition of WaitGroup.Go in the standard library has been repeatedly proposed and rejected. See golang/go#18022, golang/go#23538, and golang/go#39863 In summary, the argument for WaitGroup.Go is that it avoids bugs like: go func() { wg.Add(1) defer wg.Done() ... }() where the increment happens after execution (not before) and also (to a lesser degree) because: wg.Go(func() { ... }) is shorter and more readble. The argument against WaitGroup.Go is that the provided function takes no arguments and so inputs and outputs must closed over by the provided function. The most common race bug for goroutines is that the caller forgot to capture the loop iteration variable, so this pattern may make it easier to be accidentally racy. However, that is changing with golang/go#57969. In my experience the probability of race bugs due to the former still outwighs the latter, but I have no concrete evidence to prove it. The existence of errgroup.Group.Go and frequent utility of the method at least proves that this is a workable pattern and the possibility of accidental races do not appear to manifest as frequently as feared. A reason *not* to use errgroup.Group everywhere is that there are many situations where it doesn't make sense for the goroutine to return an error since the error is handled in a different mechanism (e.g., logged and ignored, formatted and printed to the frontend, etc.). While you can use errgroup.Group by always returning nil, the fact that you *can* return nil makes it easy to accidentally return an error when nothing is checking the return of group.Wait. This is not a hypothetical problem, but something that has bitten us in usages that was only using errgroup.Group without intending to use the error reporting part of it. Thus, add a (yet another) variant of WaitGroup here that is identical to sync.WaitGroup, but with an extra method. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
The API for WaitGroup is very easy for people to misuse. The documentation for
Add
says:However, it is very common to see this incorrect pattern:
This usage is fundamentally racy and probably does not do what the user wanted. Worse yet, is that it is not detectable by the race detector.
Since the above pattern is common, I propose that we add a method
Go
that essentially does theAdd(1)
and subsequent call toDone
in the correct way. That is:The text was updated successfully, but these errors were encountered: