Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/sync/semaphore: Acquire should not wait for context to close when required weight is larger than size #59002

Open
wilkice opened this issue Mar 13, 2023 · 5 comments
Labels
NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Milestone

Comments

@wilkice
Copy link

wilkice commented Mar 13, 2023

What version of Go are you using (go version)?

$ go version
go version go1.20.1 darwin/arm64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/darcy/Library/Caches/go-build"
GOENV="/Users/darcy/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/darcy/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/darcy/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/homebrew/Cellar/go/1.20.1/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/homebrew/Cellar/go/1.20.1/libexec/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.20.1"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/darcy/code/temp/ccc/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/2f/0m8mf47s50x4zfvsm3bxyn4r0000gn/T/go-build3257843051=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

https://go.dev/play/p/hqcrWEMRu-e

What did you expect to see?

This program should return immediately after compare request weight with size. For the request weight, 1000, is larger than size, 8. The request won't be fulfilled at any time.

source code: https://cs.opensource.google/go/x/sync/+/master:semaphore/semaphore.go;l=48-53;drc=43a5402ce75a95522677f77c619865d66b8c57ab

I think source code at line 51 should by removed

<-ctx.Done()

What did you see instead?

Waited 10 second before exit.

➜  ./myprogram  
error is  context deadline exceeded
10.001491166 seconds has passed
@gopherbot gopherbot added this to the Unreleased milestone Mar 13, 2023
@bcmills
Copy link
Contributor

bcmills commented Mar 13, 2023

This program should return immediately after compare request weight with size.

Why?

The documented semantics of Acquire are: “Acquire acquires the semaphore with a weight of n, blocking until resources are available or ctx is done.” Returning early for another reason would add yet another condition there.

Moreover, for callers that want the early-return behavior, it is easy enough to save the original n passed to NewWeighted and do the comparison on the caller side before calling Acquire.

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Mar 13, 2023
@seankhliao seankhliao changed the title x/sync: semaphonre.Acquire should not wait for context to close when required weight is larger than size x/sync/semaphore: Acquire should not wait for context to close when required weight is larger than size Mar 13, 2023
@wilkice
Copy link
Author

wilkice commented Mar 13, 2023

Acquire acquires the semaphore with a weight of n, blocking until resources are available or ctx is done

Yes, I agree this. If request weight is not larger than size, wait for resource or ctx is ok because there is chance for this request to be accepted.

But if the request weight is larger than size, this request won't be accepted at any time. So what's the point of waiting for ctx even when we know it already fails?

if n > s.size {
    // Don't make other Acquire calls block on one that's doomed to fail.
    s.mu.Unlock()
    <-ctx.Done()
    return ctx.Err()
}

@seankhliao
Copy link
Member

fwiw, golang.org/x/sync/rate.Limiter.WaitN, which is similar to a semaphore will error on exceeding max size

https://pkg.go.dev/golang.org/x/time/rate#Limiter.WaitN

@seankhliao seankhliao added NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Mar 19, 2023
@xiaohu-zhang
Copy link

xiaohu-zhang commented Apr 23, 2023

I am very agree with darcy.If I did not set ctx WithTimeout , there will be a Memory leak.
for example code:
https://go.dev/play/p/roy3vs1s4OD
sem.Acquire(ctx, 1000)will never return ,that will get a Memory leak. It is very dangerous

@ibrahimk9000
Copy link

line 51 <-ctx.Done()
when using context.TODO() or context.Background() , it will block forever if weight > size

godoc WorkerPool example with sem.Acquire(ctx, 1) changed to sem.Acquire(ctx, 9) // GOMAXPROCS=8
https://go.dev/play/p/33E7JxU0zyh
https://pkg.go.dev/golang.org/x/sync/semaphore#example-package-WorkerPool

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Projects
None yet
Development

No branches or pull requests

6 participants