Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

math/rand: NewSource Method memory overflow #66867

Open
huyouba1 opened this issue Apr 17, 2024 · 2 comments
Open

math/rand: NewSource Method memory overflow #66867

huyouba1 opened this issue Apr 17, 2024 · 2 comments
Labels
WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.

Comments

@huyouba1
Copy link

Go version

go version go1.19.1 darwin/amd64

Output of go env in your module/workspace:

GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/xxx/Library/Caches/go-build"
GOENV="/Users/xxx/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/xxx/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/xxx/go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.19.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/v2/70nz7fqs4bsf1nmqtbk8111h0000gp/T/go-

What did you do?

develop

What did you see happen?

Variable rng memory is not released and continues to grow

Type: inuse_space
Time: Apr 17, 2024 at 10:32am (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 2098.23MB, 93.58% of 2242.26MB total
Dropped 130 nodes (cum <= 11.21MB)
Showing top 10 nodes out of 30
      flat  flat%   sum%        cum   cum%
  848.84MB 37.86% 37.86%   848.84MB 37.86%  math/rand.NewSource (inline)
  377.14MB 16.82% 54.68%   388.64MB 17.33%  github.com/IBM/sarama.(*asyncProducer).newPartitionProducer
  345.61MB 15.41% 70.09%  1221.45MB 54.47%  github.com/IBM/sarama.(*asyncProducer).newTopicProducer
  298.12MB 13.30% 83.38%   298.12MB 13.30%  runtime.malg
   56.51MB  2.52% 85.91%    61.51MB  2.74%  github.com/IBM/sarama.(*brokerProducer).run
   49.50MB  2.21% 88.11%    49.50MB  2.21%  github.com/IBM/sarama.(*asyncProducer).newBrokerProducer
      42MB  1.87% 89.99%    74.51MB  3.32%  github.com/IBM/sarama.newAsyncProducer
      27MB  1.20% 91.19%    76.51MB  3.41%  github.com/IBM/sarama.(*asyncProducer).getBrokerProducer
      27MB  1.20% 92.39%   415.64MB 18.54%  github.com/IBM/sarama.(*topicProducer).dispatch
   26.50MB  1.18% 93.58%    26.50MB  1.18%  github.com/IBM/sarama.newTransactionManager
(pprof) list NewSource
Total: 2.19GB
ROUTINE ======================== math/rand.NewSource in /usr/local/go/src/math/rand/rand.go
  848.84MB   848.84MB (flat, cum) 37.86% of Total
         .          .     39:
         .          .     40:// NewSource returns a new pseudo-random Source seeded with the given value.
         .          .     41:// Unlike the default Source used by top-level functions, this source is not
         .          .     42:// safe for concurrent use by multiple goroutines.
         .          .     43:func NewSource(seed int64) Source {
  848.84MB   848.84MB     44:   var rng rngSource
         .          .     45:   rng.Seed(seed)
         .          .     46:   return &rng
         .          .     47:}
         .          .     48:
         .          .     49:// A Rand is a source of random numbers.
(pprof)

What did you expect to see?

none
@seankhliao
Copy link
Member

do you have a reproducer for this issue?
does this reproduce with the latest release?
there's now a math/rand/v2 package, does it reproduce with that?

@robpike
Copy link
Contributor

robpike commented Apr 17, 2024

It looks to me like you have a loop that is calling NewSource and storing the result in some other structure that is also not being freed. The traceback implies it's the caller of NewSource that's the problem, not NewSource itself.

Also, as @seankhliao wrote, there is a rand/v2. It was created because the old rand's Source is far too large. The new one should reduce the footprint enormously. However, I still believe the real problem lies in the code calling NewSource itself.

@seankhliao seankhliao added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Apr 17, 2024
@seankhliao seankhliao changed the title math/rand NewSource Method memory overflow math/rand: NewSource Method memory overflow Apr 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@robpike @seankhliao @huyouba1