Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

all: repeatedly running all.bash in an unchanged tree grows the Go build cache without limit #61484

Open
siebenmann opened this issue Jul 20, 2023 · 7 comments
Open

all: repeatedly running all.bash in an unchanged tree grows the Go build cache without limit #61484

siebenmann opened this issue Jul 20, 2023 · 7 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.23

Comments

@siebenmann
Copy link

What version of Go are you using (go version)?

$ go version
go version devel go1.22-890b96f7ab Thu Jul 20 08:03:42 2023 +0000 linux/amd64

Does this issue reproduce with the latest release?

This issue doesn't reproduce when building Go 1.20.6 from a 'git checkout go1.20.6' in the source tree.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/cks/.cache/go-build'
GOENV='/home/cks/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/cks/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/cks/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/w/620/cks/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/w/620/cks/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='devel go1.22-890b96f7ab Thu Jul 20 08:03:42 2023 +0000'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/w/620/cks/go/src/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3572262321=/tmp/go-build -gno-record-gcc-switches'

What did you do?

git clone https://go.googlesource.com/go
cd go/src
./all.bash
du -hs ~/.cache/go-build
./all.bash
du -hs ~/.cache/go-build
[repeat]

The first all.bash from source will create about 2.3 GB of Go build cache. Subsequent all.bash runs appear to increase this by about 200 Mbytes each time. If you repeatedly build for long enough (if, for example, you're using this as a little stress test on a new system), this will fill up your $HOME with hundreds of GBytes of ~/.cache/go-build. 'go clean -cache' cleans it all away.
@heschi heschi changed the title Repeatedly running all.bash in an unchanged tree grows the Go build cache without limit cmd/go: repeatedly running all.bash in an unchanged tree grows the Go build cache without limit Jul 20, 2023
@heschi
Copy link
Contributor

heschi commented Jul 20, 2023

cc @bcmills @matloob

@heschi heschi added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 20, 2023
@heschi heschi added this to the Go1.22 milestone Jul 20, 2023
@ianlancetaylor
Copy link
Contributor

Just a note that every 24 hours the cache is checked for old entries, and entries that are older than 5 days are deleted. So in the long run there is a limit.

But it does seem odd that running all.bash with the same sources would increase the cache size.

@bcmills
Copy link
Contributor

bcmills commented Jul 20, 2023

@siebenmann, some questions:

  • Does the cache grow when you run make.bash, or only all.bash?
  • Does the output of go tool compile -V=full change between runs?
  • Do you see similar growth if you run go test -short std cmd?

@bcmills
Copy link
Contributor

bcmills commented Jul 20, 2023

Specifically, I wonder if that ~200 MB of data per run is due to caching packages built from within test temp directories. If so, maybe some tests should be updated to build with -trimpath or similar to get the temp directories out of the cache keys.

@bcmills bcmills changed the title cmd/go: repeatedly running all.bash in an unchanged tree grows the Go build cache without limit all: repeatedly running all.bash in an unchanged tree grows the Go build cache without limit Jul 20, 2023
@bcmills
Copy link
Contributor

bcmills commented Jul 20, 2023

(There may be a cmd/go bug here as well — for example, it is possible that this data is in package indexes, which IIRC currently contain absolute paths.)

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jul 20, 2023
@siebenmann
Copy link
Author

This growth doesn't happen with just 'make.bash' (which only seems to use 455 MBytes or so from a clean cache). Running go test -short std cmd repeatedly doesn't seem to cause growth, but I'm not sure I'm doing it right because it fails (when all.bash succeeds). There's no difference between go tool compile -V=full outputs across multiple 'all.bash' runs, despite the build cache growing.

In terms of the cache's growth rate versus periodic trimming, the cache size can reach hundreds of GBytes (I saw over 350 GB) if you're running 'all.bash' in a loop repeatedly.

@bcmills
Copy link
Contributor

bcmills commented Jul 20, 2023

Running go test -short std cmd repeatedly doesn't seem to cause growth, but I'm not sure I'm doing it right because it fails (when all.bash succeeds).

Huh. Could you file a separate issue for the failures you observe from that? (It ought to work.)

@siebenmann siebenmann mentioned this issue Jul 20, 2023
Closed
@seankhliao seankhliao removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Sep 10, 2023
@gopherbot gopherbot modified the milestones: Go1.22, Go1.23 Feb 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.23
Development

No branches or pull requests
6 participants
@siebenmann @ianlancetaylor @bcmills @gopherbot @seankhliao @heschi