I'm several hours into go cache -clean.

That seems surprising. Could you file a separate issue for go cache -clean being slow? 🙂

What should I have done (or better, what should cmd/go do) to prevent runaway cache growth?

Compiling executables tens of thousands of times with tiny variations is a very unusual mode of usage. It doesn't seem entirely unreasonable to expect you to run go cache -clean manually every so often during that experiment...

That said, if there is some portable way we can detect free space, it might be a good idea for the go command to periodically check the ratio of free space to cache usage and clean if it gets too high.

I think this is a dup of #29561.

Definitely unfortunate behavior though. I don't think we can put an upper bound on cache size, but we ought to have a smarter eviction policy that takes cache size and free space into account.

For reference, the current cache eviction policy is hard-coded in cmd/go/internal/cache. We scan for deletable files at most once per 24 hours. We delete files that haven't been used in 5 days.

That seems surprising. Could you file a separate issue for go cache -clean being slow? 🙂

I'm not sure that it is slow. I think the problem is that it had to delete 400gb of smallish files. rm -rf would be similarly slow.

It doesn't seem entirely unreasonable to expect you to run go cache -clean manually every so often during that experiment...

One other sad note, just for posterity. Running go cache -clean concurrently with builds causes those builds to fail. So I'm now having to coordinate pausing and restarting all the other compilation jobs in order to clean the cache occasionally.

@josharian That doesn't seem like good behavior either. What should go clean -cache do with concurrent builds?

• We could have a "clean" lock that would block go build and other commands until go clean -cache finishes.
• go clean -cache could leave files created after it started. This might be hard to do reliably and portably though.

I think that locking scheme is possible to implement using the existing cmd/go/internal/lockedfile API.

go build would obtain a read-lock on the file (os.O_CREATE|os.O_RDONLY)
go clean would obtain a write-lock on the file (os.O_CREATE|os.O_WRONLY)

We don't currently rely on file-locking for the cache, but since this would only prevent a build / clean race perhaps it's not so bad.

I'll file a new issue to discuss, thanks!

#31948