New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
x/build/internal/sourcecache: increase download size limit to accommodate larger repositories in 2021 #46379
Comments
It seems that the compressed content of that CL is a 55 MB tarball, which exceeds our limit, currently 50 MB: const maxSize = 50 << 20 // talks repo is over 25MB; go source is 7.8MB on 2015-06-15 (Source.) It's probably time we increase that limit. |
Indeed. x/website is only going to get bigger over time. |
I'll send a CL to double the current limit. That should last some years. |
Change https://golang.org/cl/322658 mentions this issue: |
Change https://golang.org/cl/365974 mentions this issue: |
Change https://golang.org/cl/366135 mentions this issue: |
Update on 2021-11-22: x/talks content (74 MB compressed) is being moved into x/website in CL 365135, pushing it over 100 MB. CL 366135 updates the limit for x/website (only) to be 200 MB. |
The build system enforces a limit on the compressed tree tarball size that it's willing to fetch and propagate throughout the rest of the system. If that limit is exceeded, consider it a terminal error and fail the build, since fetching the exact same commit again will not produce a different result. Such situations will always involve some human intervention: • if the limit was exceeded unexpectedly, for example due to unwanted content being accidentally checked in, a different commit will be pushed and tested • if enough time has passed that what used to be deemed as enough no longer holds, the size limit will need to be updated Factor in buildgo.FetchSubrepo into the two places in cmd/coordinator. Its behavior was so trivial (just 2 function calls) that the overhead of creating an additional abstraction overshadowed any small benefits. Fixes golang/go#49707. Updates golang/go#46379. Updates golang/go#11224. Change-Id: I7219da97a888ce621d71a51f828400a11e343bf9 Reviewed-on: https://go-review.googlesource.com/c/build/+/365974 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com>
The x/website repository is unusually large due to having content from x/blog, and as of CL 365135 also from x/talks. Increase its limit to 200 MB, but leave other repositories as they were. Updates golang/go#46379. Change-Id: Ib18766cb5903e602899c1777f1fba7d7cb2a5767 Reviewed-on: https://go-review.googlesource.com/c/build/+/366135 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org>
TryBots are failing on CL 322750 with:
It seems like the underlying error might be coming indirectly from Gerrit at this point.Update: It's our own limit in x/build, see comment below.We should determine if TryBots should be able to handle a large CL (it's adding approximately 70 MB of content from x/blog repo), or otherwise improve the error message to say that a known limit is being exceeded.
CC @golang/release, @rsc.
The text was updated successfully, but these errors were encountered: