-
Notifications
You must be signed in to change notification settings - Fork 17.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
x/build: dragonfly-amd64 builders may need a longer TEST_TIMEOUT_SCALE #45216
Comments
@bcmills is that some configuration option? Let me know if you think I should change something. |
@tuxillo Yes, it allows providing a custom multiplier for the default test timeout. See: Lines 149 to 161 in 189c694
It's useful for cases where a builder has limited CPU/IO/network resources that result in default test timeouts being too aggressive, since it gives tests more time to finish. See golang/build@3689a13 and golang/build@3a0fcf7 for past examples where that configuration was adjusted. |
Ok, I'll submit a patch to build with a timescale of 2, unless you want to set another value? |
This sets the GO_TEST_TIMEOUT_SCALE=2 for dragonfly builders in order to try to solve intermittent test failures. For golang/go#45215. For golang/go#34034. Fixes golang/go#45216.
Change https://golang.org/cl/315329 mentions this issue: |
Change https://golang.org/cl/360735 mentions this issue: |
Updates golang/go#45216 Change-Id: I9e714582c4d8c3c7ce7571359d7f9bbe2c6c9497 Reviewed-on: https://go-review.googlesource.com/c/tools/+/360735 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Robert Findley <rfindley@google.com> gopls-CI: kokoro <noreply+kokoro@google.com> TryBot-Result: Go Bot <gobot@golang.org>
We've moved the builder to a host with a faster disk (on October 18th), it should be a bit better, isn't it? |
From the test logs above, it looks like things improved somewhat on Oct. 18 but then got worse again around Nov. 1. Curiously, all of the failures since Nov. 1 include at least one deadline error in |
I have noticed that the time in the VM we use gets screwed somehow (see a similar case in https://bugs.dragonflybsd.org/issues/3299). We're not sure yet why this happens. I've enabled an ntp client now and it's making a lot of adjustments, which looks suspicious.
In any case, is it performing better right now? It definitely should. |
Updates golang/go#45216 Change-Id: I9e714582c4d8c3c7ce7571359d7f9bbe2c6c9497 Reviewed-on: https://go-review.googlesource.com/c/tools/+/360735 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Robert Findley <rfindley@google.com> gopls-CI: kokoro <noreply+kokoro@google.com> TryBot-Result: Go Bot <gobot@golang.org>
I've noticed frequent
cmd/go
test timeouts on thedragonfly-amd64
builders recently (#45215). Some other tests tend to time out on these builders too (#34034).I notice that the
dragonfly-amd64
builders are reverse builders, so I wonder if the builders are just on the slow side. Perhaps they need aGO_TEST_TIMEOUT_SCALE
to allow the more intensive tests to complete?(See also #44584.)
CC @tuxillo @golang/release
The text was updated successfully, but these errors were encountered: