-
Notifications
You must be signed in to change notification settings - Fork 18k
proposal: testing/synctest: create bubbles with Start rather than Run #73062
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Thanks for the new proposal! Out of curiosity I tried to rewrite the example you included for each of the alternatives you presented, and I figured I might as well share them here to save others the trouble. Original, just as written in the current text of the proposal: func TestAfterFunc(t *testing.T) {
synctest.Start()
ctx, cancel := context.WithCancel(context.Background())
funcCalled := false
context.AfterFunc(ctx, func() {
funcCalled = true
})
cancel()
synctest.Wait()
if !funcCalled {
t.Fatalf("AfterFunc function not called after context is canceled")
}
} Make bubbles a feature of the testing package: func TestAfterFunc(t *testing.T) {
bubble := t.Isolate()
ctx, cancel := context.WithCancel(context.Background())
funcCalled := false
context.AfterFunc(ctx, func() {
funcCalled = true
})
cancel()
bubble.Wait()
if !funcCalled {
t.Fatalf("AfterFunc function not called after context is canceled")
}
} Make bubbles a feature of the testing package, but keep the Run function: func TestAfterFunc(t *testing.T) {
t.RunIsolated("bubbled", func (t *testing.T, bubble *testing.Bubble) {
ctx, cancel := context.WithCancel(context.Background())
funcCalled := false
context.AfterFunc(ctx, func() {
funcCalled = true
})
cancel()
bubble.Wait()
if !funcCalled {
t.Fatalf("AfterFunc function not called after context is canceled")
}
})
} Same as one of the above, but make the func TestAfterFunc(t *testing.T) {
// Ignoring the return value here because it doesn't seem to
// actually be useful. Was the intention to change this to not return a
// value for this variant?
t.Isolate()
ctx, cancel := context.WithCancel(context.Background())
funcCalled := false
context.AfterFunc(ctx, func() {
funcCalled = true
})
cancel()
t.Sync()
if !funcCalled {
t.Fatalf("AfterFunc function not called after context is canceled")
}
} func TestAfterFunc(t *testing.T) {
// NOTE: I assume the intention was to remove the bubble arg
// here because otherwise it would be unused in this example.
t.RunIsolated(func (t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
funcCalled := false
context.AfterFunc(ctx, func() {
funcCalled = true
})
cancel()
t.Sync()
if !funcCalled {
t.Fatalf("AfterFunc function not called after context is canceled")
}
})
} Keep the synctest package, but allow it to create a bubble-aware testing.T/B/F: func TestAfterFunc(t *testing.T) {
synctest.Run(func (t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
funcCalled := false
context.AfterFunc(ctx, func() {
funcCalled = true
})
cancel()
synctest.Wait()
if !funcCalled {
t.Fatalf("AfterFunc function not called after context is canceled")
}
})
} I assume the |
On balance, I think of all of these alternatives I lightly prefer the one you labeled "Make bubbles a feature of the testing package":
This is not a strong opinion, though. I think exactly what you proposed would be okay too. |
Yes, this is one of the things I've struggled with on making Run a testing.T method: Either it starts a subtest (but what if you don't want a named subtest?) or it doesn't (but its weird for t.Run to start a subtest and t.RunIsolated not to). Or you have one method that doesn't start a subtest and one that does, but now you've got twice the API surface for not much gain. On the balance, making the create-a-bubble operation move the current goroutine into a bubble seems to let us have the most orthogonal API. |
I like the idea of My proposed implementation of // Isolate signals that this test is to be run inside an isolated "bubble".
// The test function will first be run as normal. When this function is called
// for the first time, the test function will exit immediately (via runtime.Goexit()),
// and then the test function will be called again from inside a new bubbled
// goroutine. Once inside the bubbled goroutine, the second call to this function will
// be a no-op.
//
// Since any code placed before the Isolate call will be run twice, this should ideally
// be the first function called within the test.
func (t *T) Isolate() How an example test would run: func TestSleep(t *testing.T) {
fmt.Println("hello")
t.Isolate()
a := time.Now()
time.Sleep(24 * time.Hour)
b := time.Now()
if b.Sub(a) != 24 * time.Hour {
t.Error("sleep not exact")
}
}
Curious to your thoughts! |
Naming suggestion to avoid confusion -- instead of so func TestAfterFunc(t *testing.T) {
// NOTE: I assume the intention was to remove the bubble arg
// here because otherwise it would be unused in this example.
t.Isolate(func (t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
funcCalled := false
context.AfterFunc(ctx, func() {
funcCalled = true
})
cancel()
t.Sync()
if !funcCalled {
t.Fatalf("AfterFunc function not called after context is canceled")
}
})
} |
I'm tempted to impute fork/exec semantics on this, where |
I've seen this technique (restart the test after calling a function) used to fairly good effect. It's a quite subtle behavior, however, and the edge cases can be surprising and difficult to debug. For example:
I think under the proposed implementation, this sleeps for one real second, restarts the test, sleeps for one virtual second, and then completes. It's even more confusing if the test starts a new goroutine before calling The solution to avoid confusion is to not do that: Always call Another problem is that the test-restarting approach doesn't compose well. On the balance, I think moving the current goroutine into a bubble has fewer edge cases than restarting tests midway. |
This is a revised version of #67434. I'm creating it as a separate proposal to make it easier to discuss the proposed changes.
In Go 1.24, we added the testing/synctest package as an experiment, enabled by setting GOEXPERIMENT=synctest. See #67434 and https://go.dev/blog/synctest for more details. So far, response has been reasonable favorable.
A few issues have been consistently raised as pain points in the existing API, notably:
When a bubble contains a long-lived background polling goroutine, such as one reading from a time.Ticker, the bubble will advance time indefinitely. This manifests as a seemingly hung test. (The test is actually running indefinitely rather than hung, but this is a distinction without much difference.) (Example: proposal: testing/synctest: new package for testing concurrent code #67434 (comment))
T.Cleanup
functions registered inside a bubble run outside the bubble. This is usually not what you want, especially when a cleanup function is used to shut down bubbled goroutines. (Example: proposal: testing/synctest: new package for testing concurrent code #67434 (comment))T.Context
returns a context created outside the bubble. Again, this is usually not what you want since the context's Done channel will not be part of the bubble. (Example: proposal: testing/synctest: new package for testing concurrent code #67434 (comment))The first concern (long-lived background pollers) may be addressed by advancing time within the bubble only so long as the bubble's ancestor goroutine exists. After it exits, we will stop advancing time. If any goroutines are still alive and sleeping or reading from a ticker, we will report a deadlock and panic. This requires no changes to the existing API.
The second two concerns (
T.Cleanup
andT.Context
) point at a need to maketesting.T
(andB
andF
) bubble-aware. There are a variety of possible approaches to doing this, but so far as I can tell they all require changes to thetesting/synctest
API. This is one of them.This proposal changes the
Run
function to aStart
function.The
Run
function has some nice properties that we lose: Passing a function toRun
makes the bubble lifetime very clear in a way thatStart
does not, andRun
provides a good place to produce a panic when a bubble deadlocks. (In the revised proposal withStart
, a deadlocked bubble is now a fatal panic.)However,
Start
has the advantage of allowing us to move the point of bubble cleanup into thetesting
package. I additionally propose that:T.Context
is called within a bubble, it returns a context with aDone
channel that belongs to the bubble.T.Cleanup
is called within a bubble, the cleanup function is run inside the bubble.synctest.Start
thetesting
package arranges to wait for goroutines in the bubble to exit before finishing the test.As an example, converting one of the examples from https://go.dev/blog/synctest:
Migration
The testing/synctest package is experimental, so we are free to make incompatible changes to it. However, it would be good to avoid causing problems for anyone kind enough to try out the package in its current experimental state.
I propose that we add
synctest.Start
and leavesynctest.Run
in place for now. If we promotesynctest
out of its experimental status, theRun
function will continue exist only whenGOEXPERIMENT=synctest
is enabled. Ifsynctest
continues to exist in Go 1.26 (as an experiment or otherwise), we will remove theRun
function in that version.Alternatives
These are some alternative APIs that satisfy the same goals as this proposal. I don't have a strong preference for any of them; I chose the above proposal because it keeps the synctest API small (still only two functions) and contained within a single package.
Make bubbles a feature of the testing package:
Make bubbles a feature of the testing package, but keep the Run function:
Same as one of the above, but make the
Wait
operation a method oftesting.TB
:Keep the synctest package, but allow it to create a bubble-aware testing.T/B/F:
The text was updated successfully, but these errors were encountered: