/CC: @jadekler (apropos the text we are working on)

I don't think it makes too much sense to do this for the stdlib in the approx 60 cases where context.Background() appears in non test code. These are almost all used where the exposed API doesn't take a context

• there is no ambiguity in which context to use, because none are available and the user explicitly asked for a blocking function
• it can't be "not yet available" because of the compatibility promise

CC @Sajmani

it can't be "not yet available" because of the compatibility promise

Maybe? But for many of those cases we can deprecate the old API and provide a parallel new API that does accept a context. (Consider http.NewRequest vs. http.NewRequestWithContext.)

So in that sense it really is a TODO — either the user needs to call something else to fill in the context (such as (*net/http.Request).WithContext), or the API needs to be augmented and/or deprecated.

FWIW this seems to me like good way to lead by example wrt how context.TODO should be used, an easy (if a bit toilsome) change, and no downsides.

I like this idea, but I'd want to know how this will turn into actionable guidance for the user. For example, suppose we had a tool to notice when a function that has a context parameter transitively calls context.TODO(). In this case, the user likely wants to "plumb" its context down to the context.TODO(). As @bcmills points out, this probably means switching some of the intervening function calls to the versions that take a context parameter. This seems like a tractable problem (and IIRC people have also tried creating automatic refactoring tools to do context plumbing), but what I want to avoid is confusion for users reading this code or getting notice about transitively-reachable context.TODO() without guidance on what they should do instead.

I have been mulling a smaller proposal to add a section to the Godoc entry for package context to explain the retrofitting techniques, what context.TODO means in the context of retrofitting, etc. As a high-level sketch, do you think that would satisfy the desire for actionable guidance?

Here is what I have come up (as a sketch) for the documentation for retrofitting:

// TODO returns a non-nil, empty Context. Code should use context.TODO when
// it's unclear which Context to use or it is not yet available (because the
// surrounding function has not yet been extended to accept a Context
// parameter).
//
// Retrofitting Context Support onto APIs
//
// Retrofitting means adding a context to an API that otherwise did not
// originally have support for it. Suppose we want to retrofit a function
// called Load:
//
//      // Load reads data from the network.
//      func Load() ([]byte, error) {
//              // ...
//      }
//
// Updating a function usually requires updating its callers. This means a new
// API should be introduced that supports it. Conventionally it is the original
// name with the word "Context" appended to its end.
//
//      // LoadContext reads data from the network subject to context
//      // cancellation.
//      func LoadContext(ctx context.Context) ([]byte, error) {
//              // ... use ctx ...
//      }
//
// Afterwards the old Load function should be updated to call the new
// LoadContext function. When Load does, it should pass context.TODO()
// for the context argument.
//
//      // Load reads data from the network.
//      func Load() ([]byte, error) {
//              return LoadContext(context.TODO())
//      }
//
// Callers of Load can incrementally be upgraded to LoadContext. If the callers
// do not have access to a context already, they should use context.TODO(). The
// retrofitting process continues until the APIs reach a function with a
// suitably-scoped context not derived from context.TODO().
//
// Maintainers may optionally deprecate the old APIs that they have
// retrofitted.
//
//      // Load reads data from the network.
//      //
//      // Deprecated: Please use LoadContext instead and provide a context.
//      func Load() ([]byte, error) {
//              return LoadContext(context.TODO())
//      }
func TODO() Context {
        return todo
}

I put modest work into the text above and am not particularly attached to its exact phrasing.

Callers of Load can incrementally be upgraded to LoadContext.

See also #32816.

Many of the uses of context.Background could in theory change to context.TODO, but we are never going to "DO" those changes. It will make it look like there's work we can do that we can't. (For example Dial vs DialContext.) I'm not sure I see what benefit there is in making the code look like there is work to do when there isn't. I believe that all these routines are documented as using the background context.

Maybe it would make sense to proceed with clearer docs instead?

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

Discussed this with @mattproud a bit last week.
It sounds like the main thing we need to do is to document clearly which routines in the standard library use context.Background internally (and will always do that).

There may be room for a doc explaining how to use TODO in a migration adding contexts, but that might be a separate doc rather than a long doc comment in package context.

At this point, what's left doesn't need to go through the proposal process. It's fine to just send CLs for the comments.
The kind of comment I imagine would be:

// Dial uses context.Background internally; to specify the context, use DialContext.

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

Change https://golang.org/cl/296152 mentions this issue: docs: clarify when APIs use context.Background.

I had been thinking a lot about #44143 (comment), and it was referenced in another context (no pun intended). My thoughts are roughly this:

Enclosing context.Background in non-program roots should be avoided (especially for retrofitting), because it insinuates that casual use of context.Background in non-roots is permissible, especially in library code. This remains significant enough problem in code review that we even codified guidance about it in local style guide documentation. I want to avoid the appeal-to-authority from because-the-standard-library-does-it-I-can-do-it-even-though-I-probably-misdesigned-my-library-in-other-ways-alluded-to-below problem. context.TODO evokes a stop-and-think response.

While the implementation of the two contexts is the same, they do convey different information. context.Background is permanent fixture of program roots, whereas context.TODO decisively indicates that something is a transitionary state to be made better later (when is another question[0]). Rather what we should prefer is having APIs let the user specify the context.Context to be used, for many of the same reasons as this article alludes to.

Even in the cases of RPC or HTTP request dispatching, I am still not 100 percent sure that I would consider those roots but rather branches of a central program root context. These branches often have request-scoped data and very well-defined lifetimes. And it would be wrong to attach request-scoped data to the root context (that itself is server/program lifetime scoped). This is a whole other topic I have been mulling. There are a number of interesting things the single-root approach enables: full program tracing, universal credential passing, etc.

I've been failing to come up with a good litmus of when context.Background is really safe to use in libraries as non-roots. Brainstorm:

1. What's derived therefrom is always interruptible/cancelable by other means and exposed API? This entails no goroutine leaking.

2. Callers of that type to express cancellation or deadline of their own.

Classically there has been a risk of this going wrong in network resolution code like request balancers (e.g., gRPC) or name resolution where these libraries perform operations in the request path using an encapsulated background context. It's an error-prone pattern for experts.

Then we have the question of whether arbitrary cancellation is necessarily safe and how APIs can be designed to either handle cleanup gracefully with it or when another mechanism through formal API is advisable. @jadekler and I had been in discussion around this topic as a potential new article or library authorship exercise to help explain how to do this safely.

[0] — I was refereeing a set of ideas around "when" with respect to the compatibility guarantee. For instance, perhaps as a "Go 2" proposal, we replace the non-injectable context APIs in the standard library with injectable ones (assuming context still lives in its current form). Or potentially we could deprecate the non-injectable forms before "Go 2" and nudge folks toward the injectable ones with the "Deprecated: " documentation protocol. As you can see, I did not want to cloud an otherwise large proposal with something like this that could turn it into a kitchen sink of a proposal.

Small nudge: do you think https://go-review.googlesource.com/c/go/+/296152 will be alright for the raw documentation part? I’d like to get that squared away before looking at next steps.

Change https://golang.org/cl/301069 mentions this issue: net/http: revert change to generated file from CL 296152