Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: spec: error handler operator ?() #72149

Open
4 tasks done
smyrman opened this issue Mar 6, 2025 · 8 comments
Open
4 tasks done

proposal: spec: error handler operator ?() #72149

smyrman opened this issue Mar 6, 2025 · 8 comments
Labels
error-handling Language & library change proposals that are about error handling. LanguageChange Suggested changes to the Go language LanguageChangeReview Discussed by language change review committee Proposal
Milestone

Comments

@smyrman
Copy link

smyrman commented Mar 6, 2025

This should not be considered a formal Go proposal yet, but the pre-work needed in order to create one. When all requirements are in place, this may transition into a proposal.

Contribution guide:

  • Discuss changes first; either, either on the Go issue, or by raising an issue at the GitHub repo.
  • Commits should be atomic; rebase your commits to match this.
  • Examples with third-party dependencies should get it's own go.mod file.
  • Include both working Go 1.24 code (with .go suffix), and a variant using the proposed ? syntax (with .go2 suffix). -
  • Note that only files that are affected by the proposal syntax, needs a .go2 file.

Contributions that may be of particular interest for now:

  • Contributions demonstrating how this change would help improve application code.
  • Pointing out potential issues.

Links:

Remaining content is aligned with the issue text.

Go Programming Experience

(Contributors may add their own replies)

@smyrman: Experienced

Other Languages Experience

(Contributors may add their own replies)

@smyrman: C++, C, Python

Related Idea

  • Has this idea, or one like it, been proposed before?
  • Does this affect error handling?
  • Is this about generics?
  • Is this change backward compatible? Breaking the Go 1 compatibility guarantee is a large cost and requires a large benefit

Has this idea, or one like it, been proposed before?

The proposal is inspired by Go discussion #71460. Compared to the discussed proposal, this is similar in syntax, but different in semantics.

Key differences to #71460:

  • Instead of the proposed syntax ?{...}, we use syntax ?(...).
  • Instead of acting as a control statement (like if), ? in the proposal acts more like a normal function call.
  • This proposal allows usage within a struct and chain statements.
  • Instead of allowing N return arguments, this proposal allows a maximum of two return arguments.
  • The proposal is paired with a standard library addition to make the language change useful.

Key similarities to #71460:

  • Both proposals use the ? character.
  • Both proposals only aim at handling error types (not bool or other return types).

Semantically, this proposal is somewhat similar to try-catch proposal, but simpler. The syntax and ergonomics are different.

Does this affect error handling?

Yes

This proposal includes:

  1. An addition to the errors standard library package.
  2. A new syntax for handling errors.

Is this about generics?

It's not about generics, but the proto-type is using generics for it's implementation.

Cases

Before discussing the proposal, we will demonstrate a few use-cases that could benefit from it. The cases will be relatively simple. Real use-cases may be more complex, and could therefore expect to result in saving more lines.

Return directly

The direct return of an error is a commonly used case for error handling when adding additional context is not necessary.

Old syntax:

pipeline, err := A()
if err != nil {
	return err
}
pipeline, err = pipeline.B()
if err != nil {
	return err
}

New syntax:

pipeline := A()?.B()?

Return wrapped error

To wrap an error before return is a commonly used case for error handling when adding additional context is useful.

Old syntax:

pipeline, err := A()
if err != nil {
	return fmt.Errorf("a: %w", err)
}
pipeline = pipeline.B()
if err != nil {
	return fmt.Errorf("a: %w (pipeline ID: %s)", err, id)
}

New Syntax:

pipeline :=
	A() ?(errors.Wrap("a: %w")).
	B() ?(errors.Wrap("b: %[2]w (pipeline ID: %[1]s)", id))

Collect errors

The case for collecting errors is likely not common in library code. However, it is likely useful for application code. Possible use-cases include form validation or JSON APIs.

Old syntax:

func ParseMyStruct(in transportModel) (BusinessModel, error) {
	var errs []error
	a, err := ParseA(in.A)
	if err != nil {
		errs = append(fmt.Errorf("a: %w", err))
	}
	b, err := ParseB(in.A)
	if err != nil {
		errs = append(fmt.Errorf("b: %w", err))
	}
	if err := errors.Join(errs...); err != nil {
		return BusinessModel{}, err
	}

	return BusinessModel{
		A: a,
		B: b,
	}, nil
}

New Syntax:

func ParseMyStruct(in transportModel) (BusinessModel, error) {
	var c errors.Collector
	out := BusinessModel{
		A: ParseA(in.A) ?(errors.Wrap("a: %w"), c.Collect),
		B: ParseB(in.B) ?(errors.Wrap("b: %w"), c.Collect),
	}
	if err := c.Err(); err != nil {
		return BusinessModel{}, err
	}
	return out, nil
}

Custom error wrapping

Custom error type:

type PathError struct{
	Path string
	Err  error
}

func (err PathError) Error() string {
	return fmt.Sprintf("%s: %v",err.Path, err.Err)
}

Old syntax:

func ParseMyStruct(in transportModel) (BusinessModel, error) {
	var errs []error
	a, err := ParseA(in.A)
	if err != nil {
		errs = append(PathError{Path:"a", Err: err))
	}
	b, err := ParseB(in.A)
	if err != nil {
		errs =  append(PathError{Path:"b", Err: err))
	}
	if err := errors.Join(errs...); err != nil {
		return BusinessModel{}, err
	}

	return BusinessModel{
		A: a,
		B: b,
	}, nil
}

New Syntax (inline handler):

func ParseMyStruct(in transportModel) (BusinessModel, error) {
	var errs []error
	out := BusinessModel{
		A: ParseA(in.A) ?(func(err error) error{
			errs = append(PathError{Path:"a", Err: err))
		}),
		B: ParseB(in.B) ?(func(err error) error{
			errs = append(PathError{Path:"b", Err: err))
		}),
    }
    if err := errors.Join(errs...) {
     		return BusinessModel{}, err
     }
     return out, nil
 }

Proposal

The proposal has two parts:

  • An addition to the Go syntax, using ?() /? to catch errors.
  • Helper functions in the errors package.

The proposal follows the principal of the now implemented range-over-func proposal in making sure that the solution can be described as valid Go code using the current language syntax. As of the time of writing, this is the syntax of Go 1.24.

The ?/?() syntax can be used to move handling of errors from the left of the expression to the right. The default handler (no parenthesis), is to return on error. When parenthesis are provided, errors pass though handlers of format func(error) error. If any handler return nill, the code continuous along the happy path. If the final handler returns an error, the function with the ? syntax returns.

It's not yet clear if the ? syntax should be allowed inside functions that does not return an error. If it's allowed, the suggestion is that the ? syntax would result in a panic. See options for more details.

The standard library changes involve adding handlers for the most common cases for error handling.

Standard library changes

The following exposed additions to the standard library errors package is suggested:

// Wrap returns an error handler that returns:
//
//	fmt.Errorf(format, slices.Concat(args, []error{err})...)
func Wrap(format string, args ... any) func(error) error {
	return func(error) error {
		nextArgs := make([]any, 0, len(args)+1)
		nextArgs = append(nextArgs, args...)
		nextArgs = append(nextArgs, err)
		return fmt.Errorf(format, nextArgs...)
	}
}

 // Collector expose an error handler function [Collect] for collecting
 // errors into a slice. After the collection is complete, A joined error
 // can be retrieved from [Err].
type Collector struct {
	errs []error
}

// Collect is an error handler that appends err to c.
func (c *Collector) Collect(err error) error {
	if err != nil {
		c.errs = append(c.errs, err)
	}
	return nil
}

// Err returns an joined
func (c *Collector) Err() error {
	return Join(c.errs...)
}

Language Spec Changes

No response

Informal Change

The proposal introduce a new ? operator, which can be used after calls to functions that has any of the following signatures:

func f1(...) error             // One return parameter, which must be an error
func f2[T any](...) (T, error) // Two return parameters, where the last one is an error

The syntax of ? is similar to that of a function call, except the parenthesis () are optional. That is ? and ?() are equivalent. The signature of the operator can be described as:

func ?(handlers ...func(error) error)

When using the ? syntax, the last return parameter of the function is passed to the ? operator to the right, instead of to the left as normal.

func F(...) (..., error) {
	f1()?              // One return parameter without handlers; equivalent to ?()
	f1()?(h1,h2..)     // One return parameter with handlers
	v := f2()?(h1,...) // Two return parameters with
	...
}

The processing rules for error handlers is as follows:

If the ? operator receives a nil error value, execution continues along the "happy path."

If the ? operator receives an error, the error is passed to each handler in order. The output from each handler becomes the input to the next, as long as the output is not nil. If any handler return nil, the handler chain is aborted, and execution continues along the "happy path."

If after all handlers are called, the final return value is an error, then the flow of the current statement is aborted similar to how a panic works. If ? is used within a function where the final return statement is an error, then this panic is recovered and the error value is populated with that error value and the function returns at once.

Is this change backward compatible?

Yes

This work leans on the work done for %71460, that highlights that the ? operator is invalid to use in any existing code. Thus it's expected that no existing code will be able to break due to the introduction of the new syntax.

Orthogonality: How does this change interact or overlap with existing features?

No response

Would this change make Go easier or harder to learn, and why?

Any addition to the Go syntax, including this one, will make it harder to learn Go. However, people coming from an exception handling paradigm may find the new syntax less intrusive then the explicit return.

Cost Description

The highest cost of this proposal is likely that there will now be multiple patterns for handling errors. There could be discrepancies and disagreement between different projects about which style to use.

Changes to Go ToolChain

vet, gopls, gofmt

Performance Costs

No response

Prototype

The proto-type code is found in the pre-work repo.

Following the example of range-over-func, the implementation of the ? semantics is not magic. A tool could be written to generate go code that rewrites the ? syntax to valid go 1.24 syntax.

With proposed syntax:

func AB() (Pipeline, error) {
	id := "test"
	result :=
		A() ?(errors.Wrap("a: %w")).
		B() ?(errors.Wrap("b: %[2]w (pipeline ID: %[1]s)", id))
	return result, nil
}

Can be written using the proto-type library as:

func AB() (_ Pipeline, _err error) {
	defer	errors.Catch(&_err) // Added to the top of all function bodies that contain a `?` operator.

	id := "test"
	result :=
		xerrors.Must2(A())(xerrors.Wrap("a: %w")).                             // function syntax for ?
		xerrors.Must2(B())(xerrors.Wrap("b: %[2]w (pipeline ID: %[1]s)", id))  // function syntax for ?
	return result, nil
}

We defined the following functions in the xerrors package for our proto-type. This is proto-type code only. The final implementation will likely be handled by the compiler directly:

package xerrors

type mustError struct {
	error
}

func (err mustError) Unwrap() error {
	return err.error
}

// Catch recovers from panics raised by Must or Must2 error handler returns
// only. Other panics are passed through. The error from Must or Must2 is
// passed through all handlers, if any. If the error is not set to nil by
// any of the handlers, then target will be set with the final error value.
// If target is nill, and the final error is not nil, Catch will panic instead.
//
// Likely not exposed in the final implementation. The final implementation may
// or may not use panics for it's control flow.
func Catch(target *error, handlers ...func(error) error) {
	r := recover()
	switch rt := r.(type) {
	case nil:
	case mustError:
		nextErr := rt.error
		for _, h := range handlers {
			nextErr = h(nextErr)
			if nextErr == nil {
				return
			}
		}
		if target == nil {
			panic(nextErr)
		}
		*target = nextErr
	default:
		panic(r)
	}
}

// Must implements '?' for wrapping functions with one return parameter when
// combined with a deferred Catch. Handlers are called in order given the input
// from the previous handler. If a handler returns nil, then that value is
// returned immediately. If the final handler returns an error, we raise a panic
// that is recovered by Catch. If there are no handlers, then Must will panic
// with the original error if it is not nil.
//
// Likely not exposed in the final implementation. The final implementation may
// or may not use panics for it's control flow.
func Must(err error) func(handlers ...func(error) error) {
	if err == nil {
		return func(_ ...func(error) error) {}
	}
	return func(handlers ...func(error) error) {
		for _, h := range handlers {
			err = h(err)
			if err == nil {
				break
			}
		}
		if err != nil {
			panic(mustError{error: err})
		}
	}
}

// Must2 implements '?' semantics for wrapping functions with two return
// parameter when combined with Catch. Handlers hare called in order given the
// input from the previous handler. If a handler returns nil, then that value is
// returned immediately. If the final handler returns an error, we raise a panic
// that is recovered by Catch. If there are no handlers, then Must2 will panic
// with the original error if it is not nil.
//
// Likely not exposed in the final implementation. The final implementation may
// or may not use panics for it's control flow.
func Must2[T any](v T, err error) func(handlers ...func(error) error) T {
	if err == nil {
		return func(_ ...func(error) error) T {
			return v
		}
	}
	return func(handlers ...func(error) error) T {
		for _, h := range handlers {
			err = h(err)
			if err == nil {
				break
			}
		}
		if err != nil {
			panic(mustError{error: err})
		}
		return v
	}
}

Options

Options could be applied to change the proposal in various ways.

1: Disallow usage within non-error functions

We could choose to disallow the ? syntax inside functions that doesn't return errors. This included the main function.

This would ensure that the ? syntax can not lead to panics.

2: Allow explicit catch

An option could be to expose the Catch function from the proto-type, and allow supplying a set of error handlers that run on all errors.

When an explicit Catch is added, then an implicit Catch is not added.

If the Catch is called with a nil pointer, then any error that isn't fully handled (replaced by nil), results in a panic.

3: Chain via ? syntax

Use syntax ? handler1 ? handler2 as shown in this comment by @733amir.

Why not...

Why not allow more than two return values?

a, b, err := A()
if err != nil {
	return err
}
a, b := A()?  // Not allowed

Most functions that return an error, return either a single parameter, or two parameters. So it wouldn't be many cases where it's useful. It's also assumed that error handling syntax is mostly useful if it allows to continue the flow of our programs. That is, we allow are allowed to chain functions A()?.B()?, or assign to struct fields from functions that return errors. Cases with two or more return values typically can not be chained.

Allowing for more return values risks complicating the implementation, and is likely offer little value in return.

Why require the final return parameter to be an error?

a := os.Getenv("VARIABLE")? // not allowed
a := os.Getenv("VARIABLE")?(func(bool) error, ...func(error) error) // not allowed
a, bc := strings.Cut("a.b.c", ".")? // not allowed

If we allowed for other return values for the naked syntax, it's not clear what the error return value should be.

If we allow for explicit handlers, then we need a conversion from bool to error before we can pass it to the handlers. Thus the argument list changes.

@smyrman smyrman added LanguageChange Suggested changes to the Go language LanguageChangeReview Discussed by language change review committee Proposal labels Mar 6, 2025
@gopherbot gopherbot added this to the Proposal milestone Mar 6, 2025
@ianlancetaylor ianlancetaylor added the error-handling Language & library change proposals that are about error handling. label Mar 6, 2025
@smyrman
Copy link
Author

smyrman commented Mar 7, 2025

Edit: copied in cases from pre-work.

smyrman added a commit to smyrman/error-handling-proposal that referenced this issue Mar 10, 2025

Verified

This commit was signed with the committer’s verified signature.
smyrman Sindre Røkenes Myren
@smyrman
Copy link
Author

smyrman commented Mar 10, 2025

Edit2: update description according to this diff.

  • Improve motivation / language.
  • Fix indenting and syntax bugs in examples.
  • Move pre-work disclaimer to the top.

@apparentlymart
Copy link

Hi @smyrman! Thanks for working on this proposal.

As you noted in the text, this proposal seems to belong to the genre of error handling proposals that's similar to the try function proposal, where error handling changes from being a statement-oriented affair to being an expression-oriented affair, and in particular allows an expression in the middle of a statement to cause the effect of a return statement without there being such a statement.

While that isn't necessarily wrong, it was presented as a repeated point of disagreement in the original proposal:

(I tried my best to include both the arguments against and the responses to them in the above, but that thread is very long so I apologize if I missed anything.)

This question of whether it's desirable for expressions to be able to cause the effect of a return statement seems to arise in most other error-handling proposals too, and so I expect you will get similar feedback on this proposal too. Given that, it might help to include some text in your proposal that preemptively argues either that this particular formulation avoids some/all of the drawbacks previously raised, or that expressions that can cause the effect of a return are the better tradeoff overall despite these objections.

I don't intend this comment as an argument for or against this question or against this proposal. Instead, I'm making this suggestion in the hope that it will lead to a more productive discussion on the unique merits of this particular proposal, rather than rehashing the same discussion points as earlier proposals.

@smyrman
Copy link
Author

smyrman commented Mar 10, 2025

Thanks for the extensive list @apparentlymart .

I will need to go through those systematically.

This proposal is really written to handle the most important use cases for #71203 in an expression based solution. The importance of making it expression based, being to allow usage within chained API calls and inline struct assignments.

I haven't done much work yet comparing it to the try proposal, but I agree with comment #32437 (comment); the rust-like ? syntax is more clear.

@smyrman
Copy link
Author

smyrman commented Mar 11, 2025

PS! Other people than me have come up with more or less the same idea in previous comments either on #71203 or on the associated discussion:

As far as I can tell, they came up with it independently.

Both links include discussions.

@smyrman
Copy link
Author

smyrman commented Mar 11, 2025

Looking at #32437 (comment) first:

I actually really like this proposal. However, I do have one criticism. The exit point of functions in Go have always been marked by a return. Panics are also exit points, however those are catastrophic errors that are typically not meant to ever be encountered.

Making an exit point of a function that isn't a return, and is meant to be commonplace, may lead to much less readable code.

I agree to this comment in the case of the try proposal. Let's recheck how that would look with the ?() syntax.

Current code

func CopyFile(src, dst string) error {
	r, err := os.Open(src)
	if err != nil {
		return fmt.Errorf("copy %s %s: %v", src, dst, err)
	}
	defer r.Close()

	w, err := os.Create(dst)
	if err != nil {
		return fmt.Errorf("copy %s %s: %v", src, dst, err)
	}

	if _, err := io.Copy(w, r); err != nil {
		w.Close()
		os.Remove(dst)
		return fmt.Errorf("copy %s %s: %v", src, dst, err)
	}

	if err := w.Close(); err != nil {
		os.Remove(dst)
		return fmt.Errorf("copy %s %s: %v", src, dst, err)
	}
}

Code with ?()

func CopyFile(src, dst string) error {
	r := os.Open(src) ?(errors.Wrap("copy %s %s: %v", src, dst))
	defer r.Close()

	w := os.Create(dst) ?(errors.Wrap("copy %s %s: %v", src, dst))

	io.Copy(w, r) ?(func(err error) error {
		w.Close()
		os.Remove(dst)
		return fmt.Errorf("copy %s %s: %v", src, dst, err)
	})

	w.Close() ? (func(err error) error {
		os.Remove(dst)
		return fmt.Errorf("copy %s %s: %v", src, dst, err)
	})
}

Assessment

Obviously the returns of the error becomes less visible with the new syntax; which I suppose is part of the point. I don't necessarily find the flow less clear. However, both versions here actually leaves a score of errors unchecked. We could try to deal with this in both variants by joining errors.

Neither case is currently set-up to handle errors from deferred calls. I think that would be interesting to explore because with the proto-type as this would lead to "two panics", one coming from inside a defer statement. I will need to think about how this should be handled, or if it should be disallowed.

Will spend some time investigating the case in the linked repo once I got time.

@nullyfae
Copy link

i think we should still having to declare the full return of the function, and do the callback explicitly.

func wrapWithFnameError(fname string, err error) error {
	err = fmt.Errorf("%s: something went wrong: %w", fname, err)
	return err
}

type SomeStruct struct {
	Param1 int
	Param2 int
}


func MyFunction() (SomeStruct, error) {
	param1, param2, err := createParams()?(wrapWithFnameError("MyFunction", err))
	myStruct := SomeStruct{param1, param2}
	return myStruct, nil
}

also: what happens if you have a function that can return multiple errors? just chain the ? operator.
since you specify

Two return parameters, where the last one is an error

this is how the situation will looks.

func funcWithMultipleErrors() (int, error, error, error) {
	// ...
}

func caller() error {
	value, err3, err2, err1 := funcWithMultipleErrors()???
	fmt.Println(value)
	return nil
}

each "?" really gives to the next one all params except the last one if is of type error. so, the first ? will return if err1 is not nil, the second ? will return if err2 is not nil, and so on.

now, doing the previous code i had an idea: what if ? not always returns early? let me explain.
maybe you are doing some config validation function like this one:

func validateConfig(config AppConfig) error {
	var configErrors error
	dbErrors := config.DB.Validate()
	if dbErrors != nil {
		configErrors = errors.Join(configErrors, dbErrors)
	}
	cronErrors := config.Crons.Validate()
	if cronErrors != nil {
		configErrors = errors.Join(configErrors, cronErrors)
	}
	someOtherErrors := config.Other.Validate()
	if someOtherErrors != nil {
		configErrors = errors.Join(configErrors, cronErrors)
	}
	
	// now, if no errors were found, configErrors will be nil.
	return configErrors
}

you don't want to return the first error you encounter, you want to check for all errors before return them as a whole, so, this is what i thought:

func newValidateConfig(config AppConfig) error {
	dbErrors := config.DB.Validate() ? ()  // this will assign dbErrors = dbErrors
	cronErrors := config.Crons.Validate() ? errros.Join(dbErrors, cronErrors) // this will assign cronErrors = errros.Join(dbErrors, cronErrors)
	someOtherErrors := config.Other.Validate() ? return errors.Join(cronErrors, someOtherErrors)  // now, this one with the return will effectively return early
	return nil
}
  • ? with nothing -> return
  • ? with an empty () (or maybe a "_"?) -> don't return
  • ? with something but without a "return" -> don't return
  • ? with something and a return -> return

if you are doing just one handler, the parenthesis may be omitted (like in function return types).

oh, and if you have multiple handler functions, the return goes outside the parenthesis:

err := myFunction() ? return (handler1, handler2, handler3)

about defers...

  1. you can't do anything with a defer called function output.
  2. the defered function can't do anything with what its caller is returning, except if you use pointers, anonymous functions defined inside the caller and pointers or some black magic like that.

so... the ? can't either assign nor early return, since you are already returning. the only use i think for using ? with a defer call is for doing something like log or panic if it had an error, but that may need an implicit variable for the error param, etc.

@smyrman
Copy link
Author

smyrman commented Mar 14, 2025

I think we should still having to declare the full return of the function, and do the callback explicitly.

@nullyfae , If I understand you correctly, you are suggesting a return keyword after the ? operator. This would make it no longer an expression but a statement. By doing so, we would no longer be allowed to use the operator for inline struct assignments and chained calls. Or at least that's my understanding.

I will. try to answer the other questions.

also: what happens if you have a function that can return multiple errors? just chain the ? operator.
since you specify

This isn't really a case that's handled. The proposal is to only allow cases where there is either one or two return paraemeters, including the error:

func f1(...) error             // One return parameter, which must be an error
func f2[T any](...) (T, error) // Two return parameters, where the last one is an error

If there is a function with two error returns, that's technically allowed, but only the last one will be handled by the ? operator.

Other cases, including the one you mention with func createParams() (T1, T2, error) will need to rely on existing syntax for error handling.

now, doing the previous code i had an idea: what if ? not always returns early?

It doesn't. The ? operator only returns early if the handler function(s) return an error; otherwise it continues. The use-case you describe can be covered by using the proposed errors.Collector, or by writing your own collector that has an error-handler that returns nil.

There is a case of collecting errors in the "Cases" section of the description; please have a closer look at that.

you can't do anything with a defer called function output

There is actually three things we can do with ? on the defer statement:

  1. You can use errors.Join if the ? handler chain on defer returns and error and combine it with the original error, if any.
  2. You can panic if e the ? handler chain on the defer returns an error.
  3. You can disallow ? on defer statements.

If we go for 1 or 2, the application writers can choose to write ? handlers that logs and discards the error (return nil).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
error-handling Language & library change proposals that are about error handling. LanguageChange Suggested changes to the Go language LanguageChangeReview Discussed by language change review committee Proposal
Projects
None yet
Development

No branches or pull requests

5 participants