Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: Go 2: add ?? operator to select first non-zero value #37165

Closed
earthboundkid opened this issue Feb 11, 2020 · 41 comments
Closed

proposal: Go 2: add ?? operator to select first non-zero value #37165

earthboundkid opened this issue Feb 11, 2020 · 41 comments
Labels
LanguageChange Proposal v2 A language change or incompatible library change
Milestone

Comments

@earthboundkid
Copy link
Contributor

It is often asked why Go does not have a ternary operator. The Go FAQ says,

The if-else form, although longer, is unquestionably clearer. A language needs only one conditional control flow construct.

However, technically Go has a second form of control flow:

func printer(i int) bool {
    fmt.Println(i)
    return i%2 != 0
}

func main() {
    _ = printer(1) && printer(2) && printer(3)
    // Output: 1\n2\n
}

I believe that many of the usecases that people want a ternary operator for could be covered by adding a ?? operator that is similar to && but instead short-circuit evaluates non-boolean expressions while the resulting value is a zero-value.

For example, these two snippets would be identical:

port := os.Getenv("PORT")
if port == "" {
    port = DefaultPort
}
port := os.Getenv("PORT") ?? DefaultPort

Another use case might be

func New(c http.Client) *APIClient {
    return &APIClient{ c ?? http.DefaultClient }
}

In general, ?? would be very useful for setting default values with less boilerplate.

Another use for ?? might be

func write() (err error) {
    // ...
    defer func() {
        closeerr := w.Close()
        err = err ?? closeerr
    }()
    _, err = w.Write(b)
    // ...
}

Some rules: ?? should only work if all expressions evaluate to the same type (as is the case for other operators), and ?? should not work for boolean types, since that would cause confusion in the case of a pointer to a bool. If/when Go gets generics, you can trivially write first(ts ...T) T, so the operator is only worth adding to the language if it has short-circuit evaluation.

In summary, ternary is notoriously unclear, but ?? would not be any more unclear than &&. I believe it would be more clear than an equivalent if-statement since it would more clearly express the intent of setting a default, non-zero value.

@randall77
Copy link
Contributor

None of your examples require the short-circuit behavior. I.e., first(ts ...T) T would work fine for them.

@earthboundkid
Copy link
Contributor Author

Yes, that's fair, although so far generics are still only theoretical.

@ianlancetaylor ianlancetaylor added v2 A language change or incompatible library change LanguageChange labels Feb 12, 2020
@ianlancetaylor
Copy link
Contributor

For language change proposals, please fill out the template at https://go.googlesource.com/proposal/+/refs/heads/master/go2-language-changes.md .

When you are done, please reply to the issue with @gopherbot please remove label WaitingForInfo.

Thanks!

@gopherbot gopherbot added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Feb 12, 2020
@earthboundkid
Copy link
Contributor Author

earthboundkid commented Feb 12, 2020

Would you consider yourself a novice, intermediate, or experienced Go programmer?

Intermediate/advanced. Long-time Go user, but not a language dev.

What other languages do you have experience with?

Extensive Python, JavaScript, and PHP. Scattered others.

Would this change make Go easier or harder to learn, and why?

Incrementally harder, since there would be one more operator to learn, but the operator has syntax and semantics similar to the existing && operator, so not as bad as a entirely new language feature.

Has this idea, or one like it, been proposed before?

Not AFAICT, but ternary operators are frequently proposed.

If so, how does this proposal differ?

Rather than creating a full-blown ternary, it only allows the equivalent of (x != 0) ? x : y or || in languages that have weak-typing of booleans.

Who does this proposal help, and why?

It helps anyone who has to write or read default configuration. It helps writers because ?? is shorter. It helps readers because an if-statement can do anything, but ?? is really only useful for setting non-default values, so there's less thinking needed to absorb the meaning of the code.

What is the proposed change?

Add ?? as an operator. First ?? evaluates its LHS. If LHS != the zero value for the type, it returns that value. If LHS == the zero value, it also evaluates RHS and returns that value.

Please describe as precisely as possible the change to the language. What would change in the language spec?

After the "Logical operators" section, there would be a section "Coalescing operator" with text like:

The coalescing operator ?? applies to non-boolean values and yield a result of the same type as the operands. If the left operand value is not the zero value for the type, it yields the left operand value. If the left operand value is the zero value for the type, the right operand is conditionally evaluated and yielded.

In the spec, binary_op = "||" | "&&" | rel_op | add_op | mul_op . would change to binary_op = "??" | "||" | "&&" | rel_op | add_op | mul_op . In the operator precedence table, 2 && would change to 2 && ??. (x ?? y && z would be disallowed by the type system, so the exact precedence shouldn't matter.)

Please also describe the change informally, as in a class teaching Go.

Go provides a convenient way to select the first non-zero value in a series. For example name := input ?? "Anonymous" means if input is not blank, set name to it, otherwise use "Anonymous". If the right hand expression is a function, it will only be called if the left hand expression is a zero-value. For example, name := input ?? get_name() will only call get_name if input is "".

Is this change backward compatible? Breaking the Go 1 compatibility guarantee is a large cost and requires a large benefit.

Yes.

Show example code before and after the change.

Before:

port := os.Getenv("PORT")
if port == "" {
    port = DefaultPort
}

After:

port := os.Getenv("PORT") ?? DefaultPort

Before:

func New(c http.Client) *APIClient {
    if c == nil {
       c = http.DefaultClient
    }
    return &APIClient{ c }
}

After:

func New(c http.Client) *APIClient {
    return &APIClient{ c ?? http.DefaultClient }
}

What is the cost of this proposal? (Every language change has a cost).

The costs are that the ?? would be unavailable for other uses, and learners of Go would have to learn one more operator. Fortunately, this operator is quite similar to the nullish coalescing operator in C#, PHP, and JavaScript, and the or operator in Python, so it would be familiar to many learners.

How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?

The language grammar would change, so essentially every tool would need to learn the new grammar. :-(

What is the compile time cost?

Should be negibile.

What is the run time cost?

Negibile.

Can you describe a possible implementation?

See spec changes above.

Do you have a prototype? (This is not required.)

No.

How would the language spec change?

See above.

Orthogonality: how does this change interact or overlap with existing features?

While it could be abused for control flow, I don't see that as being very likely, since && has the same potential today but is not widely used for control flow.

Is the goal of this change a performance improvement? If so, what quantifiable improvement should we expect? How would we measure it?

No.

Does this affect error handling? If so, how does this differ from previous error handling proposals?

It has minor effects on the case of wanting to return the first non-nil error in a write and flush operation:

func write() (err error) {
    // ...
    defer func() {
        closeerr := w.Close()
        err = err ?? closeerr
    }()
    _, err = w.Write(b)
    // ...
}

Is this about generics? If so, how does this differ from the the current design draft and the previous generics proposals?

No, but if there were generics, a user could write a non-shortcircuiting first(ts ...T) T function instead of using this operator.

@gopherbot please remove label WaitingForInfo.

@gopherbot gopherbot removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Feb 12, 2020
@alanfo
Copy link

alanfo commented Feb 13, 2020

Although I'm a supporter of adding some sort of short-circuiting ternary operator or built-in function to Go, I'm not convinced that this proposal goes far enough in that direction to be worth doing.

If I'm understanding it correctly, what you're proposing is that ??, which is known as the null-coalescing operator in C# and has counterparts in several other languages, would in effect become a 'zero value' coalescing operator in Go.

So stuff like this would become possible:

a, b := 0, 1
c := a ?? b  // c is assigned 1 as a is 0

a, b = 2, 3
c = a ?? b   // c is assigned 2 as a is non-zero

s, t := "", "one"
u := s ?? t  // u is assigned "one" as s is empty

s, t = "two", "three"
u  = s ?? t  // u is assigned "two" as s in non-empty

As such it's equivalent to writing the following in a C-like pseudo-code:

// d and e are expressions of someType (non-boolean)
someType f = d != zeroValue(someType) ? d : e;       

This is really quite limited compared to a 'full-fat' ternary operator where the condition can be any boolean expression. So, whilst I congratulate you on a novel proposal, I'm finding it difficult to get enthused about it.

@deanveloper
Copy link

Note that this is similar to ||, not && in JavaScript and similar weakly-typed languages. && means "if the left side is true, evaluate to the right side" while || means "if the left side is false, evaluate to the right side".

Many languages have similar concepts and are widely used, however I don't see the use as much in Go. Go relies less on classical sentinel values than other languages, and instead uses errors to denote when errors happen, rather than returning a sentinel value.

It still does have it's uses of course as you've shown, but I still think coalescing is a bit limited in the context of Go.

@earthboundkid
Copy link
Contributor Author

This is really quite limited compared to a 'full-fat' ternary operator

Yes, exactly. :-)

I believe this covers a huge amount of the situations where ternary is actually useful without allowing the complexity of ternary that the Go authors specifically don't want.

Your examples use single letters, so it makes the ?? operator seem pointless, but in real code, these sorts of things come up all the time:

port := os.Getenv("PORT") ?? ":8000"
timeout = timeout ?? DefaultTimeout
logger := s.logger ?? DefaultLogger
client = c ?? http.DefaultClient

etc.

@earthboundkid
Copy link
Contributor Author

Note that this is similar to ||, not && in JavaScript and similar weakly-typed languages.

Yes, thanks for pointing that out. I realized that I had written the wrong thing before when I was writing up the change proposal template but didn't go back and correct it.

@infeno
Copy link

infeno commented Feb 13, 2020

For future proof (if this is the right scenario), we should avoid ??. I’m a beginner in Go but I can see major issue in this proposal.

I have often assume empty string/out of range/garbage value as either valid or not valid, how can we be sure the user supply value/input is correct?

Invalid value could also cause runtime errors that I prefer to stick to the current way to handle such cases with regexp or custom validations in a function that can be change without breaking change and less noise.

port := os.Getenv("PORT") ?? ":8000”  // :-8080 what if I want 9090 or 8000-8181 is in use in other environment? That gonna made lots of changes and adding more validation check.
timeout = timeout ?? DefaultTimeout  // If timeout is -1, if it valid?
logger := s.logger ?? DefaultLogger  // What should contain in logger?
client = c ?? http.DefaultClient  // c is string or non-string type?

In my opinion, I would wish to avoid less boilerplate for this case. I guess it was possible for JavaScript and PHP as they are scripting languages that can benefits from smaller file size transfer over the slow connectivity and seem fragile for Go, and opportunity to make an assumption that it work in my specifications/requirements.

I see Github CLI as one of an example:
https://github.com/cli/cli

@infeno
Copy link

infeno commented Feb 14, 2020

TLDR; There are many developers who simply want to solve short-term problem using ?? and someone/other team wish to make changes could be concern if there are other "validations" or whether a ?? b could be replace with func aorb() {...}.

@earthboundkid
Copy link
Contributor Author

There is a Go proverb, "Make the zero value useful." I believe this makes the zero value easier to use because you can easily write code to test for the zero and use a good default if the zero is provided. Of course there will always be code that needs better validation like "ensure timeout is a positive value" or "port string must begin with colon". This just makes the simple case more simple. The advantage of this vs. a ternary is that a) it's actually simple (IMO, ternary is hard to read) and b) it cannot be abused to create unreadable monstrosities like nested ternaries (a ?? b ?? c is still pretty easy to understand, unlike a nested ternary which everyone agrees is unreadable).

@ianlancetaylor
Copy link
Contributor

There don't seem to be any examples above that require lazy evaluation. If lazy evaluation is not an essential part of this, then it seems that it would be possible to write a generic zero-coalescing function. Using the syntax from the current generics design draft, it would look like

func Default(type T comparable) (a, b T) T {
    var zero T
    if a != zero {
        return a
    }
    return b
}

This is longer to write than ??, but it doesn't require adding a new operator. It seems that we should put this proposal on hold until we have generics, and revisit at that time.

@earthboundkid
Copy link
Contributor Author

earthboundkid commented Feb 19, 2020

Nitpick, do you need the type to be comparable? Types that aren't fully comparable, such as functions, can still be compared to the zero value.

func Default(type T) (a, b T) T {
    const zero T
    if a != zero {
        return a
    }
    return b
}

Edit: This doesn't compile because zero isn't a constant zero, but const zero T doesn't compile because the value is omitted:

type T = func()

func Default(a, b T) T {
	var zero T
	if a != zero {
		return a
	}
	return b
}

Making a constant of zero value for arbitrary type T seems to be impossible?

@theohogberg
Copy link

If {} else {} does exactly what the ternary operator does. Why would the language need more than one way to express the same thing?

@earthboundkid
Copy link
Contributor Author

As proposed, ?? is not a ternary operator.

@theohogberg
Copy link

theohogberg commented Jun 17, 2020

As proposed, ?? is not a ternary operator.

Sorry, but is this then basically the nullish coalescing operator (??) from javascript?

https://github.com/tc39/proposal-nullish-coalescing

The problem with javascript is that the || operator evaluates empty strings '' and 0 as false and not as strings and ints. This is a "feature" of JavaScript which leads to the use case of adding a nullish coalescing operator. GO does not follow this scenario when it comes to evaluation, thereby the use for a ?? operator seems pointless to be honest. If I want to return a non nil value I would just use if x =! nil {}

@earthboundkid
Copy link
Contributor Author

Please read the proposal before criticizing:

this operator is quite similar to the nullish coalescing operator in C#, PHP, and JavaScript, and the or operator in Python, so it would be familiar to many learners.

Yes, it is similar to JavaScript, which in turn has taken an operator from C#/PHP.

Your input that you don’t want a new operator is valuable. Giving your criticisms when you clearly haven’t read the proposal is not.

@theohogberg
Copy link

Your input that you don’t want a new operator is valuable. Giving your criticisms when you clearly haven’t read the proposal is not.

I'm merely explaining why I personally don't think this proposal will add value to the language as is. I think that was pretty clear in my original answer.

Thank you.

@JohnCGriffin
Copy link

Thank you for a well considered feature proposal.

The current Go language has conditional flow control, notably switch and if. However, among serious programming languages, Go is unique in its lack of a conditional expression. With respect to this common programming need, Go comes up short by comparison. The proposed feature would greatly improve Go coding and readability, while taking nothing away from simplicity or compatibility.

@seankhliao seankhliao changed the title Proposal: Add ?? operator to select first non-zero value proposal: Go 2: add ?? operator to select first non-zero value Aug 9, 2022
@kurahaupo
Copy link

An example where short-circuiting would be important would be along the lines of

 foo := cli.option.foo ?? os.getenv("Foo") ?? complicated.database.fetch("Foo")

The behaviour of the usual 3-part ("ternary") conditional operator is only important here in that it illustrates how this operator is simpler: it coalesces A ? A : B into A ?? B thus avoiding repeating A and making the code clearer. The assertion that " if A{B}else{C} is the same as A?B:C" is misleading: the former is a statement while the later is an expression, and therefore the former cannot directly replace the latter.

I agree with the sentiment of "make zero useful", and the conciseness of A??B is definitely a win, but I wonder whether it might be useful to consider an optional enhancement that makes explicit what comparison is being done against the first operand:

My idea here is to read these as "use left operand provided the assertion is met, otherwise use the right operand":

i := GetNum() ??         GetDefNum()  // default
i := GetNum() ??{_!=0}   GetDefNum()  // equivalent
p := GetPtr() ??         GetDefPtr()  // default
p := GetPtr() ??{_!=nil} GetDefPtr()  // equivalent
s := GetStr() ??         GetDefStr()  // default
s := GetStr() ??{_!=""}  GetDefStr()  // equivalent

s := GetStr() ??{_!="none"} ""        // any string except "none"

i := LinuxSysCall() ??{_>=0} -1       // emulate libc; system calls return -errno on error

f := GetFloat() ??{min<=_ && _<=max} math.NaN()  // defer handling range error

a  := FindAnswer(TheQuestion                ??{_!="" && _!="where's Waldo"}
                 os.Getenv("thequestion")   ??    // still not Waldo
                 MyDB.Lookup("thequestion") ??{}  // default _!=""
                 "what is six times nine")

@golightlyb
Copy link
Contributor

Would prefer a genuine ternary operator

@earthboundkid
Copy link
Contributor Author

Would prefer a genuine ternary operator

func cond[T any](match bool, ifVal, elseVal T) T {
  if match { 
    return ifVal 
  }
  return elseVal
}

absoluteDoubleN := cond(n > 0, func() int { return n*2 }, func() int { return n*-2 })()

@kurahaupo
Copy link

@seankhliao does this have to wait for Go2? It's not a breaking change, since it adds a new token from a byte sequence that's currently invalid.

@carlmjohnson effective, but rather verbose. Does the current compiler elide the trampolines and extra call frames?

Or maybe we approach this from another angle: some form of auto-lambda for function parameters.

func cond[T any](match bool, ifFn, elseFn lambda T) T {
  if match {
    return ifFn();
  }
  return elseFn();
}

absoluteDoubleN := cond(n > 0, n*2, n*-2)

I would initially allow this only for parameters of functions that can be fully inlined.

I suggest that until Go2, lambda could have an alternative (non-conflicting) spelling, such as func ....

@ianlancetaylor
Copy link
Contributor

@kurahaupo We conventionally label all language changes as v2. We don't currently expect that there will ever be a Go 2 in the sense of adding breaking changes; see https://go.googlesource.com/proposal/+/refs/heads/master/design/28221-go2-transitions.md .

@ianlancetaylor
Copy link
Contributor

As I said over at #60204 (comment), I think that if we do add an operator it should be ||. Not that I'm necessarily in favor of adding an operator, but || is already quite close to what we want here and it seems better to reuse it than to add a new operator.

@ianlancetaylor
Copy link
Contributor

On the subject of auto-lambda, it may be interesting to read #37739.

@earthboundkid
Copy link
Contributor Author

The argument for a new operator is so you can tell at a glance if a variable is a Boolean or not. Then again in JavaScript it doesn’t seem to present much problem in practice.

@ianlancetaylor
Copy link
Contributor

That is one argument, and it's a good one, but after all if you just look at a + b you don't know if it is an integer type, a floating-point type, a complex type, or a string type.

@DeedleFake
Copy link

@carlmjohnson

Though I am somewhat in favor of a new operator over reusing ||, I think it would be even less of a problem to do so here than it is in JS because of static typing catching the rare instance where there is a mixup.

@jimmyfrasche
Copy link
Member

Would && be similarly expanded?

@ianlancetaylor
Copy link
Contributor

I don't see what the semantics of an expanded && would be. What did you have in mind?

@jimmyfrasche
Copy link
Member

Return the first zero or last nonzero value.

It's what other languages with "expanded" || do for &&. It allows a backdoor ternary in x && y || z but that would be somewhat mitigated by them presumably having to all be the same type.

@earthboundkid
Copy link
Contributor Author

How would a && b == c bind?

@jimmyfrasche
Copy link
Member

I would presume that nothing else changes.

@earthboundkid
Copy link
Contributor Author

If a b and c are all eg strings, I would want (a && b) == c, but if a is a bool and b c are strings, I want a && (b == c).

@kurahaupo
Copy link

kurahaupo commented Jun 4, 2023 via email

@rsc
Copy link
Contributor

rsc commented Jun 21, 2023

Marking this obsoleted by #60204.

@rsc
Copy link
Contributor

rsc commented Jun 21, 2023

This proposal has been declined as obsolete.
— rsc for the proposal review group

@rsc rsc closed this as completed Jun 21, 2023
@kurahaupo
Copy link

kurahaupo commented Jun 22, 2023

@rsc

Marking this obsoleted by #60204.

This proposal would appear to be significantly broader than #60204:

  • Short circuit evaluation
  • Any type with a "zero" value, not just string
  • Compact syntax (as an operator)

So I am unclear why "obsoleted by" wasn't in the opposite direction.

If there are other reasons to decline this proposal, could we have some feedback on them please?

@earthboundkid
Copy link
Contributor Author

I think using || would be confusing because of type issues like #60933, and using ?? is a big change because it introduces a new operator. I hope that #26842 does get approved some day though.

@ianlancetaylor
Copy link
Contributor

@kurahaupo

  • Short circuit evaluation

True.

  • Any type with a "zero" value, not just string

#60204 applies to any comparable type, not just string.

  • Compact syntax (as an operator)

In the Go world that is a minor advantage.

Adding a new operator is a language change that has a much much higher bar than adding a new function to the standard library. Supporting short-circuit evaluation simply isn't a strong enough argument for adding an operator. Sorry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
LanguageChange Proposal v2 A language change or incompatible library change
Projects
Status: Declined
Development

No branches or pull requests