Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: Go 2: add kind-specific nil predeclared identifier constants #22729

Open
ianlancetaylor opened this issue Nov 15, 2017 · 60 comments
Open
Labels
LanguageChange NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Proposal v2 A language change or incompatible library change
Milestone

Comments

@ianlancetaylor
Copy link
Contributor

A common error for people new to Go is misunderstanding that a interface that is not nil can contain a nil pointer. This is one of the most commonly cited entries in the Go FAQ: https://golang.org/doc/faq#nil_error. A quick search shows at least 24 threads on golang-nuts discussing this, even though it is already in the FAQ.

It is not new to observe that one of the causes of this common mistake is that nil is overloaded. Since changing that would not be backwards compatible, I propose the following changes.

  1. We add six new predefined identifiers: nilptr, nilinterface, nilslice, nilchan, nilfunc, nilmap. These new identifiers are untyped constants that only resolve to a certain kind, much as 1.1 is an untyped constant that only resolves to a float or complex type. nilptr may be used as the zero value for any pointer type, nilinterface may be used as the zero value for any interface type, and so forth. An attempt to use, for example, nilptr in an assignment or comparison with a variable or value that is not a pointer type is invalid.

  2. We add a vet check that warns about any comparison of a value of any type with plain nil. We encourage people to change x == nil to x == nilinterface or x == nilptr or whatever is appropriate. Since initially this vet check will trigger all the time, we can not turn it on when running go test. It could be on by default when running go vet.

  3. At this point people who run go vet will no longer make the common mistake. If v is a value of interface type, then writing v == nilptr will be a compilation error. Writing v == nilinterface will not cause people to erroneously think that this is testing whether v contains a nil pointer.

  4. In some later release, we turn on the vet check when running go test.

  5. If we are ever willing to make a backward incompatible change, we can make v == nil a compilation error rather than simply being a vet error.

Something to consider is that one could imagine permitting v == nilptr when v has an interface type, and having this be true if v is not nilinterface, if v holds a value of pointer type, and if the pointer value is nilptr. I don't know that this is a good idea, and I'm not proposing it. I'm only proposing the above.

@ianlancetaylor ianlancetaylor added v2 A language change or incompatible library change LanguageChange labels Nov 15, 2017
@gopherbot gopherbot added this to the Proposal milestone Nov 15, 2017
@cespare
Copy link
Contributor

cespare commented Nov 15, 2017

One consequence of this would be that the extremely common if err != nil { ... } would be significantly longer: if err != nilinterface { ... }.

@ianlancetaylor
Copy link
Contributor Author

@cespare Very valid point.

@bradfitz
Copy link
Contributor

I'd be happier with something in the middle: only adding nilinterface, but naming it none or something shorter. I don't think nilmap, nilslice, etc generate enough confusion.

@ianlancetaylor
Copy link
Contributor Author

ianlancetaylor commented Nov 15, 2017

@bradfitz It sounds like you are suggesting that we move toward a different name for the zero value of an interface type, but keep the general overloading of nil for non-interface types. Does that sound right?

I don't really see none as conveying the right idea, but I'm having trouble coming up with a good name. We could try nili or inil, but those might be too close to nil. Maybe empty? I hope someone else has a better idea.

(C++ has a similar confusion in which 0 is the zero value for both integers and pointers, with the preprocessor macro NULL often being used in code. In C++ this has now been resolved by preferring the new keyword nullptr as the pointer zero value.)

@bradfitz
Copy link
Contributor

bradfitz commented Nov 15, 2017

It sounds like you are suggesting that we move toward a different name for the zero value of an interface type, but keep the general overloading of nil for non-interface types. Does that sound right?

Yes.

Maybe empty?

I like empty.

Seeing lots of if err != empty { ... } is still pretty verbose, though.

@cznic
Copy link
Contributor

cznic commented Nov 15, 2017

Maybe empty?

empty is nice, and it would be enough to predeclare it in the universe scope without breaking any existing Go1 program.

package main

import (
	"fmt"
	"io"
)

var empty interface{}

func main() {
	var w io.Writer
	fmt.Println(w == empty)
}

Playground

@cznic
Copy link
Contributor

cznic commented Nov 15, 2017

it would be enough to predeclare it in the universe scope without breaking any existing Go1 program.

Which is actually what the proposal says, but I noticed that only now, mea culpa 👎

@davecheney
Copy link
Contributor

@ianlancetaylor thank you for raising this issue. I think this proposal adds a lot of complexity to solve a single issue, returning a typed nil. What about just making a typed nil equal to nil? It is after all what the author of the check intended.

@bradfitz
Copy link
Contributor

What about just making a typed nil equal to nil?

That breaks valid code.

@neild
Copy link
Contributor

neild commented Nov 15, 2017

If interface{}((*T)(nil)) == nil, how do you test to see if an interface value is zero? I also think changing this equality might be superficially more intuitive, but is more confusing when you dig into it.

Instead of changing the name of a zero-valued interface, it would also be possible to change the name of a zero-valued pointer. null, nullptr, or maybe just 0.

@davecheney
Copy link
Contributor

davecheney commented Nov 15, 2017 via email

@faiface
Copy link

faiface commented Nov 15, 2017

This issue of nil vs nil comes up all the time.

I think it's time to reconsider what the real issue is. I personally think the current implementation is sound, useful, and when someone understands how it works, they're not going to make any mistakes with it. Not any more than with any other feature.

Maybe the only real problem is education around the issue. People come to Go with expectations. And, when they are not explicitly and repeatedly told about this behavior, they assume it works differently than it actually does. But, that doesn't mean it works bad, it just means that people's expectations are bad. Maybe the only thing we need to fix is people's expectations.

@neild
Copy link
Contributor

neild commented Nov 16, 2017

Perhaps the problem is less interface equality and more the ease of accidentally assigning a nil pointer to an interface value when you didn't intend to.

@ianlancetaylor
Copy link
Contributor Author

@davecheney Even for Go 2, I don't think it's OK to remove the ability to test whether an interface is the zero value for interfaces.

@faiface You're not wrong, but if people consistently get something wrong it's worth exploring ways to change it to make it less error prone.

@neild suggested in conversation that we should simply change the zero value of pointers to null. A comparison of an interface value with null would be a compilation error. Assigning nil to a pointer, or comparing a pointer with nil, would be a vet error at least for now. Another option would be to copy C++ and use nullptr. The advantage of these approaches is that all the err != nil code would still be fine. The disadvantage is that all the p != nil code, where p is some pointer type, would want to change to p != null.

@jimmyfrasche
Copy link
Member

I'm concerned that this would just push the misunderstanding around while breaking everyone else's muscle memory—but another alternative:

Accept #19642 for making _ a universal zero value and then (initially via a vet check) disallow checking an interface (and possibly other things) against untyped nil.

All the err == nil would change but they'd shorten to err == _.

This could of course expand the problem by allowing one to think it possible to see if whatever is in the interface is zero, but that would come with allowing a universal zero value anyway.

@griesemer
Copy link
Contributor

griesemer commented Nov 16, 2017

I'm not a fan of introducing so much nomenclature (nilptr, nilmap, etc.) for relatively little benefit. I also think that we should keep nil as the zero value for pointers, it seems to me that nil is more commonly used with pointers than interfaces (to be verified).

It also would be a mistake to equate a test of (interface) x == nil as a test that ignores typed nils (as @davecheney suggested). @bradfitz already pointed out that it would not be backward-compatible. But it's actually important that we have this facility in the first place: A common idiom in Go is to define data types whose zero values are values that are ready to use. It can make a lot of sense to define a pointer type whose zero value (nil) is a ready-to use value. It may have methods that work with nil receivers. Such a type may even implement the error interface and thus represent a valid error (and we might not know about it if such a type is defined in some external library). It is imperative that in such a case the test err == nil doesn't yield true. Perhaps far-fetched a scenario for an error type, but not so much for other situations.

But there is another side to this problem, and that is that there is no easy way (*) to test if a variable of interface type holds a (typed) nil pointer (or map, or channel) without knowing the actual type. (This does come up, e.g., in iterators walking over graphs such as the Go syntax tree in go/ast.)

If the language had a facility to test for such non-nil interfaces x holding nil values (perhaps via a builtin function isnilptr(x)), presented squarely next to the x == nil test, as a programmer one might be more acutely aware of the two possibilities and automatically think twice.

(*) One way of doing it is using reflection

func isnilptr(x interface{}) bool {
	v := reflect.ValueOf(x)
	return v.IsValid() && v.Kind() == reflect.Ptr && v.IsNil()
}

@jimmyfrasche
Copy link
Member

@griesemer why single out pointer-y things for such a builtin? Why not a more general holdsZero?

func holdsZero(i interface{}) bool {
	v := reflect.ValueOf(i)
	if !v.IsValid() { // maybe drive the point home further and let this panic if i == nil
		return false
	}
	z := reflect.Zero(v.Type()).Interface()
	return i == z
}

(I wouldn't be surprised if that's missing some edge cases)

In either case, if it were a builtin it should fail to compile unless it's parameter is an interface, to avoid confusion like holdsZero(0) being true.

@neild
Copy link
Contributor

neild commented Nov 16, 2017

Why single out testing for the presence of a zero value?

An interface type specifies the desired properties of implementations of that interface; if zero-ness is a significant property of an interface, then shouldn't it be part of the type definition?

type T interface {
  IsZero() bool
}

It feels to me that interface equality is approaching the problem from the wrong direction; I think that in cases where someone is trying to test for a typed nil value in an interface they almost always really wanted an untyped nil interface value instead. e.g., problem the nil error FAQ entry describes is not that error((*myerror)(nil)) != nil, but that the error contains a typed nil in the first place.

@griesemer
Copy link
Contributor

@jimmyfrasche Maybe. I chose the pointer case because this is the one that I've actually seen in practice.

@neild You have a point with using a method.

@earthboundkid
Copy link
Contributor

I love this proposal.

I think it's worth considering null as the name to add. It's a very commonly used keyword. Semantically, it would make more sense for everything but interface to use null, and interface would continue to use nil. OTOH, if we use none, then interface could use that and everything else could continue using nil. In either case, there's an easy transition using go vet from Go 1 to Go 2.

@ugorji
Copy link
Contributor

ugorji commented Dec 11, 2017

What about making nil a variable OR a function that has a context i.e.

  • nil // current interpretation of nil
  • nil(map)
  • nil([])
  • nil(func)
  • nil(interface)
  • nil(*)
  • nil(chan)

This way, there's no new keyword, but we expand the use of the nil keyword (like we expanded the use of make).

If we combine this with a builtin iszero(x) function where x is any value, and accompanying reflect.IsZero(reflect.Value) function, we should capture most of the use cases without introducing many new keywords.

@griesemer
Copy link
Contributor

@ugorji Just for the record: nil is not a keyword. It's a predeclared identifier. There's a difference.

@faiface
Copy link

faiface commented Dec 11, 2017

I would like to point out one thing. Having interfaceValue == nil or even interfaceValue == nilptr is inconsistent with current Go anyway. Let me explain why. It basically assumes, that the nilptr value will get automatically converted to the concrete type of interfaceValue and the true comparison will happen afterwards.

This is not how it happens in Go. Let's take a look at numbers for a similar example:

var x interface{}
x = float64(0)
if x == 0 {
    fmt.Println("this doesn't happen!")
}

Did you guess correctly what happens in the above example? It is similarly counter-intuitive at first, but it makes sense. The 0 value gets converted to its default type, which is int and comparison proceeds and results in false, because we are comparing different types.

With interfaces it's similar. Does it even make any sense whatsoever, to ask whether an interface value is a nil value of an arbitrary pointer type?

@ianlancetaylor
Copy link
Contributor Author

@faiface I agree: it doesn't. But the evidence is clear that that is what many people new to Go think is happening. It's a FAQ, and it's visibly a stumbling block for new Go programmers. I think it's worth examining whether there is something we can do to cause fewer people to stumble here.

@faiface
Copy link

faiface commented Dec 11, 2017

@ianlancetaylor How about the numbers? Would you solve those too? If not, being able to check if interface contains arbitrarily typed nil pointer, but not being able to check if it contains an arbitrarily typed zero, or one, would be an inconsistency that could cause more confusion than clarification.

@ianlancetaylor
Copy link
Contributor Author

@faiface This proposal would not fix the comparisons of an interface value with 0. Current experience is that that is a much less significant problem: it is rarely reported as a bug, and there has been no need to create a FAQ entry for it. I'm not particularly worried about that inconsistency myself.

(Note that this specific proposal is not for a way to check whether an interface contains a nil pointer of arbitrary type. I mentioned that as "something to consider," but it is not part of the proposal.)

@tandr
Copy link

tandr commented Jul 20, 2019

Thanks Ian. If I understood that comment correctly, it talks about "plain uninitialized pointer is nil", (maybe zero initialized memory to be equivalent to nil), but I am looking at "checking for nil on the receiver" part.
You see, I am not sure if checking receiver against fat-nil instead of "just" nil will change what happens. Even so, it feels like quite a lot of places of "naked" nil comparison will work fine under fat-nil comparison, and for the rest of the cases - maybe something else is needed, like an explicit cast to something. Forbidding untyped nil comparison sounds like an overkill, since compiler could help here with type deduction and have nil correctly typed/wrapped/casted. But still, it might need "go vet" to the rescue...

Now, how many of these cases of comparing something to nil would behave differently (say in Go sources themselves, and maybe some libraries) under these changes is a task that is out of my league (i.e. I have no clue how do it right now).

Thank you for your time.

PS. "fat nil" is fat pointer, "typed nil"

@alanfo
Copy link

alanfo commented Jul 31, 2019

Whatever the merits of this proposal, perhaps I could make one simple point:

If anything is done which would change if err != nil, then the folks who disliked the try proposal will go nuclear!

I have a lot of sympathy with what @faiface said earlier, namely that all that is really needed here is better education on how interfaces work.

However, if something must be done to draw attention to the difference between an interface variable being nil and its dynamic type having a nil value, then I thought @griesemer's idea of an isnilptr built in function to test for the latter would be the best solution though holdsnil might be a better name for it. I can't see the point of testing for zero values generally as the confusion only arises when the dynamic value is nil.

@ianlancetaylor
Copy link
Contributor Author

Adding an isnilptr function really only helps people who already understand the problem. The goal of this issue is to help guide people who do not understand the problem.

@alanfo
Copy link

alanfo commented Aug 1, 2019

Adding an isnilptr function really only helps people who already understand the problem.

I think it might in fact also help those people who don't understand the problem or are confused about it because, as @griesemer said earlier:

If the language had a facility to test for such non-nil interfaces x holding nil values (perhaps via a builtin function isnilptr(x)), presented squarely next to the x == nil test, as a programmer one might be more acutely aware of the two possibilities and automatically think twice.

Even if we had something like nilinterface I am not convinced it would necessarily solve the problem because a beginner might still think that if x == nilinterface was testing the interface's dynamic type for 'nilness' and that (regardless of the dynamic type) you had to use nilinterface because x was an interface variable.

Reading the proposal again, your alternative idea of allowing x == nilptr for interfaces would be equivalent to isnilptr(x) though I prefer the latter as the former would (in effect) be overloading the == operator.

@pcostanza
Copy link

@alanfo You seem convinced that interface nil vs. pointer nil is just a matter of teaching and better understanding of the language. I don't agree with that. I rather think it's objectively confusing. Consider:

n = nil
e = n
if e != nil {
   fmt.Println(e)
}

This code fragment prints <nil> depending on the involved types. It's a pity you have to know the involved types in a language that otherwise (rightfully) prides itself of being particularly easy to read and understand.

The problem is that interface nil and pointer nil are conceptually two vastly different concepts, and it was not a good idea in the original design of Go to give them the same name.

I believe this particular problem should not anymore be the rite of passage for Go programmers that it currently is, because it's unnecessary and doesn't serve a purpose. (Backwards compatibility is not a particularly convincing purpose in a language that is currently seeing some dramatic changes anyway.)

Just my 0.02€... ;)

Pascal

@alanfo
Copy link

alanfo commented Aug 1, 2019

@pcostanza

The problem is that interface nil and pointer nil are conceptually two vastly different concepts, and it was not a good idea in the original design of Go to give them the same name.

They are different but presumably, back in v1.0 days, the Go team didn't consider they were different enough to justify the use of another predefined identifier (such as empty or void) to indicate that an interface contained no value at all. So we are where we are.

Although we are likely to see a dramatic change to the language in the introduction of generics (but probably not now error handling) the current generics proposal is still backwards compatible (or, at any rate, can be) and I know the Go team regard this as a very important consideration.

As I don't really see how anyone can use interfaces successfully without being able to distinguish between an interface variable and what (if anything) it currently contains, then education must play an important part and, whilst I don't object to a language change being made to reinforce this, I do think such a change should be simple, proportionate and (preferably) backwards compatible.

@ianlancetaylor
Copy link
Contributor Author

Reading the proposal again, your alternative idea of allowing x == nilptr for interfaces would be equivalent to isnilptr(x) though I prefer the latter as the former would (in effect) be overloading the == operator.

This proposal has a series of five steps aimed to fix the problem over time. Now maybe it's a bad proposal--in fact, it probably is--but it's more than just one idea.

@alanfo
Copy link

alanfo commented Aug 1, 2019

When I referred to your 'alternative idea' I was alluding to your final paragraph:

Something to consider is that one could imagine permitting v == nilptr when v has an interface type, and having this be true if v is not nilinterface, if v holds a value of pointer type, and if the pointer value is nilptr. I don't know that this is a good idea, and I'm not proposing it. I'm only proposing the above.

As you made clear, this wasn't part of the actual proposal but an idea to consider.

@ianlancetaylor
Copy link
Contributor Author

Ah, sorry.

@andig
Copy link
Contributor

andig commented Apr 10, 2020

This is a little off-topic, but since https://golang.org/doc/faq#nil_error was mentioned I feel it's not precise enough:

func returnsError() error {
	var p *MyError = nil
	if bad() {
		p = ErrBad
	}
	return p // Will always return a non-nil error.
}

func returnsError() error {
	if bad() {
		return ErrBad
	}
	return nil
}

It's a good idea for functions that return errors always to use the error type in their signature (as we did above) rather than a concrete type such as *MyError, to help guarantee the error is created correctly.

It seems that the bad (first) example is "use(ing) the error type in their signature" as advised, yet still broken. It should not only use the error type but also return the nil interface instead of a differently-typed nil (if that's the correct wording).

@rogpeppe
Copy link
Contributor

@griesemer wrote:

It also would be a mistake to equate a test of (interface) x == nil as a test that ignores typed nils (as @davecheney suggested). @bradfitz already pointed out that it would not be backward-compatible. But it's actually important that we have this facility in the first place: A common idiom in Go is to define data types whose zero values are values that are ready to use. It can make a lot of sense to define a pointer type whose zero value (nil) is a ready-to use value. It may have methods that work with nil receivers. Such a type may even implement the error interface and thus represent a valid error (and we might not know about it if such a type is defined in some external library). It is imperative that in such a case the test err == nil doesn't yield true. Perhaps far-fetched a scenario for an error type, but not so much for other situations.

I've been thinking about this for a long time, and while I buy the theoretical argument, I'm still not entirely convinced.
I haven't yet seen many significant cases where this property is used to advantage.

One point in favour of disallowing typed nils as valid receivers: the pointer method set also contains the value method set, but if you happen to call a value method when the pointer is nil, you'll get a panic, so that property isn't as useful as it could be. Passing the zero value is one possible way of fixing that, but disallowing nil receivers would be another. If you disallow nil receivers, then perhaps it's reasonable to change a typed nil pointer into an untyped nil when converting to an interface. The problem is much less acute for slices, channels and maps, so perhaps they could be left as is.

But as @ianlancetaylor points out in the above-mentioned issue, doing this would change the behaviour of existing code, which probably rules it out as a language change.

@tandr
Copy link

tandr commented Aug 24, 2020

Would completely disallowing a naked (uncasted) nils in comparisons solve this? Like, enforce a rule that nil cannot be naked, it has to be dressed in a type wrapper. (if only for "complex" type comparisons at the beginning) go vet should be able to help with transition.

Sorry, I don't remember if it was suggested and declined as "too much existing code affected".

@ianlancetaylor
Copy link
Contributor Author

@tandr Anything that requires us to rewrite if err != nil is a non-starter. (That is also a problem with this proposal.)

@pcostanza
Copy link

@ianlancetaylor You could have nil take over the meaning of your nilinterface and drop all other current uses of nil. nil would then only be valid for interface types, and for nothing else anymore. (At least after a transitional period.)

@earthboundkid
Copy link
Contributor

The options are:

  • Add a new identifier for interfaces (+slice +map +channel?) and leave nil pointers alone
  • Add a new identifier for pointers (+slice +map +channel?) and leave nil interfaces alone
  • Add two new identifiers (or more with slice/map/channel)

Whichever is done, for backwards compatibility reasons p == nil and err == nil can probably never be changed, but you could add a lint check + rewriter to used the preferred one.

On identifiers, my two cents are that nilinterface is too long to type regularly. (interface{} is already long enough, and you don't have to type it that often.) null is good on its own, but probably too close to nil if we have both in the language. none seems fine. So, I lean towards, just add none for interfaces and get a rewriter for err == nil to err == none. You could also make none valid for map/slice/chan, but I don't think that solves a real problem, so I would lean against it.

@andig
Copy link
Contributor

andig commented Aug 24, 2020

To pick up thw comment by @tandr: why to we need the ability to invoke methods on a nil value at all? I think I understand why its technically possible and permitted by the language as-is, but is it necessary for any purpose?

@pcostanza
Copy link

@andig It’s useful for exactly the same reason that dynamic dispatch is useful in other cases: You can implement the if receiver == nil case in a central place, and don’t have to spread it all over the rest of the code. (No language constructs are strictly speaking “necessary,” it’s all about convenience.)

@xuan-nguyen-swe
Copy link

xuan-nguyen-swe commented Aug 11, 2023

I did not read all comments in this very long topic, sorry for that.
I am also new to this language, so sorry if I don't understand the details.

But is that the same story as Dart?

Dart 1: no null-safety
Dart 2: null-safety feature is optional. migration-tool is provided.
Dart 3: null-safety feature is a must

100% sound null safety
Dart 2.12 introduced null safety more than two years ago. In Dart 2.12, users needed to enable null safety with a pubspec setting. In Dart 3, null safety is built in; you cannot turn it off.

Why don't we apply this successful story to Go?

@ianlancetaylor
Copy link
Contributor Author

@xuan-nguyen-swe This issue doesn't have anything to do with null safety. That is a completely different topic. For null safety see, for example, #28133.

@xuan-nguyen-swe
Copy link

@ianlancetaylor thanks for redirecting me to the correct topic!
I am new to Go, so please apologies to me if I misunderstood something.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
LanguageChange NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Proposal v2 A language change or incompatible library change
Projects
None yet
Development

No branches or pull requests