proposal: "Filled types": a mechanism to guarantee types are not nil at compile time #33078

alvaroloes · 2019-07-12T14:50:58Z

Introduction and justification

The main idea of this proposal is to achieve safer applications by having a mechanism to ensure, at compile time, that a type can't be `nil`.

For example, when you have functions that receive a pointer or an interface, like this:

func parse(entity *Entity, data []byte) error {
	// ...
}

func save(store io.Writer, data io.Reader) error {
	// ...
}

It doesn't make sense for those parameters to be nil (except the slice). It is almost always an error to pass nil and, if we don't add the corresponding:

if x == nil { 
	return errors.New("x can't be nil")
}

The program could crash at some point with an Invalid memory address or nil pointer dereference error.

We could use defer-recover to try to solve this somehow, but if you have spawned a goroutine and it panics due to a nil dereference, the whole program crashes, no matter if you had defer-recover in your main function.
This is especially dangerous in servers. Take the following example:

func couldPanic(w http.ResponseWriter, r *http.Request) {
	var pointer *int;
	*pointer = 42
}

func main() {
	http.HandleFunc("/", couldPanic)

	if err := http.ListenAndServe(":8080", nil); err != nil {
		log.Fatal(err)
	}
}

This is a simple web server with a handler that could panic. In this contrived example, it always panics because "pointer" is nil.
This is not that problematic, as our server keeps running and receiving other requests after that handler panicked.
However, imagine that the handler spawns a goroutine and that goroutine panics:

func couldPanic(w http.ResponseWriter, r *http.Request) {
	go func() {
		var pointer *int;
		*pointer = 42
	}()
}

func main() {
	http.HandleFunc("/", couldPanic)

	if err := http.ListenAndServe(":8080", nil); err != nil {
		log.Fatal(err)
	}
}

Again, this handler panics, but this time our whole server is down. The application has crashed.
This is a really important problem.

Finally, but not least, it is very common to use a pointer to structs when passing them to or returning them from functions to avoid the cost of copying them, even if the function is not meant to modify its content:

func sumPoints(p1 *Point, p2 *Point) *Point {
	//...
}

Ideally, we use pointers as function parameters to express that:

"I expect the function to modify its content"
"There could be no value to pass, so I expect you to handle that case too" (common in database queries)

But most of the cases (in the majority of the Go code I have seen) they are used as an optimization to avoid copying. This is good, but then we lose the real intent of the function so we end up having the need to always check for nil if we don't want to crash.

Note: This proposal is not meant to be final nor address all panics. The idea is to get feedback and keep iterating it if we all find it useful

Note 2: I have tried the proposal to be as backward compatible as possible (see section "Backward compatibility"). Please check the "Alternatives" to see other variations that are not backward compatible

Syntax

For this proposal, I will use the filled type modifier, although this could change.

Expand this for more details

We need a way to express that a type can't be nil by using some kind of syntax. However, I don't want the syntax to get too much attention as it is not important right now (it will be later). Here I show some options if you want to play with them:

// Wordy options
// - "nonil" or "nonnil"
var myPointer nonil *int
var myFunc nonil func(int) int
var myInterface nonil interface{}

// - "valid"
var myInterface valid interface{}

// - "full"
var myInterface full interface{}

// - "fill" This seems syntactically incorrect (as it is a verb, not an adjective), but it has some similarity with "nil".
// I kind of like the symmetry: for "null" => "full"; for "nil" => "fill" 
var myInterface fill interface{}

// - "filled" This is, for me, the closest to the semantic we want to express
var myInterface filled interface{}

// Symbolic options (I can't think of a good one). To be honest I don't like to use a symbol
- "!"
var myInterface !interface{}

- "#"
var myInterface #interface{}

The idea would be to choose one that is short and meaningful (the Holy Grail). Let's choose filled for now. We could change it later if this proposal moves forward.

Proposal description

Go has the following nillable types: pointers, functions, maps, interfaces, channel and slices. Nil slices, however, are equivalent in behavior to empty slices, so I'd say we don't need to express that a slice is not nil (we can decide this later).

This proposal suggests to add a type modifier to express that a nillable type cannot be nil, so it can be checked at compile time.
For example, the following normal Go code is correct:

var x io.Reader = nil
fmt.Println(x)

But it could panic if we do x.Read(data).

We could write the same code this way:

var x filled io.Reader = nil
fmt.Println(x)

And it won't compile, as a filled io.Reader type can't be nil.

This is the basic idea, let's go with the details:

Filled types don't have zero value

You must initialize a filled type with some value different than nil. This is a requirement to guarantee the filled type is never nil.
If you want to benefit from a zero value, use a nillable type.

Expand this for more details

var x filled interface{} // Won't compile
var x filled interface{} = 42 // Compiles
fmt.Println(x)

If a struct contains a field of a fillable type, then it doesn't have a zero value and you must initialize it:

type PersonWithNillable struct {
	name String
	friendsByName map[string]friend
}

type PersonWithFilled struct {
	name String
	friendsByName filled map[string]friend
}

var person PersonWithNillable // Compiles
var person PersonWithFilled // Doesn't compile
var person PersonWithFilled = PersonWithFilled { // Or person := PersonWithFilled {...}
	name: "R2D2"
	friendsByName: make(map[string]friend)
} // Compiles

Preallocated arrays and slices of `filled` types (or struct with filled fields)

We must provide an initial value when using make to preallocate the slice.

Expand this for more details

As filled types do not have zero value, you could not preallocate a slice/array of filled types with make:

// Which initial value would we use for these 10 elements?
pointersToInt := make([]filled *int, 10) 
peopleWithFilled := make([]PersonWithFilled, 10)

The easiest thing to do here is to forbid preallocating arrays of filled types with make.
You can always use the nillable equivalent type (and use nillable pointers in the case of a struct with filled fields)

I think this is something too restrictive and there is a way to overcome this. We can extend make to allow specifying the initial value for the slice elements with a function.

The previous example would be like this:

func initIntPointer() *int {
	return new(int)
}
pointersToInt := make([]filled *int, initIntPointer, 10) 


func initPersonWithFilled() PersonWithFilled {
	return PersonWithFilled {
		name: "R2D2"
		friendsByName: make(map[string]friend)
	}
}
peopleWithFilled := make([]PersonWithFilled, initPersonWithFilled, 10)
peopleWithFilled := make([]PersonWithFilled, initPersonWithFilled, 10, 20) // Version with capacity too

By using this new version of make, everything would compile correctly.
If this kind of initialization degrades the performance for huge slices (I guess it depends on how it is done internally, which is beyond my current expertise), then use the nillable equivalent type.

Converting a filled type to its nillable type

This has no problems. Following with the Go style, we should be explicit about the conversion:

var theAnswer filled *int = new(int)

var maybeTheAnswer *int = (*int)theAnswer

Converting a nillable type to its fillable type

We must provide an initial value when doing the conversion.

Expand this for more details

This is a bit more problematic, as we need a way to guarantee the value is not nil. We can go down two paths:

Let the compiler do static analyses to verify that we have checked previously that the value is not nil (for example, a previous if v != nil check). This seems too complicated
Provide a default value when doing the conversion

I think number 2 is simpler, more explicit and easier to understand for a person reading the code.

How to provide the default value?
The simplest and most Go-like way I have found is by providing an extra argument in the type conversion, like this:

var maybeTheAnswer *int = nil
var theAnswer filled *int = (filled *int)(maybeTheAnswer, new(int)) // maybeTheAnswer is nil, so we use the second argument to get the initial value.

A legibility improvement could be the following: if the filled type we want to convert to has the equivalent nillable type, then we can avoid specifying the nillable type:

var maybeTheAnswer *int = nil
var theAnswer filled *int = filled(maybeTheAnswer, new(int)) // It is OK

var theFloatAnswer filled *float64 = filled(maybeTheAnswer, new(float64)) // Not permitted types are different
var theFloatAnswer filled *float64 = (filled *float64)(maybeTheAnswer, new(float64)) // OK

A more complex example with functions:

type reducer func([]int) int

func adder(values []int) int {
	res := 0
	for _, value := range values {
		res += value
	}
	return res
}

// A simple function that takes a slice of ints and produces one int by applying the reducer passed
// As you can see, the type of "myReducer" is filled, so it can't be nil and, thus, "applyReducer" doesn't need
// to do the `if myReducer != nil` check nor can panic with a "nil pointer dereference"
func applyReducer(values []int, myReducer filled reducer) int {
	return myReducer(values)
}

func main() {
	values := []int{1,2,3,4}

	// The following compiles. "adder" is treated like an untyped constant, 
	// so it can be used here as "filled reducer"
	applyReducer(values, adder) 
	
	var myAdder reducer = adder

	// Does not compile: type "reducer" is not compatible with "filled reducer". 
	// Nothing new here
	applyReducer(values, myAdder) 

	// Does not compile: to convert from a nillable to a filled type you need to provide a default value
	applyReducer(values, filled(myAdder)) 

	applyReducer(values, filled(myAdder, nopReducer)) // Compiles
	
	var myFilledAdder filled reducer = adder
	applyReducer(values, myFilledAdder) // Compiles
}

func nopReducer([]int) int {
	return 0
}

Short variable declarations

In Go, when we use one of the forms of variable declarations where we don't specify the type (:= or var x =), the type is inferred from the right expression.

There will be nothing new here: if the right expression is of a filled type, then the variable would of that filled type.

If the right expression is of a nillable type and you want the variable to be of the equivalent filled type, then apply the conversion rules specified above.

Expand this for more details

var theAnswer := 42

// We need to provide the default value. Maybe it is clear here
// that &theAnswer is not nil, but take into account that it could come from a function call
filledPointerToTheAnswer := filled(&theAnswer, new(int)) 

type struct Pet {
	name string
}

filledPointerToPet := filled(&Pet {
	name: "Toby"
}, new(Pet)) // We need to provide the default value

myFilledFunc := filled(func(x int) int {...}) // No need to provide the default value here as it is like a "untyped constant"


func getTheFinalAnswer() filled *int {
	var answer filled *int = 42
	return answer
}

theFinalAnswer := getTheFinalAnswer() // Here, the type of theFinalAnswer is `filled *int` (as expected)

Maybe we could consider simplifying some obvious cases, like "filledPointerToTheAnswer" and "filledPointerToPet", and do not require a default value, but that is not a requirement for the proposal

Comparing filled types with `nil`

It doesn't compile. nil is not a possible value for filled types. It would be like comparing an "int" or "string" to nil.

Method set of filled pointer types

There is nothing unexpected. The method set of a filled pointer type filled *T is the set of all methods declared with receiver T, *T or filled *T.
In other words: the method set of filled *T is the same as *T plus those methods defined with filled *T receiver.

Expand this for more details

Put it in the form of a table:

A value of	Has the method set of
`T`	`T`
`*T`	`T` and `*T`
`filled *T`	`T`, `T`, and `filled T`

// The Dog type has methods defined on all kind of receivers
type Dog struct {}
func (Dog) woof() { /*...*/ }
func (*Dog) bark() { /*...*/ }
func (filled *Dog) yelp() { /*...*/ }

// Interfaces for each method
type woofer interface {
	woof()
}
type barker interface {
	bark()
}
type yelper interface {
	yelp()
}

// Functions that accept each defined interface
func doWoof(w woofer) { /*...*/ }
func doBark(b barker) { /*...*/ }
func doYelp(y yelper) { /*...*/ }

// Let's see it in action:
func main() {
	var dogStruct Dog
	// Direct calls:
	dogStruct.woof()// OK (current behavior)
	dogStruct.bark() // OK (current behavior). "bark" requires a pointer, but "dogStruct" is addressable
	dogStruct.yelp() // OK. "yelp" requires a filled pointer and the address of a struct is always filled
	// Interface satisfaction. "dogStruct" method set only contains "woof", so:
	doWoof(dogStruct) // OK (current behavior)
	doBark(dogStruct) // WRONG (current behavior). The "bark" method in "barker" interface is NOT defined in "Dog", but in "*Dog"
	doYelp(dogStruct) // WRONG. The "yelp" method in "yelp" interface is NOT defined in "Dog", but in "filled *Dog"

	var dogPointer *Dog
	// Direct calls:
	dogPointer.woof() // OK (current behavior). However, in this case this will throw a "nil pointer dereference" error
	dogPointer.bark() // OK (current behavior). However, in this case the method "bark" will receive a nil receiver
	dogPointer.yelp() // WRONG. "yelp" requires a filled pointer and "dogPointer" is not.
	// Interface satisfaction. "dogPointer" method set only contains "woof" and "bark", so:
	doWoof(dogPointer) // OK (current behavior)
	doBark(dogPointer) // OK (current behavior)
	doYelp(dogPointer) // WRONG. The "yelp" method in "yelper" interface is NOT defined in "*Dog", but in "filled *Dog"

	var filledDogPointer filled *Dog = &Dog{}
	// Direct calls:
	filledDogPointer.woof() // OK
	filledDogPointer.bark() // OK
	filledDogPointer.yelp() // OK
	// Interface satisfaction. "dogPointer" method set contains "woof", "bark", and "yelp" so:
	doWoof(filledDogPointer) // OK
	doBark(filledDogPointer) // OK
	doYelp(filledDogPointer) // OK
}

Compatibility with functionality that requires a zero value

I can only think of one case that can be problematic:

reflect package. There is a function to get the zero value of the type: reflect.Zero(<type>). If the type is filled, that function will panic. Of course, there would be an extra function to know if the type is filled (reflect.IsFilled(), etc.)

Backward compatibility

The only thing that could not be backward compatible is the addition of the new keyword filled. Maybe we could find a word that is less commonly used or a symbol (although I prefer a word). We can discuss this if this proposal moves forward.

A note about performance

Right now, the compiler needs to add a check for any nillable type so that, if it is nil, a panic is launched. This is not needed anymore with filled types so the check can be removed.
Would this mean that accessing to filled type would be more performant than to nillable type?

Alternatives

Click to show them

These are the alternatives that I discarded for some reason. However, they are useful for context and because maybe a person finds the solution to its drawbacks

A) Define zero values for filled types

In this variation, filled types do have zero value. They will be the following:

Type	Zero value	Comment
`filled *T`	new(T)
`filled map[K]V`	map[K]V{}
`filled func`	noop	So the dynamic zero value of a function does nothing and returns zero values. If the return values list finishes in `error`, then a default error is returned saying the function is a "no operation"
`filled chan T`	make(chan T)
`filled interface`	-	a default implementation where all methods are initialized with the `noop` function described above

This improves the current proposal in the following aspects:

It aligns with the philosophy of Go of "everything has a meaningul zero value".
It simplifies the filled type initialization and the make function for creating preallocated array/slices

However, I discarded it because of the following problems:

For pointers, maps and chan this works fine. However, for functions and interfaces, it is too magical and can hide incorrect behavior. For example, you might forget to initialize an interface and the application would not crash and would keep working as it was correct but just doing nothing when you call the interface methods. This is too dangerous.

B) Filled types is the default and we have nillable types

This would reverse the roles. For example, var p *int could not contain nil and it would be a compilation error if you assign nil to it. If you want it to contain nil, then you would need to use its corresponding nillable type: var p nillable *int.

As you might expect, this is absolutely no-backward compatible and will break practically all existing Go programs in a way that would be really hard to fix. There is no such value in this proposal that justifies that.

C) Remove nillable types and have only filled types

I thought that programs would be safer if no type can be nil. However, Go is pretty balanced and, in that balance, performance plays an important role. Sometimes it is useful to avoid allocating pointers or interfaces.
Another point against this alternative is that nil can be useful in a behavioral way. Not only nil channels can be leveraged, but also you can use a nillable type to "communicate" the intention that the value could contain nothing.

I think that having the two possibilities (nillable and non-nillable types) can bring even more value than only having non-nillable types.

Other similar proposals

I have been inspired by the following similar proposals:

The text was updated successfully, but these errors were encountered:

urandom · 2019-07-13T18:17:06Z

Maybe it will just be easier to make any pointer type implement an Option interface (Rust, Java, Swift, Scala, etc) and encourage people to use that instead of raw pointers, since the later can't realistically be disabled or it will break every program in existence. Seems the world has more or less settled on that type or similar for null safety.

nebiros · 2019-07-13T20:18:04Z

Maybe it will just be easier to make any pointer type implement an Option interface (Rust, Java, Swift, Scala, etc) and encourage people to use that instead of raw pointers, since the later can't realistically be disabled or it will break every program in existence. Seems the world has more or less settled on that type or similar for null safety.

You mean Optional, right?, swift does a pretty good job with optional types

alvaroloes · 2019-07-15T10:42:33Z

@urandom @nebiros Thanks for your comments.

Maybe it will just be easier to make any pointer type implement an Option interface (Rust, Java, Swift, Scala, etc) and encourage people to use that instead of raw pointers, since the later can't realistically be disabled or it will break every program in existence. Seems the world has more or less settled on that type or similar for null safety.

That's a good point of view, but if we allow the possibility of using the "nillable types" (I'm calling them this way instead of "pointer types" to leave it clear that we are not talking only about pointers) both through the Optional interface and directly, then we would lose compile-time safety, as they can still be nil and, thus, we should check them.

That's one of the reasons why I ended up adding a type modifier. Other reasons are related to adhering to the Go philosophy as much as possible (simplicity, readability, etc.)

For example, here are three examples:

A) Using filled types

func handleStockChangeEvent(product filled *Product, event filled *StockEvent) filled *Product {
    return &Product{
        ID: product.ID,
        Stock: product.Stock + event.StockIncrement,
    }
}

func main() {
    // ...
    newProduct := handleStockChangeEvent(product, event)
    fmt.Println("New product stock:", newProduct.Stock
}

B) Implicit Optional types; using them directly: There is nothing different here than using normal pointers:

// We need to change the signature so that we return the new Product and an error
// as, now, the parameters could be nil and, in that case, the function could not proceed.
func handleStockChangeEvent(product *Product, event *StockEvent) (*Product, error) {
    if product == nil {
        return nil, errors.New("product parameter is nil")
    }
    if event == nil {
        return nil, errors.New("event parameter is nil")
    }
    return &Product{
        ID: product.ID,
        Stock: product.Stock + event.StockIncrement,
    }
}

func main() {
    // ...
    newProduct, err := handleStockChangeEvent(product, event)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println("New product stock:", newProduct.Stock
}

C) Implicit Optional types; using them through the Optional interface

// We would still need to return a possible error, as the parameters can still be "empty". 
// However, for the shake of showing another option, let's assume that our program 
// doesn't care about the "empty" parameters case
func handleStockChangeEvent(product optional *Product, event optional *StockEvent) optional *Product {
    // Lets assume the "Java" interface for optionals. 
    // We could also think about the `if let` Swift version. 
    if product.empty() || event.empty() {
        return optional(nil) 
    }
    return &Product{
        // However, this `.get()` function can still panic if you didn't check for "nil" before
        ID: product.get().ID, 
        Stock: product.get().Stock + event.get().StockIncrement,
    }
}

func main() {
    // ...
    newProduct := handleStockChangeEvent(product, event)
    if newProduct.empty() {
        return
    }
    fmt.Println("New product stock:", newProduct.get().Stock
}

For me, the example A) is the one that provides more value:

· We have full "compiler assistance": You are completely sure that the parameters "are there" and that it is impossible to have a panic just by inspecting their values.
· You have one less thing to think about: you can just use the parameters without the need for checking for its emptiness. Optimizing the "thinking energy" is one of the most valuable things a technology could bring to you, as that energy can then be spent in solving the real problem.
· The code is simpler and, in my opinion, more Go-like. Although this should be decided by the whole community, I'd say.

Finally, regarding:

Seems the world has more or less settled on that type or similar for null safety

I would say that something refreshing about Go is that it didn't take for granted what the world was settled on (like object orientation, inheritance, exceptions, etc.). It rethought it and created something simpler (or removed it :-) )
Let's try to do the same thing for nils!

In conclusion: There are several ways of overcoming the "nil dereference errors", each one with its strengths and weaknesses. We need to find the one that fits best in Go, not only with its philosophy but with its current state of the art: the mindset of all the developers using it for years, backward compatibility with current programs, orthogonality with existing and planned functionality, etc.
This is the hardest part!

alvaroloes · 2019-07-22T12:42:22Z

I have made some updates to the proposal:

New section called "Method set of filled pointer types" added
New section "A note about performance" added
Collapsed the details of each section. This helps to understand the foundation and improves readability
Added a table of contents

Now the idea is to get some feedback, either positive or negative, especially to find out points that do not fit well in Go.
Thanks!

Nathan-Fenner · 2019-07-23T16:10:10Z

Zero values currently exist in quite a few places in the language. I think you need to address how filled types (which have no zero value) behave in each case.

For example, (map[int]filled*int{})[5]) would produce a zero value of type filled*int. Does this mean that you cannot use filled types as values in maps?

Similarly, slices can be resliced, exposing zero elements:

var x int

var slice []filled*int = []filled*int{&x, &x, &x}

for i := 0; i < 300; i++ {
    slice = append(slice, &x)
}
// at this point, cap(slice) > len(slice)
// (This is not guaranteed by the language, but is very likely)
slice = slice[:cap(slice)]

slice[len(slice)-1] // zero value!

Without allowing capacity to grow larger than length, the performance of append suffers substantially. Does this mean that indexing into a valid index in a slice of filled types can now panic? Or does growing the slice to its capacity do that?

Similarly, when reading from a closed chan filled *int, what do you obtain? Does this mean that you can't make channels of filled types?

The same issues apply whenever a filled type is used as a struct field.

Do you intend to allow implicit conversions from filled X to X when used in arguments or returns from functions? Otherwise, using filled types in user code will make calling filled-unaware library code inconvenient. As a general rule, I think casts should be looked upon with suspicion (since they usually indicate that you're breaking some abstraction moving between a custom type and its underlying abstraction). But if a library expects a *FooRequest and I have a filled *FooRequest I shouldn't need to write something so suspicion-inducing as a cast just to call that one function. Either the implicit cast should be permitted, or some new syntax should be used to indicate that the cast is safe and doesn't change semantics.

Lastly I think the method sets for filled pointer types might lead to some confusion when performing interface assertions.

type Fooer interface {
    Foo()
}

type Example struct{}
func (e filled *Example) Foo() {}

func example() {
    var blob interface{} = &Example{}

    fooer, ok := blob.(Fooer)
    // Is this ok or not? I have a non-nil *Example, so calling Foo should be safe!
}

I think this could lead to the same sort of confusion as "not-nil interfaces can still hold nil". Ensuring that this works in a way that is predictable and intuitive I think is important to preserving Go's readability.

I think it would be best to not distinguish between filled T and T when stored as interface values, and instead when performing a type assertion, check whether the value is nil to decide which method sets it possesses. This provides the best reverse-compatibility between existing filled-unaware code and new filled-aware code and has a low cost to understanding what's going on (a filled T is just a T that isn't nil).

ianlancetaylor · 2019-07-23T17:29:31Z

@Nathan-Fenner Thanks for the great analysis.

JelteF · 2019-07-23T21:17:01Z

I would definitely like to have non nillable types in Go (I even wrote an experience report). But @Nathan-Fenner makes some very good points. And I want to add to that with another reason why it currently doesn't fit well in Go: error handling. When there's an error right now you normally return a zero value as well. So what do you do when the return type is nonnil/filled? Do you make it nillable for every function that can return an error?

stevenblenkinsop · 2019-07-24T01:03:35Z

Another place nil values appear is in struct fields, meaning non-zero-ness would have to be contagious, or else such structs would themselves have to be marked as filled before any filled fields can be treated as non-zero.

Regarding make for slices, make([]int, 0, 10) is already a thing, meaning a slice with zero length and a capacity of ten, so you can't use that syntax to provide a default value. However, if you start with a zero length and then only grow the slice with append, then you don't need to specify a default value. The trick is getting the compiler to enforce this.

Another factor in designing this is that there are many places where Go has a contract based on multiple values. For example

func Foo1() (T, error) { ... }

Generally, the contract is that if the error is nil, then the T value will be initialized. However, if the caller needs filled T instead, they'd have to both check for an error and also then somehow assert that the T is initialized. Without having some way to encode this contract explicitly in the language, using this common Go pattern with filled types would be problematic. This also applies to the v, ok := ... pattern, which is also a first class feature of the language.

One way to allow non-filled values to be promoted to filled that would be relatively painless to use would be flow-aware typing. Basically, if you have if v == nil { return ... }, then after this statement v can be assigned to a filled type. To solve the slicing problem, the compiler would have to understand if i <= len(s) { s = s[x:i] } preserves the filled status of the elements of s, which seems a bit less explicit than you'd expect from Go.

Another way would be to support assertions, like y, ok := x.(filled T). In branches where ok is known to be true, y is known to be filled. This could also work for other comma-ok language features. Unfortunately, this wouldn't work with slices or structs containing filled types. Also, it doesn't really suggest a solution to the slicing problem.

This suggests a potential solution to the multi-value contracts problem. If Go had a couple built-in generic types, like result[T, U...] and optional[T] which were assignable to t, u..., err and t, ok respectively, then these could be used as return types to encode the contract that if err is nil or ok is true, then the other values in the assignment are initialized. Being able to use result in a generic context would require variadics, however, and the language team has not show an interest in supporting variadic generics.

Speaking of generics. The existence of types without zero values suggests that generic code would have to behave as if all type parameter types don't have zero values unless constrained by a some sort of zeroable bound. This complicates adding generics.

bcmills · 2019-07-24T01:30:00Z

Regarding slices and cap: given that index operations and append are so much more frequent than reslicing beyond len, it seems to me that the obvious solution is the same one that I mentioned in #24956 (comment): elements of a filled slice up to len must be initialized, and reslicing past cap should panic.

halmai · 2019-07-24T03:28:31Z

I think you wanted to write if x == nil instead of if x != nil. ;)

alvaroloes · 2019-07-25T18:06:00Z

@Nathan-Fenner
Thank you very much for such a nice critic showing the weak points of the proposal!
I will try to address each point one by one, as each one requires a lot of thinking, time and energy:

Regarding the zero value when accessing maps, slices, and channels of filled types

Good point! After thinking a lot, several ideas came to my mind but I think a suggestion by JHunz on Reddit is the one that suits better:

When declaring/creating a map, slice or channel of a filled type, you need to provide a function that provides a default value to be used in those situations where a zero value must be returned.

The syntax (one of the trickiest part in this case) for this would be consistent with the section "Converting a nillable type to its fillable type"

Here are examples for each container type and for all the different ways of initialization:

Maps
For example, a map of filled pointers:

func zeroIntPointer() filled *int {
	return new(int)
}

// With make
var m = make(map[string]filled *int, zeroIntPointer)
// With make and capacity
var m = make(map[string]filled *int, 100, zeroIntPointer)
// With map literal
var legs = 4
var m = map[string]filled *int{
	"Dog":     &legs,
	"Cat":     &legs,
	"Octopus": &legs,
}(zeroIntPointer)

Channels
For example, a channel of functions

func zeroProcessor() filled func() error {
	return func() error {
		return nil
	}
}

// With make
var c = make(chan filled func() error, zeroProcessor)
// With make, buffered channel
var c = make(chan filled func() error, 10, zeroProcessor)

Slices
For example, a slice of interfaces

type defaultStringer struct {}
func(defaultStringer) String() string {
	return ""
}

func zeroStringer() filled fmt.Stringer{
	return defaultStringer{}
}

// With make
var s = make([]filled fmt.Stringer, 3, zeroStringer)
// With literal
var s = []filled fmt.Stringer{a, b, c}(zeroStringer)

This is the most flexible way I have come up with. With this, we don't need to do any exceptions to the already-established pattern of returning zero-values from maps, slices, and channels when using filled types (which is one of the intentions of this proposal: try to be orthogonal and less disruptive as possible).

What is more, if Generics make it to the language one day, those functions to provide the default value could be greatly simplified, as even the standard library could provide generic implementations for some cases:

// This could be in the standard library
func zeroPointer(type T)() filled *T {
	return new(T)
}

// Then we could use it in the Maps example

There is always the alternative of forbidding having maps, slices, and channels of filled types. You can always use normal types in those cases. For me, this would be too restrictive.

Freeaqingme · 2019-07-25T18:35:39Z

@alvaroloes

// With make
var c = make(chan filled func() error, zeroProcessor)
// With make, buffered channel
var c = make(chan filled func() error, 10, zeroProcessor)

If we were to go for zero processors, I'd suggest to at least keep the arguments always the same. So argument 2 is always capacity. If you want to declare a non-buffered channel, one would do:

// With make (non-buffered)
var c = make(chan filled func() error, 0, zeroProcessor)

ianlancetaylor · 2019-10-01T22:00:20Z

Thanks for the extensive proposal. It would be nice to have ways to make the language safer. But this approach seem out of place with how Go works today. Go currently has only one rarely-used type qualifier (send-only and receive-only channels). If people started using filled or some shorter equivalent, it would likely appear all over Go programs. That would give the language a different feel.

The notion of the zero value of a type is built into the language at a pretty deep level, as @Nathan-Fenner discusses in the comment above. Adding additional arguments to make to pass in a function to return a non-zero value is a complex workaround.

We would also need additional functions in the reflect package, to return initialized values of filled types.

All of this additional complexity only helps us avoid some programming errors, at least some of which can be detected by static analysis. Is this change worth the cost?

alvaroloes · 2019-10-02T11:08:29Z

Thanks for the comment @ianlancetaylor . Yes, I'm aware of the added complexity and it is something I don't like much, but I haven't had time lately to simplify the proposal.
This is my third attempt to deal with nils, but I know it needs some more iterations to make it suitable for Go.

For me and other folks I know, forgetting about nil issues is something that adds a lot of value (for the reasons stated in the "Introduction and justification" section). I remember, when I started with Go, loving the fact that strings are never nil, unlike other languages. I wish all the types work in the same way.

Some quick comments about some of your sentences:

If people started using filled or some shorter equivalent, it would likely appear all over Go programs. That would give the language a different feel.

Yes, I thought about this. My first attempt was to make all nilable types "filled" by default, and then you would need to use the modifier nilable if you want that type to contain nil. This will make the modifier to appear rarely.
But this will break backward compatibility. This is the reason I opted for the opposite behavior (nilable by default, as it is now)

The notion of the zero value of a type is built into the language at a pretty deep level, as @Nathan-Fenner discusses in the comment above. Adding additional arguments to make to pass in a function to return a non-zero value is a complex workaround.

I know! To be honest, I'm not proud of that workaround.

Once I get some time, my idea is to explore the zero value "issue" in more detail and see if there is something we can do to make the zero value of all the current "nilable" types useful (without being nil), in the same way as the zero value of strings or slices.

ianlancetaylor · 2019-10-15T21:50:24Z

For the reasons given above at #33078 (comment), this is a likely decline.

There is probably something we can do in this area, based on static checking, and perhaps annotations and dynamic checking. Complicating the type system doesn't seem like the right approach for Go.

Leaving open for four week for further comments.

alvaroloes · 2019-11-04T13:08:34Z

All right.
Right now I don't have more spare time to spend on this proposal, so feel free to close it once the final comment period finishes.

Thanks for your time

bradfitz · 2019-11-12T22:13:35Z

Right now I don't have more spare time to spend on this proposal, so feel free to close it once the final comment period finishes.

Will do.

gopherbot added this to the Proposal milestone Jul 12, 2019

gopherbot added the Proposal label Jul 12, 2019

alvaroloes mentioned this issue Jul 12, 2019

Proposal: Panics returned as errors if the function has an error result type #32871

Closed

ianlancetaylor added v2 A language change or incompatible library change LanguageChange labels Jul 12, 2019

griesemer mentioned this issue Oct 1, 2019

proposal: Go 2 review meeting minutes #33892

Open

ianlancetaylor added the Proposal-FinalCommentPeriod label Oct 15, 2019

bradfitz closed this as completed Nov 12, 2019

golang locked and limited conversation to collaborators Nov 11, 2020

gopherbot added the FrozenDueToAge label Nov 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

proposal: "Filled types": a mechanism to guarantee types are not nil at compile time #33078

proposal: "Filled types": a mechanism to guarantee types are not nil at compile time #33078

alvaroloes commented Jul 12, 2019 •

edited

A) Define zero values for filled types

B) Filled types is the default and we have nillable types

C) Remove nillable types and have only filled types

urandom commented Jul 13, 2019

nebiros commented Jul 13, 2019

alvaroloes commented Jul 15, 2019

alvaroloes commented Jul 22, 2019

Nathan-Fenner commented Jul 23, 2019

ianlancetaylor commented Jul 23, 2019

JelteF commented Jul 23, 2019

stevenblenkinsop commented Jul 24, 2019 •

edited

bcmills commented Jul 24, 2019

halmai commented Jul 24, 2019

alvaroloes commented Jul 25, 2019

Freeaqingme commented Jul 25, 2019

ianlancetaylor commented Oct 1, 2019

alvaroloes commented Oct 2, 2019 •

edited

ianlancetaylor commented Oct 15, 2019

alvaroloes commented Nov 4, 2019

bradfitz commented Nov 12, 2019

proposal: "Filled types": a mechanism to guarantee types are not nil at compile time #33078

proposal: "Filled types": a mechanism to guarantee types are not nil at compile time #33078

Comments

alvaroloes commented Jul 12, 2019 • edited

Table of contents

Introduction and justification

Syntax

Proposal description

Filled types don't have zero value

Preallocated arrays and slices of filled types (or struct with filled fields)

Converting a filled type to its nillable type

Converting a nillable type to its fillable type

Short variable declarations

Comparing filled types with nil

Method set of filled pointer types

Compatibility with functionality that requires a zero value

Backward compatibility

A note about performance

Alternatives

A) Define zero values for filled types

B) Filled types is the default and we have nillable types

C) Remove nillable types and have only filled types

Other similar proposals

urandom commented Jul 13, 2019

nebiros commented Jul 13, 2019

alvaroloes commented Jul 15, 2019

alvaroloes commented Jul 22, 2019

Nathan-Fenner commented Jul 23, 2019

ianlancetaylor commented Jul 23, 2019

JelteF commented Jul 23, 2019

stevenblenkinsop commented Jul 24, 2019 • edited

bcmills commented Jul 24, 2019

halmai commented Jul 24, 2019

alvaroloes commented Jul 25, 2019

Regarding the zero value when accessing maps, slices, and channels of filled types

Freeaqingme commented Jul 25, 2019

ianlancetaylor commented Oct 1, 2019

alvaroloes commented Oct 2, 2019 • edited

ianlancetaylor commented Oct 15, 2019

alvaroloes commented Nov 4, 2019

bradfitz commented Nov 12, 2019

alvaroloes commented Jul 12, 2019 •

edited

Preallocated arrays and slices of `filled` types (or struct with filled fields)

Comparing filled types with `nil`

stevenblenkinsop commented Jul 24, 2019 •

edited

alvaroloes commented Oct 2, 2019 •

edited