Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: Allow differentiating between a nil string and an empty string #44219

Closed
eacp opened this issue Feb 11, 2021 · 14 comments
Closed

proposal: Allow differentiating between a nil string and an empty string #44219

eacp opened this issue Feb 11, 2021 · 14 comments
Labels
Milestone

Comments

@eacp
Copy link

eacp commented Feb 11, 2021

Currently, there is no diferrence between declaring a string variable without initiallizing it and declaring an empty string

var a string
b := ""

a == b // True

There is no way to know if the string was purpously set to an empty string, or was not actually
assigned for some reason. This could be problematic for JSON encoding and decoding.

Stripe encountered this issue and resolved it by requiring the users of the Stripe SDK to use a
custom string pointer type, and the library user ends up writing a lot boilerplate code.

Proposal

Given the Go string is actually implemented by an array at internally the string type uses a pointer, we could create a built in function (or method) to detect if said array (or pointer) is either nil, or some allocated space of size zero. It could look like this

var a string
b := ""

isNil(a) // true
isNIl(b) // false

// Another aporach would be to check if it was initialized

initialized(a) // false
initialized(b) // true

This proposal requires work, but it could be potentially helpful to some users, but I welcome your feedback 😄

@gopherbot gopherbot added this to the Proposal milestone Feb 11, 2021
@davecheney
Copy link
Contributor

This could be problematic for JSON encoding and decoding.

could you explain this a bit more? What problems? Could give some examples?

@robpike
Copy link
Contributor

robpike commented Feb 11, 2021

There is no such thing as a nil string. Truly. You will find that

var  s string

and

s := ""

Both have a nil pointer and are therefore truly indistinguishable. You're asking for a fundamental change in the language. It's not going to happen.

@seankhliao seankhliao added LanguageChange v2 A language change or incompatible library change labels Feb 11, 2021
@eacp
Copy link
Author

eacp commented Feb 11, 2021

This could be problematic for JSON encoding and decoding.

could you explain this a bit more? What problems? Could give some examples?

Of course

Say I receive the following input from an http client

{“title”: “Example”, “desc”: “”}

And I also have this input

{“title”: “Example 2”}

And I have a struct with a Title and a Desc string field. I decode the input using the json lib into a struct.

json.NewDecoder(r).Decode(&data)

How do I know if the user provided an empty description, or if the user did NOT provide a description?

Another use case could be writing and reading using the SQL package, because you can have an empty string in a db, and an empty string in a DB (unless you mark its col as not null)

@eacp
Copy link
Author

eacp commented Feb 11, 2021

Thank you so much for answering. Should I close the issue?

@ianlancetaylor
Copy link
Contributor

I can see two possible meanings here.

One is changing the zero value of the string type, such that the empty string "" is not the zero value. That would be a significant change to the language that would likely break many existing working programs. We aren't going to do that.

The other possible meaning is that we keep "" as the zero value of string, such that after var s string the expression s == "" remains true, but that we record in a string variable whether it was ever set to some value. That seems like a very subtle and error-prone distinction to make. I think it would lead to confusion along the lines of https://golang.org/doc/faq#nil_error. So I think that also seems unlikely to be adopted.

The usual way to distinguish a string that was never set from a string that was set to "" is to use a pointer of type *string. Then the value nil indicates that the value was never set, and a non-nil value holds the string value if it was set.

@jfesler
Copy link

jfesler commented Feb 11, 2021

We have this problem with all types and distinguishing their zero values. ie integer 0, boolean false. Struct pointers are a good signal. Always check for nil before using! Pretty straight forward - if tedious.

For marshalling, it's a bit tougher; we don't have a good json tag for "omit nil" (vs "omit empty"). And even then, sometimes you want the explicitly set nil pointer emitted (generating JSON for PATCH requests) to clear a field. I'm aware of no good options other than making wrappers for all struct types, that can indicate whether a field was set, or an unset nil, or explicit nil.

@D1CED
Copy link

D1CED commented Feb 11, 2021

With the now accepted generics we could introduce an Option[T] type into the standard library that would cover this use case.

They would be similar in spirit to the Null* types in database/sql.

@zephyrtronium
Copy link
Contributor

With the now accepted generics we could introduce an Option[T] type into the standard library that would cover this use case.

Perhaps off-topic, but this is false. Type parameters do not give us sum types. Option[T] would require the caller to specify whether the option is Some or None, at the time of instantiating the type. Later code, e.g. the bodies of functions parameterized by Option[T], cannot change which variant is used.

@randall77
Copy link
Contributor

@zephyrtronium I think @D1CED was suggesting

type Option[T any] struct {
   Value T
   Present bool
}

You don't need to choose presence when instantiating the type, but when making an instance.

@beoran
Copy link

beoran commented Feb 12, 2021

And the standard library database/sql solves this problem by having a host of NullXXX types that are not generic but have a bool field to record whether they are set or not. https://golang.org/pkg/database/sql/

@eacp
Copy link
Author

eacp commented Feb 12, 2021

And the standard library database/sql solves this problem by having a host of NullXXX types that are not generic but have a bool field to record whether they are set or not. https://golang.org/pkg/database/sql/

This is a very interesting approach. We could eventually land at a generic (parametric type) Optional[T] defined as

type Option[T any] struct {
   Value T
   Present bool
}

like @randall77 suggests. Interestingly, the language could add some sugar and use a special
symbol as some sort of shorcut to the Option generic, like so

var example string? // Equivalent to Option[string]

and allow the retrieval of the data with sugar. If the data type is a primitive, then the compiler would just
the value field. So example+" something" would evaluate to example.value + something

But that would be another proposal, and I dont know if it would fit well with the spirit of the lang

@ianlancetaylor
Copy link
Contributor

Based on the discussion above, and the lack of support via emoji voting, this is a likely decline. Leaving open for four weeks for final comments.

@kevlar700
Copy link

kevlar700 commented Mar 3, 2021

Performance has caused so many security issues. Personally I see nil as an unused marker in JSON, as an inferior API design. I use an additional bool, when needed. Or a marker. After all, even a name that apparently could be anything, will never actually be a long enough UUID. Crypto often relies on this property. This way, it is obvious and could even be easily detected when something is nil, when it was never meant to be.

Personally I now use a function that checks err for nil and returns err.Error() or "". Avoiding panic potential, for the rare cases where you might want your api to return err as a string, whether nil or not.

I would like to see less nils if anything.
#30208

@ianlancetaylor
Copy link
Contributor

No change in consensus.

@golang golang locked and limited conversation to collaborators Apr 6, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests