Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: encoding/json: allow returning nil from MarshalJSON to omit the field #50480

Closed
mitar opened this issue Jan 6, 2022 · 10 comments
Closed

Comments

@mitar
Copy link
Contributor

mitar commented Jan 6, 2022

Currently even if one implements MarshalJSON on a struct there is no way to prevent its inclusion into the parent struct. E.g.:

type Foo struct {
	Field Field `json:"field"`
}

type Field struct {
	DataPublic bool
	Value      interface{}
}

func (f Field) MarshalJSON() ([]byte, error) {
	if f.DataPublic {
		return json.Marshal(f.Value)
	} else {
		return []byte("null"), nil
	}
}

(Full example: https://go.dev/play/p/A0wKfuZgURw)

The best one can do is return null. But what if one wants to fully omit the field? This is not possible. Or one has to implement MarshalJSON on the parent struct, but that is not nice from composability perspective (one would have to implement this for every struct which includes Field).

I propose that it should be allowed to return nil, nil from MarshalJSON to signal that the field should be omitted in the parent struct. Currently returning nil, nil does fails with error unexpected end of JSON input because the resulting JSON is malformed. That means that nobody can rely on this behavior really for correct JSON output (maybe only to catch errors?), so I do not think we would be breaking any real backwards compatibility if we introduce this, or at least I believe benefits are bigger here. In any case, if you want previous behavior, you can return []byte{} and then it will continue to error in the same way. Doing JSON marshal of a struct itself which returns nil is compatible enough in my view: if caller just assumes they are getting []byte slice, nil will mostly behave for them as an empty slice. If they are able to handle it anyway.

Implementing this would also provide a solution for the issue that omitempty does not work with structs: one could implement MarshalJSON which checks if the value is zero and return nil if it is so. This would be useful only when implementor of the type is the user of it at the same time, but one can always make a new type as a user and then implement MarshalJSON there. I think it is an useful step in addressing that.

Returning nil to omit the field seems to be also something others expected to work and has been also proposed in the past, but inside a bigger discussion, so this can serve as a proposal for just this feature.

@mitar mitar added the Proposal label Jan 6, 2022
@gopherbot gopherbot added this to the Proposal milestone Jan 6, 2022
@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) Jan 7, 2022
@ianlancetaylor ianlancetaylor changed the title proposal: encoding/json: Allow returning nil from MarshalJSON to omit the field proposal: encoding/json: allow returning nil from MarshalJSON to omit the field Jan 7, 2022
@earthboundkid
Copy link
Contributor

I worry this could result in an accidental omission. How about return nil, json.OmitFieldErr instead?

@LittleFox94
Copy link

What is the state for this, would a PR for this be accepted?

I really like the return nil, json.OmitFieldErr idea, though it should be either ErrOmitField to match conventions or strip the Err completely - maybe json.OmitFieldHint or something?

@LittleFox94
Copy link

I just saw this comment is created from one of the ideas #11939, explicitly for return nil, nil and return nil, json.$someErr is one of the other proposals in that original issue.

@icholy
Copy link

icholy commented Apr 30, 2022

@mitar what happens if MarshalJSON returns nil without omitempty.

type Nil struct{}

func (Nil) MarshalJSON() ([]byte, error) {
	return nil, nil
}

func main() {
	data, _ := json.Marshal(Nil{})
	fmt.Println(string(data)) // what's the expected output?
}

@icholy
Copy link

icholy commented Apr 30, 2022

@mitar I implemented your idea here https://github.com/icholy/json if you want to play with it https://go.dev/play/p/7Cf6BgrNGpP

edit: I've also implemented the err_empty branch which requires the error return. Ex:

type MarshalNil struct{}

func (MarshalNil) MarshalJSON() ([]byte, error) {
	return []byte("null"), json.ErrEmpty
}

This type will marshal to null unless it's a struct field with omitempty set.

@smikulcik
Copy link

Regarding #50480 (comment)

Wouldn't nil, nil yield the invalid JSON of empty string?

@rsc
Copy link
Contributor

rsc commented Jun 1, 2022

This proposal is a duplicate of a previously discussed proposal, as noted above,
and there is no significant new information to justify reopening the discussion.
The issue has therefore been declined as a duplicate.
— rsc for the proposal review group

@rsc rsc closed this as completed Jun 1, 2022
@mitar
Copy link
Contributor Author

mitar commented Jun 1, 2022

Sorry, where exactly has this already been discussed? So I found a mention in one long issue (on omitempty and structs), a much broader issue, where this was proposed but not acted upon, while this issue itself tries to be focused on exactly one proposal. So which discussion is this reopening? This is just trying to focus the discussion which have not happened. (And this issue has also 14 upvotes.)

@joeshaw
Copy link
Contributor

joeshaw commented Jun 1, 2022

@mitar #45669 is the one that's been put on the docket for this.

There are lots of different proposals to address the same underlying problem, but it's not clear yet which is the best approach to take. Hopefully discussion there will address it.

@mitar
Copy link
Contributor Author

mitar commented Jun 1, 2022

Hm, I saw that one, but I disagree that this one would be addressed with omitzero. But the opposite holds true: implementing this proposal would also provide a way to solve the issue which omitzero attempts to address. But omitzero (nor omitempty) do not influence JSON marshaling when struct implements MarshalJSON. And this issue is about that. If a struct has MarshalJSON, then you cannot omit it anymore. omitzero will not fix this.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
No open projects
Development

No branches or pull requests

8 participants