proposal: encoding/json: allow struct encoder to handle undefined fields #64515
Comments
|
what's wrong with using a pointer? |
seankhliao
changed the title
Firstly, declaring a variable as pointer is prone to null pointer dereference bugs. Declaring a variable as non-pointer and examining whether to include that variable into JSON output is the safer. Also, IsDefined() approach can give developers full control over whether to include that variable into JSON output, unlike "omitempty" flag which can give developers control of only zero value of limited data types. Secondly, asking developers to use pointers only for controlling whether a field of a struct should be included into JSON output is boring, for golang does not allow directly assiging constant values / fuction return values into a pointer variable like and therefore have to do like . On the other hand, is easier to write. |
|
Additional information: Since the API client (JavaScript programs) treats var1 === 0 and var1 === null and var1 === undefined differently, I need to be able to handle all three cases listed below.
Since a field of a struct declared as "*int32" with "omitempty" cannot handle case 2, I need to implement MarshalJSON() using a struct. That is, in the previous comment will actually be and reading/writing these fields safely will become more complicated. Therefore, I want to declare struct without using pointer and allow struct encoder to control whether to include each field. |
|
Hmm, it seems that we can avoid defining a "struct" if we use "**int32" and wrapper functions for hiding temporary variables. I'm not sure which approach is more friendly to developers. |
|
From the motivating example it seems like you want to be able to represent both "property totally absent" and "property present but set to One notable gap in the current design -- unless I've just been missing it all these years! -- is a way for a Personally I tend to think that type MarshalerEmpty interface {
Marshaler
MarshalJSONEmpty() bool
}The You could then implement your "value that might be null or absent" concept in separate library code. For example: type JSONValue[T any] struct {
value T
defined bool
null bool
}
func (v JSONValue[T]) MarshalJSON() ([]byte, error) {
if !(v.defined && v.null) {
return []byte(`null`), nil
}
return json.Marshal(v.value)
}
func (v JSONValue[T]) MarshalJSONEmpty() bool {
return !v.defined
}Since the concept of "omitempty" is not applicable when a value appears somewhere other than directly in a struct field, This is, of course, essentially an entirely different proposal for getting the same effect. After some searching in the history, I think I've essentially just written up a JSON-specific version of #11939, and so if this seems promising then probably better to continue discussion over there rather than here, I suppose. |
Yes.
Yes, that limitation is the problem. I tried an example (shown below) that makes use of "map[bool]" instead of "struct" so that isEmptyValue() test is applied to pseudo-struct. Do you think that this approach make sense? |
|
Using a single-element map which can be nil to represent absence is a clever trick! |
|
Thank you. Based on your example, I tried below version and it seems working as expected. I wish that json package provides a global flag for configuring encOpts{} for json.Marshal() because my API needs to use escapeHTML = false. Then, I will be able to use json.Marshal() instead of using json.NewEncoder() and calling SetEscapeHTML(false)... |
This bootstraps a new repository for the `oapi-codegen` organisation's standards, and then implements the `Nullable` type as per [0] and [1]. We can make sure this is a multi-module project, similar to other projects, so we can isolate test-only dependencies from the core project, which has zero dependencies. In the top-level project we can use runnable examples to indicate the example usage and cover all the test cases we need, and then use the `internal/test` package to perform further checks. Co-authored-by: Sebastien Guilloux <sebastien.guilloux@elastic.co> Co-authored-by: Ashutosh Kumar <ashutosh.kumar@elastic.co> [0]: golang/go#64515 (comment) [1]: https://github.com/sebgl/nullable/
This bootstraps a new repository for the `oapi-codegen` organisation's standards, and then implements the `Nullable` type as per [0] and [1]. We can make sure this is a multi-module project, similar to other projects, so we can isolate test-only dependencies from the core project, which has zero dependencies. In the top-level project we can use runnable examples to indicate the example usage and cover all the test cases we need, and then use the `internal/test` package to perform further checks. Co-authored-by: Sebastien Guilloux <sebastien.guilloux@elastic.co> Co-authored-by: Ashutosh Kumar <ashutosh.kumar@elastic.co> [0]: golang/go#64515 (comment) [1]: https://github.com/sebgl/nullable/
This bootstraps a new repository for the `oapi-codegen` organisation's standards, and then implements the `Nullable` type as per [0] and [1]. Using a `map` as the underlying type allows us to take advantage of `json.Marshal`'s inbuilt checks to determine whether to `omitempty` a JSON value, which isn't possible with a `struct`. We can make sure this is a multi-module project, similar to other projects, so we can isolate test-only dependencies from the core project, which has zero dependencies. We can also add convenience helpers for `NewNullableWithValue` and `NewNullNullable` as they can be useful when constructing `struct`s in tests. In the top-level project we can use runnable examples to indicate the example usage and cover all the test cases we need, and then use the `internal/test` package to perform further checks. Co-authored-by: Sebastien Guilloux <sebastien.guilloux@elastic.co> Co-authored-by: Ashutosh Kumar <ashutosh.kumar@elastic.co> [0]: golang/go#64515 (comment) [1]: https://github.com/sebgl/nullable/
ianlancetaylor
changed the title
Proposal Details
I'm aware of #63397, but here is a simple approach which I want to use.
I want to use "struct" for defining API responses. But the API is so complicated that some of fields in that "struct" are conditionally initialized (i.e. needs to hide fields when values are not assigned). But due to golang's zero values, I can't hide fields in that "struct".
As a result, I'm currently using "map[string]any" instead of "struct" in order to make sure that only fields that have values defined (they might be golang's zero values) are included in the API responses. But since "struct" is more easy to understand, I want to make it possible to hide fields in a "struct" when values are not assigned.
The diff for json package and an example usage of this proposal are shown below.
The text was updated successfully, but these errors were encountered: