New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
encoding/json: multiple anonymous struct fields shouldn't be ignored #11006
Comments
This is the documented behavior of encoding/json, so we http://golang.org/pkg/encoding/json/#Marshal The Go visibility rules for struct fields are amended for JSON when
Handling of anonymous struct fields is new in Go 1.1. Prior to Go 1.1, You can add struct tags to the embedded fields to achieve what |
@minux The issue is about the multiple anonymous fields that are ignored due the rule# 3. The tags have their place but I think that by default the encoder should not ignore these fields. It looks wrong to me to ignore fields/loose data by default. I think they should be encoded and if the developer doesn't like the output it can use tags to alter it(i.e ignore them) just like it does for non-multiple-anonymous fields. Let me know if this makes sense to you. |
Unfortunately, according to Go 1 api guarantee, we can't
change the behavior of encoding/json now.
|
I think the proposal is backward compatible. It provides/encodes fields that were otherwise ignored(the multiple anonymous fields) but it doesn't change the encoding for the other fields. It's basically one step ahead after the change in Go 1.1. |
I could try a CL if the issue is accepted. I might be wrong but I don't see any reason to ignore the multiple anonymous fields unless there is a technical issue (i.e. the performance), even though I don't think the correctness of the encoder and the developer time should be sacrificed for a small performance gain. |
It's not that multiple anonymous struct fields are ignored.
Only the conflicting fields are ignored.
http://play.golang.org/p/fJlUlMKst8
Your proposed change is not backward compatible, because
if people are relying on the fact that Name and Address fields
are not marshaled, they will now have to add explicit `json:"-"`
build tags to achieve the same result. This is a behavior change.
|
The issue is about the conflicting(aka duplicate) multiple anonymous fields which falls under the special rule # 3. I thought it's enough to specify it at the beginning of the thread. Concerning the breaking change I'm wondering how is this different than the change in go 1.1. Was that a breaking change too? It looks same to me. People relying on the fact that the anonymous fields were not marshalled had to to add explicit [0] |
There are provisions for the Go 1.1 change.
If you read the doc of encoding/json in Go 1.0, you will
find this listed in the BUGS section:
This package ignores anonymous (embedded) struct fields during encoding
and decoding. A future version may assign meaning to them. To force an
anonymous field to be ignored in all future versions of this package, use
an explicit `json:"-"` tag in the struct definition.
|
That makes sense but it begs the same question: how is the change I propose a breaking change? It just adds meaning to anonymous (embedded) struct fields that are currently ignored(the "conflicting" ones), isn't it? |
It's a reversal of the existing documented behavior, why
is that not a breaking change?
|
I fail to see how marshalling the "conflicting" fields would cause a breaking change. |
OK, what should objects of type T be marshaled?
type Type1 struct {
Secret, A string
}
type Type2 struct {
Secret, B string
}
type T struct {
Type1
Type2
}
In the current rules, it will be:
{ "A": "a", "B":"b" }
The developers might rely on the fact that the two Secret fields
are ignored as per the docs, so they don't bother to put the -
struct tag.
|
@minux If they assume that fields without |
Trying to find an analogy I imagine what would be if the "conflicting" fields would not be exported in a Go program (unless you use tags; the behaviour we currently have on the json package). That would pretty much suck isn't it? I think it's the same case for the encoding process as the conflicting fields are not encoded unless you use tags, thus it makes the development of JSON enabled services more laborious and prone to errors. |
They didn't assume anything. The docs explicitly said those
fields are ignored. Note the docs doesn't say it's temporarily
ignored and future version might change, unlike the docs
for Go 1.0. Thus I think it's ok to rely on this.
|
I can read this[0] on the current go documentation. I think it makes a clear statement for the fields you want to ignore, such sensitive information. Relaying on an implementation detail to avoid data leaking doesn't sounds right to me nor I believe people responsible(i.e. sane) with sensitive data do that. Not to mention that the behaviour is unstable and prone to errors because if one conflicting field is removed the data is leaked from the other one.
|
It's not about right or wrong. The behavior is documented,
so people can rely on it. and that it in turn means we can't
change it.
|
Ok then...unless additional functions in Go 1.x (i.e. MarshalExtended) with a different behaviour to handle the conflicting fields could be considered, can label this as an issue for Go 2.0? |
I've ended-up with a generator which rewrites the structs I'm using for encoding and adds tags automatically inferred from the type name or type + pkg id where so-called "conflicts" exists. |
Currently if there are duplicate anonymous struct fields they are all ignored. I think this is wrong because the encoding process shouldn't loose information unless the output format is not capable to represent it(which is not the case).
For example marshalling AllTypes produces an empty output.
type AllTypes struct {
Type1
Type2
}
type Type1 struct {
Name string
Address string
}
type Type2 struct {
Name string
Address string
}
To overcome this issue an additional step is required to create a boilerplate type specially for the encoding/decoding process. Most likely we end-up with a new type (AllTypes2) as below. This is against the composability of the language and adds additional burden to the developer(look for duplicate fields -> create a new type).
type AllTypes2 struct{
Type1 Type1
Type2 Type2
}
Any fix/convention that preserves the data would be fine to me but I present my proposal below too as starting point. I think it's also backward compatible.
Use the same addressing like the native go types.
If there are duplicate anonymous fields, they should be addressed using the type name as prefix. If the type name is duplicate too both the package name and the type name should be used as prefix. For type AllTypes I would expect the output below:
{
"Type1": {"Name":"example", "Address": "example"},
"Type2": {"Name":"example", "Address": "example"}
}
Using the same addressing as on go types makes the behaviour intuitive and easier to understand too as we have only one familiar rule instead of 3 special rules.
It would make the behaviour more consistent too because the type name is already used as key on certain cases. For example type SomeType is encoded to {"MySlice":["somedata"]}.
type SomeType struct{
MySlice
}
type MySlice []string
The text was updated successfully, but these errors were encountered: