Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: decoding JSON into anonymous struct works in go 1.8.7 but fails silently in go 1.9.4 #23861

Closed
DonkeyOatie opened this issue Feb 15, 2018 · 9 comments
Milestone

Comments

@DonkeyOatie
Copy link

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.8.7 and 1.9.4

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOOS="darwin"

What did you do?

When decoding json into an anonymous structure, the data is present when using an app built with go version 1.8.7 but empty when using go 1.9.7

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

https://play.golang.org/p/vPsyOX7wBea

Here is an example of how I discovered this using the req.Body, details in the README show the difference between the go 1.8 and 1.9 output. With help from eric_hill on IRC #go-nuts we narrowed it down the the above play link.

https://github.com/nkhumphreys/goreqbody

What did you expect to see?

localStr should be populated with the data decoded from the JSON

What did you see instead?

localStr was empty

@bradfitz
Copy link
Contributor

With help from eric_hill on IRC #go-nuts we narrowed it down the the above play link.

You two both forgot to check for errors.

That might not necessarily matter, but it's a distracting bug report, since everybody will immediately assume the answer is in the ignored error.

@DonkeyOatie
Copy link
Author

DonkeyOatie commented Feb 15, 2018

I can update the play link and check the error if you like, but there is not one. :)

https://play.golang.org/p/3ZxteJYqZMp

@bradfitz
Copy link
Contributor

Well, we can investigate, but there's no fixing Go 1.9 at this point, and Go 1.10 is out, like, today, so probably not changing there either.

I see nothing in the Go 1.9 release notes about json or reflect.

Please test new Go releases early & often! :)

@bradfitz bradfitz added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 15, 2018
@bradfitz bradfitz changed the title decoding JSON into anonymous struct works in go 1.8.7 but fails silently in go 1.9.4 encoding/json: decoding JSON into anonymous struct works in go 1.8.7 but fails silently in go 1.9.4 Feb 15, 2018
@bradfitz bradfitz added this to the Go1.11 milestone Feb 15, 2018
@DonkeyOatie
Copy link
Author

Just an FYI: The same behaviour as go 1.9 is exhibited in go 1.10rc2

@pciet
Copy link
Contributor

pciet commented Feb 16, 2018

An exported field works: https://play.golang.org/p/0kdRTJY_Xp5

Capitalizing the type works: https://play.golang.org/p/4szuu5Zydg3

From https://golang.org/pkg/encoding/json/#Marshal:

Anonymous struct fields are usually marshaled as if their inner exported fields were fields in the outer struct, subject to the usual Go visibility rules amended as described in the next paragraph.

@0xmohit
Copy link
Contributor

0xmohit commented Feb 16, 2018

The behavior changed in b817359.

@bradfitz
Copy link
Contributor

Hey @dsnet , is that expected? A pointer to a string is considered unexported?

@dsnet
Copy link
Member

dsnet commented Feb 16, 2018

Yes, this is expected behavior.

According to the language spec:

An identifier may be exported to permit access to it from another package. An identifier is exported if both:

  1. the first character of the identifier's name is a Unicode upper case letter (Unicode class "Lu"); and
  2. the identifier is declared in the package block or it is a field name or method name.

All other identifiers are not exported.

Thus, embedding a builtin type results in a field name that is not exported (since all builtin have lowercase names). Since this is unexported, the json package documents that it does not handle this.

The json rule you quote does not apply here:

Anonymous struct fields are usually marshaled as if their inner exported fields were fields in the outer struct, subject to the usual Go visibility rules amended as described in the next paragraph.

The example you provided has an anonymous string, not a struct.

@dsnet dsnet closed this as completed Feb 16, 2018
@dsnet
Copy link
Member

dsnet commented Feb 16, 2018

Furthermore, in Go1.10 and on, it wouldn't be possible for json to even handle this without using unsafe since reflect reports this field as unexported.

The reason why reflect on Go1.9 and below could actually mutate this field was due to a compiler bug, fixed by @mdempsky in CL/60410.

@dsnet dsnet removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. help wanted labels Feb 16, 2018
@golang golang locked and limited conversation to collaborators Feb 16, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

6 participants