encoding/json: Unmarshal & json.(*Decoder).Token report different values for SyntaxError.Offset for the same input #34543
Labels
help wanted
NeedsInvestigation
Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Comments
bcmills
changed the title
bcmills
added
help wanted
NeedsInvestigation
babolivier
added a commit
to babolivier/go
that referenced
this issue
Oct 1, 2019
…nter of its internal scanner Decoder.Token() goes through the bytes of the JSON payload and return the next JSON token. When it encounters a token which isn't a square bracket, a curly bracket, a colon or a comma, it calls Decoder.Decode() to process it. This last function increments the decoder's internal scanner's byte counter on each byte it encounters. However, this increment isn't done if the character appears in the list mentioned previously. This causes the scanner's bytes counter to not correctly reflect the amount of bytes it has processed in the payload, and to show incoherent values when returning errors to the caller, e.g. with SyntaxError instances. Fixes golang#34543
babolivier
added a commit
to babolivier/go
that referenced
this issue
Oct 1, 2019
…nter of its internal scanner Decoder.Token() goes through the bytes of the JSON payload and return the next JSON token. When it encounters a token which isn't a square bracket, a curly bracket, a colon or a comma, it calls Decoder.Decode() to process it. This last function increments the decoder's internal scanner's byte counter on each byte it encounters. However, this increment isn't done if the character appears in the list mentioned previously. This causes the scanner's bytes counter to not correctly reflect the amount of bytes it has processed in the payload, and to show incoherent values when returning errors to the caller, e.g. with SyntaxError instances. Fixes golang#34543
babolivier
added a commit
to babolivier/go
that referenced
this issue
Oct 1, 2019
decoder.Token() goes through the bytes of the JSON payload and return the next JSON token. When it encounters a token which isn't a square bracket, a curly bracket, a colon or a comma, it calls decoder.Decode() to process it. This last function increments the decoder's internal scanner's byte counter on each byte it encounters. However, this increment isn't done if the character appears in the list mentioned previously. This causes the scanner's bytes counter to not correctly reflect the amount of bytes it has processed in the payload, and to show incoherent values when returning errors to the caller, e.g. with SyntaxError instances. Fixes golang#34543
|
Change https://golang.org/cl/198047 mentions this issue: |
|
The CL above could be the right fix, but it has no tests. If someone else wants to send a CL with a test, that would be helpful. |
|
Change https://golang.org/cl/284078 mentions this issue: |
This was referenced Oct 13, 2021
AlexanderYastrebov
added a commit
to AlexanderYastrebov/go
that referenced
this issue
Oct 13, 2021
Stream decoder does not count whitespaces, empty objects and arrays in syntax error offset. This change removes offset tracking from the scanner and relies on the calling code to provide the correct value. Fixes golang#44811, golang#34543
|
Change https://golang.org/cl/355729 mentions this issue: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What version of Go are you using (
go version)?Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (
go env)?go envOutputWhat did you do?
https://play.golang.org/p/zkVEGITpYIo
What did you expect to see?
The same Offset for the two error cases, as it is the same input.
What did you see instead?
Different Offset.
The text was updated successfully, but these errors were encountered: