Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

encoding/json: SyntaxError's Offset is wrong after decoding some valid json #32399

Closed
itchyny opened this issue Jun 3, 2019 · 4 comments
Closed

encoding/json: SyntaxError's Offset is wrong after decoding some valid json #32399

itchyny opened this issue Jun 3, 2019 · 4 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.14

Comments

@itchyny
Copy link
Contributor

itchyny commented Jun 3, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.5 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/itchyny/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/itchyny/.go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12.5/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12.5/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/mg/nqgjy_l53n15b3_dfrjw9_600000gn/T/go-build713456458=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Run the following code (https://play.golang.org/p/gBI1qRBJvPa).

package main

import (
	"encoding/json"
	"fmt"
	"io"
	"os"
	"strings"
)

func main() {
	in := strings.NewReader(`false true [ "foo" : "bar" ]`)
	dec := json.NewDecoder(in)
	var vs []interface{}
	for {
		var v interface{}
		if err := dec.Decode(&v); err != nil {
			if err != io.EOF {
				fmt.Printf("%#v\n", err)
				os.Exit(1)
			}
			break
		}
		vs = append(vs, v)
	}
	fmt.Println(vs)
}

What did you expect to see?

Outputs &json.SyntaxError{msg:"invalid character ':' after array element", Offset:20}.

What did you see instead?

Outputs &json.SyntaxError{msg:"invalid character ':' after array element", Offset:22}
@agnivade
Copy link
Contributor

agnivade commented Jun 4, 2019

@mvdan @dsnet

@mvdan
Copy link
Member

mvdan commented Jun 4, 2019

Could you please clarify why you expected the offset to be 20?

@itchyny
Copy link
Contributor Author

itchyny commented Jun 4, 2019

I expect because the 20th character of the source string false true [ "foo" : "bar" ] is :. In addition, I expect the Offset does not exceed the source length (see https://play.golang.org/p/W73hGXHZIxq).

@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 10, 2019
@dmitshur dmitshur added this to the Go1.14 milestone Jun 10, 2019
@itchyny itchyny mentioned this issue Jun 13, 2019
Closed
@gopherbot
Copy link

Change https://golang.org/cl/182117 mentions this issue: encoding/json: fix scanner byte offset on scanEnd

t4n6a1ka pushed a commit to t4n6a1ka/go that referenced this issue Sep 5, 2019
@itchyny @t4n6a1ka
encoding/json: fix scanner byte offset on scanEnd
9f501f1 
scanEnd is delayed one byte so we decrement
the scanner bytes count by 1 to ensure that
this value is correct in the next call of Decode.

Fixes golang#32399

Change-Id: I8c8698e7f95bbcf0373aceaa05319819eae9d86f
GitHub-Last-Rev: 0ac25d8
GitHub-Pull-Request: golang#32598
Reviewed-on: https://go-review.googlesource.com/c/go/+/182117
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@itchyny itchyny mentioned this issue Mar 28, 2020
Closed
@golang golang locked and limited conversation to collaborators Sep 1, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.14
Development

Successfully merging a pull request may close this issue.

encoding/json: fix scanner byte offset on scanEnd itchyny/go
5 participants
@itchyny @agnivade @dmitshur @mvdan @gopherbot