Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

encoding/json: panic when encoding map[string]interface{} #50970

Closed
chriscasola opened this issue Feb 2, 2022 · 4 comments
Closed

encoding/json: panic when encoding map[string]interface{} #50970

chriscasola opened this issue Feb 2, 2022 · 4 comments
Labels
FrozenDueToAge

Comments

@chriscasola
Copy link

What version of Go are you using (go version)?

$ go version
$ go version go1.18beta2 linux/amd64

Does this issue reproduce with the latest release?

Reproduces with latest beta.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/opt/go/pkg/mod"
GONOPROXY=""
GOOS="linux"
GOPATH="/opt/go/"
GOPRIVATE=""
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18beta2"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/workspaces/proj/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build4164521550=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I am trying to convert an apache arrow record to JSON. This same code works correctly in 1.17.3 but not in 1.18beta1 or 1.18beta2.

I have tried removing arrow and doing something similar to what array.RecordToJSON does, but was not able to reproduce that way. I wanted to report this anyway because it seems like a regression. Please let me know if I can provide more information.

This is similar code not using arrow, in an attempt to reproduce, but it does not panic:

package main

import (
	"bytes"
	"encoding/json"
	"strconv"
)

func main() {
	buf := new(bytes.Buffer)
	enc := json.NewEncoder(buf)

	arrs := make([]lenArray, 1)
	arrs[0] = &stringArray{values: "abcd"}

	m := make(map[string]interface{})
	for i, a := range arrs {
		m[strconv.Itoa(i)] = a.(marshalArray).GetForMarshal(i)
	}

	enc.Encode(m)
}

type lenArray interface {
	Len() int
}

type marshalArray interface {
	lenArray
	GetForMarshal(int) interface{}
}

type stringArray struct {
	values string
}

func (s *stringArray) Value(i int) string {
	return s.values[i : i+1]
}

func (s *stringArray) GetForMarshal(i int) interface{} {
	return s.Value(i)
}

func (s *stringArray) Len() int {
	return len(s.values)
}

This is the code using arrow that panics:

package main

import (
	"bytes"

	"github.com/apache/arrow/go/v7/arrow"
	"github.com/apache/arrow/go/v7/arrow/array"
	"github.com/apache/arrow/go/v7/arrow/memory"
)

func main() {
	fld := arrow.Field{Name: "test", Type: arrow.BinaryTypes.String}
	schema := arrow.NewSchema([]arrow.Field{fld}, nil)
	builder := array.NewRecordBuilder(memory.DefaultAllocator, schema)
	builder.Field(0).(*array.StringBuilder).AppendValues([]string{"a", "b", "c", "d"}, nil)
	rec := builder.NewRecord()
	buf := new(bytes.Buffer)
	array.RecordToJSON(rec, buf)
}

What did you expect to see?

Exit 0

What did you see instead?

unexpected fault address 0x8c2248
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0x8c2248 pc=0x411463]

goroutine 6 [running]:
runtime.throw({0x8a394d?, 0x0?})
	/usr/local/go/src/runtime/panic.go:992 +0x71 fp=0xc0001405f0 sp=0xc0001405c0 pc=0x437491
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:809 +0x305 fp=0xc000140640 sp=0xc0001405f0 pc=0x44d925
runtime.mapiterinit(0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/map.go:821 +0x23 fp=0xc000140660 sp=0xc000140640 pc=0x411463
reflect.mapiterinit(0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/map.go:1373 +0x19 fp=0xc000140688 sp=0xc000140660 pc=0x463db9
github.com/goccy/go-json/internal/encoder/vm.Run(0xc000109860, {0xc000130400?, 0x0?, 0x400?}, 0xc000038f40?)
	/opt/go/pkg/mod/github.com/goccy/go-json@v0.7.10/internal/encoder/vm/vm.go:425 +0x1f97 fp=0xc000143b38 sp=0xc000140688 pc=0x66c6b7
github.com/goccy/go-json.encodeRunCode(0x83fe20?, {0xc000130400?, 0xc00007a400?, 0x0?}, 0xc0001465a0?)
	/opt/go/pkg/mod/github.com/goccy/go-json@v0.7.10/encode.go:307 +0x68 fp=0xc000143b70 sp=0xc000143b38 pc=0x69c288
github.com/goccy/go-json.encode(0xc000109860, {0x83fe20, 0xc00007f9e0})
	/opt/go/pkg/mod/github.com/goccy/go-json@v0.7.10/encode.go:232 +0x21c fp=0xc000143bf0 sp=0xc000143b70 pc=0x69bd9c
github.com/goccy/go-json.(*Encoder).encodeWithOption(0xc000143d38, 0xc000109860, {0x83fe20, 0xc00007f9e0}, {0x0, 0x0, 0xc00006dc01?})
	/opt/go/pkg/mod/github.com/goccy/go-json@v0.7.10/encode.go:74 +0xeb fp=0xc000143c50 sp=0xc000143bf0 pc=0x69b86b
github.com/goccy/go-json.(*Encoder).EncodeWithOption(0x83fe20?, {0x83fe20, 0xc00007f9e0}, {0x0, 0x0, 0x0})
	/opt/go/pkg/mod/github.com/goccy/go-json@v0.7.10/encode.go:41 +0x92 fp=0xc000143cb8 sp=0xc000143c50 pc=0x69b6f2
github.com/goccy/go-json.(*Encoder).Encode(...)
	/opt/go/pkg/mod/github.com/goccy/go-json@v0.7.10/encode.go:33
github.com/apache/arrow/go/v7/arrow/array.RecordToJSON({0x949c88, 0xc00007f980}, {0x9444a0?, 0xc00007f9b0?})
	/opt/go/pkg/mod/github.com/apache/arrow/go/v7@v7.0.0-20220114131701-093fdad19dc2/arrow/array/util.go:215 +0x125 fp=0xc000143d80 sp=0xc000143cb8 pc=0x6d6ec5
github.factset.com/FactSet/go-stach/internal/compute.TestJsonCrash(0x0?)
@dsnet
Copy link
Member

dsnet commented Feb 2, 2022

Thank you for the bug report. However, this appears to be a panic stemming from github.com/goccy/go-json and not the standard library implementation. Please a file bug there.

@dsnet dsnet closed this as completed Feb 2, 2022
@chriscasola
Copy link
Author

@dsnet I can file a bug with them, but this is not reproducible with go 1.17, only with the 1.18 betas.

@dsnet
Copy link
Member

dsnet commented Feb 2, 2022

goccy makes extensive use of unsafe where the package makes assumptions about how the Go runtime works. I would say the responsibility is on goccy to investigate and demonstrate that this is a bug in the Go runtime, rather than what is probably a bug in some unspecified assumption the package makes about the Go runtime that no longer holds.

@ianlancetaylor
Copy link
Contributor

The internal details of reflection map iteration changed from 1.17 to 1.18. This code is reaching into unstable runtime internals. That is always going to require careful attention when updating to a new version of Go. It would, in my opinion, be better to not do that.

@johnwhitton johnwhitton mentioned this issue Jun 13, 2022
Open
@golang golang locked and limited conversation to collaborators Feb 2, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@chriscasola @ianlancetaylor @dsnet @gopherbot