Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: encode panic #55829

Closed
xtxy opened this issue Sep 23, 2022 · 11 comments
Closed

encoding/json: encode panic #55829

xtxy opened this issue Sep 23, 2022 · 11 comments
Labels
WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.

Comments

@xtxy
Copy link

xtxy commented Sep 23, 2022

What version of Go are you using (go version)?

$ go version
go version go1.19 linux/amd64

Does this issue reproduce with the latest release?

I think so

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/clark/.cache/go-build"
GOENV="/home/clark/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/clark/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/clark/go"
GOPRIVATE=""
GOPROXY="https://goproxy.io,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build858534376=/tmp/go-build -gno-record-gcc-switches"

What did you do?

json encode map[string]interface{}

data, err = json.Marshal(dataMap)
if err != nil {
logger.Error("msg.encode.error:", err)
return nil
}

What did you expect to see?

return data or error

What did you see instead?

occasionally, it panic:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x50f704]

goroutine 1 [running]:
encoding/json.(*encodeState).marshal.func1()
/usr/local/go/src/encoding/json/encode.go:327 +0x6e
panic({0x81e580, 0xba1d20})
/usr/local/go/src/runtime/panic.go:884 +0x212
encoding/json.(*encodeState).string(0xc07876e080, {0x0, 0x20f}, 0x1)
/usr/local/go/src/encoding/json/encode.go:1034 +0x64
encoding/json.stringEncoder(0xc07876e080, {0x8008c0?, 0xc07510d2b0?, 0x1?}, {0x0?, 0x0?})
/usr/local/go/src/encoding/json/encode.go:646 +0x20a
encoding/json.mapEncoder.encode({0xc071f41680?}, 0xc07876e080, {0x81c260?, 0xc084d3b650?, 0x81c260?}, {0x60?, 0x65?})
/usr/local/go/src/encoding/json/encode.go:813 +0x5f5
encoding/json.(*encodeState).reflectValue(0x812080?, {0x81c260?, 0xc084d3b650?, 0xc084d3b6b0?}, {0xc0?, 0x55?})
/usr/local/go/src/encoding/json/encode.go:359 +0x78
encoding/json.interfaceEncoder(0xc07876e080, {0x812080?, 0xc074c4f2d0?, 0x1?}, {0x0?, 0x4?})
/usr/local/go/src/encoding/json/encode.go:715 +0xc8
encoding/json.mapEncoder.encode({0x20301d?}, 0xc07876e080, {0x81bf60?, 0xc084d3b680?, 0x81bf60?}, {0x21?, 0x30?})
/usr/local/go/src/encoding/json/encode.go:813 +0x5f5
encoding/json.(*encodeState).reflectValue(0xc0731f7958?, {0x81bf60?, 0xc084d3b680?, 0xc0731f79a8?}, {0xa0?, 0x2e?})
/usr/local/go/src/encoding/json/encode.go:359 +0x78
encoding/json.(*encodeState).marshal(0x8006c0?, {0x81bf60?, 0xc084d3b680?}, {0x7?, 0xd8?})
/usr/local/go/src/encoding/json/encode.go:331 +0xfa
encoding/json.Marshal({0x81bf60, 0xc084d3b680})
/usr/local/go/src/encoding/json/encode.go:160 +0x45

@xtxy xtxy changed the title affected/package: encoding/json encoding/json encode panic Sep 23, 2022
@xtxy xtxy changed the title encoding/json encode panic encoding/json: encode panic Sep 23, 2022
@mvdan
Copy link
Member

mvdan commented Sep 23, 2022

Please provide a way to reproduce this issue. Ideally, a small standalone program with the command you used to run it, or a working example on https://go.dev/play/.

@mvdan mvdan added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Sep 23, 2022
@xtxy
Copy link
Author

xtxy commented Sep 23, 2022

Please provide a way to reproduce this issue. Ideally, a small standalone program with the command you used to run it, or a working example on https://go.dev/play/.

I can't. The code is a segment in a complex server, and it doesn't happen every time.
If you can not find the reason by above call stack info, I can only add more log and wait for next panic

@mvdan
Copy link
Member

mvdan commented Sep 23, 2022

The stack trace makes little sense because it points to https://cs.opensource.google/go/go/+/refs/tags/go1.19:src/encoding/json/encode.go;l=1034, which does not use any pointers at all, so it shouldn't end up in a nil pointer dereference.

The stack does show {0x0, 0x20f} as the string argument, which perhaps means that you somehow ended up with a string value that has a nil pointer in it. Are you using unsafe or cgo to make strings by any chance?

Another option is a data race causing memory corruption. Please try running your program with -race: https://go.dev/doc/articles/race_detector

@mvdan
Copy link
Member

mvdan commented Sep 23, 2022

Whatever the case, unless I'm missing something obvious here, this doesn't look like a bug in the json package. I think you just have a corrupted string value.

@xtxy
Copy link
Author

xtxy commented Sep 23, 2022

Whatever the case, unless I'm missing something obvious here, this doesn't look like a bug in the json package. I think you just have a corrupted string value.

no race, dataMap is a local variable in function.

you mean wrong string format will cause json marshal panic?

more relative codes:

`
infos := make(map[string]string)
for k, v := range client.games {
if v.accountInfo == "" {
continue
}

infos[k] = v.accountInfo
v.accountInfo = ""

}
if len(infos) == 0 {
logger.Warning("dyn accounts info is empty")
return
}

logger.Debug("send dyn accounts info to center")

cmd := common.Cmd{
Op: common.OP_DYN_ACCOUNT, Data: infos,
}

pkg := common.CreatePkg(cmd, common.PKG_TARGET_CENTER)
...

// function common.CreatePkg:
func CreatePkg(pkgData interface{}, target uint8) []byte {
if nil == pkgData {
return nil
}

if slice, ok := pkgData.([]byte); ok {
    return CreatePkgBySlice(slice, target)
}

dataMap, ok := pkgData.(map[string]interface{})
if !ok {
    cmd, ok := pkgData.(cmdMapI)
    if !ok {
        return nil
    }
    dataMap = cmd.ToMap()
}
if len(dataMap) == 0 {
    return nil
}

format, ok := parser.ToInt(dataMap, "Fmt")
if !ok {
    format = PKG_FMT_JSON
}

data := EncodePkgData(dataMap, uint8(format))
...

}

func EncodePkgData(dataMap map[string]interface{}, format uint8) []byte {
if nil == dataMap {
return nil
}

var data []byte
var err error

switch format {
case PKG_FMT_JSON:
    data, err = json.Marshal(dataMap) // panic here
    if err != nil {
        logger.Error("msg.encode.error:", err)
        return nil
    }
    ...

}

// Cmd structure:
type Cmd struct {
Op string
Data interface{}
GameId string
Ip string
From string
Fmt int
}

func (cmd Cmd) ToMap() map[string]interface{} {
return map[string]interface{}{
"Op": cmd.Op, "Data": cmd.Data,
"GameId": cmd.GameId, "Ip": cmd.Ip,
"From": cmd.From, "Fmt": cmd.Fmt,
}
}

type cmdMapI interface {
ToMap() map[string]interface{}
}
`

@seankhliao seankhliao added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Sep 23, 2022
@bcmills
Copy link
Contributor

bcmills commented Sep 23, 2022

@xtxy

no race, dataMap is a local variable in function.

The dataMap variable itself isn't the concern. The question is, where do the strings for its keys come from? (How does the caller of CreatePkg construct the map, and/or how does cmd.ToMap() construct it?)

@bcmills bcmills added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Sep 23, 2022
@xtxy
Copy link
Author

xtxy commented Sep 26, 2022

@xtxy

no race, dataMap is a local variable in function.

The dataMap variable itself isn't the concern. The question is, where do the strings for its keys come from? (How does the caller of CreatePkg construct the map, and/or how does cmd.ToMap() construct it?)

I don't know why code tag (<Ctrl+e>) didn't work, so, I use picture instead, hope it be clear:

p1
p2
p3

@bcmills
Copy link
Contributor

bcmills commented Sep 26, 2022

@bcmills bcmills added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Sep 26, 2022
@mvdan
Copy link
Member

mvdan commented Sep 26, 2022

I think it's time to move this out of the issue tracker; so far there is little evidence that there is a bug in encoding/json. See https://golang.org/wiki/Questions.

@mvdan mvdan closed this as not planned Won't fix, can't repro, duplicate, stale Sep 26, 2022
@xtxy
Copy link
Author

xtxy commented Sep 27, 2022

@xtxy See https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks for instructions on how to format code blocks in GitHub issues.

thanks a lot

@caiweiwenjs
Copy link

caiweiwenjs commented Jun 1, 2023

@xtxy same issue for me. did you find the root cause? if so, could you share with me, thank a lot.
here is call stack in my case.

net/http.(*conn).serve.func1()
.../go-env/go1-19-linux-amd64/src/net/http/server.go:1850 +0xbf
panic({0x1292c60, 0x204bba0})
.../go-env/go1-19-linux-amd64/src/runtime/panic.go:890 +0x262
encoding/json.(*encodeState).marshal.func1()
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:327 +0x6e
panic({0x1292c60, 0x204bba0})
.../go-env/go1-19-linux-amd64/src/runtime/panic.go:884 +0x212
encoding/json.(*encodeState).string(0xc004305e00, {0x0, 0x20}, 0x1)
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:1034 +0x64
encoding/json.stringEncoder(0xc004305e00, {0x122c400?, 0xc0013934a0?, 0x20?}, {0x10?, 0x0?})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:646 +0x20a
encoding/json.(*encodeState).reflectValue(0x1264bc0?, {0x122c400?, 0xc0013934a0?, 0xc000f965d0?}, {0x48?, 0x15?})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:359 +0x78
encoding/json.interfaceEncoder(0xc004305e00, {0x1264bc0?, 0xc001393c80?, 0x1?}, {0x0?, 0x0?})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:715 +0xc8
encoding/json.mapEncoder.encode({0x13f?}, 0xc004305e00, {0x1285200?, 0xc000f963f0?, 0x1285200?}, {0xc0?, 0xc1?})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:813 +0x5f5
encoding/json.(*encodeState).reflectValue(0xc00176d0a8?, {0x1285200?, 0xc000f963f0?, 0xdd9320955363c06c?}, {0x60?, 0xc6?})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:359 +0x78
encoding/json.(*encodeState).marshal(0xf?, {0x1285200?, 0xc000f963f0?}, {0x9d?, 0x56?})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:331 +0xfa
encoding/json.Marshal({0x1285200, 0xc000f963f0})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:160 +0x45

@golang golang locked as resolved and limited conversation to collaborators Jun 1, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

5 participants