Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

encoding/json: tags can't have Unicode punctuation #53267

Open
chipaca opened this issue Jun 7, 2022 · 1 comment
Open

encoding/json: tags can't have Unicode punctuation #53267

chipaca opened this issue Jun 7, 2022 · 1 comment
Labels
Documentation NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Unplanned

Comments

@chipaca
Copy link
Contributor

chipaca commented Jun 7, 2022

What version of Go are you using (go version)?

1.18.3

Does this issue reproduce with the latest release?

1.18.3 is the latest release at the time of writing

What operating system and processor architecture are you using (go env)?

go env Output
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/john/.cache/go-build"
GOENV="/home/john/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/john/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/john/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/john/sdk/go1.18.3"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/john/sdk/go1.18.3/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.3"
GCCGO="/usr/bin/gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1281614776=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Try to use a struct with a tag that has non-ASCII symbols in it, e.g.

type T struct {
	Foo31 string `json:"foo-3·1"`
}

https://go.dev/play/p/IwgJHZCedCE

What did you expect to see?

Tags can be anything, and JSON places no limits on this either (the name of a name/value pair is “a string”), and the encoding/json package mentions no limits, so I'd expect it to just work: the above playground link should produce

{"foo-3·1":"hi"}
<nil>

What did you see instead?

the tag is silently ignored and you get

{"Foo31":"hi"}
<nil>

this is because encoding/json's isValidTag only allows (some) ASCII symbols:

go/src/encoding/json/encode.go

Lines 972 to 987 in 95b68e1

func isValidTag(s string) bool {
if s == "" {
return false
}
for _, c := range s {
switch {
case strings.ContainsRune("!#$%&()*+-./:;<=>?@[]^_{|}~ ", c):
// Backslash and quote chars are reserved, but
// otherwise any punctuation chars are allowed
// in a tag name.
case !unicode.IsLetter(c) && !unicode.IsDigit(c):
return false
}
}
return true
}

@seankhliao seankhliao changed the title encoding/json: tags can only have ASCII symbols encoding/json: tags can't have Unicode punctuation Jun 7, 2022
@seankhliao
Copy link
Member

related #39189

cc @mvdan @dsnet

@seankhliao seankhliao added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 7, 2022
@seankhliao seankhliao added this to the Unplanned milestone Aug 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Documentation NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
2 participants
@chipaca @seankhliao