x/text: panic in language.ParseAcceptLanguage while parsing -u- extension #42535

ph1048 · 2020-11-12T11:07:01Z

What version of Go are you using (`go version`)?

$ go version
go version go1.15.4 linux/amd64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/sasha/.cache/go-build"
GOENV="/home/sasha/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/sasha/goenv/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/sasha/goenv"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build111267796=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version go1.15.4 linux/amd64
GOROOT/bin/go tool compile -V: compile version go1.15.4
uname -sr: Linux 4.19.128-microsoft-standard
Distributor ID:	Kali
Description:	Kali GNU/Linux Rolling
Release:	2020.2
Codename:	kali-rolling
/lib/x86_64-linux-gnu/libc.so.6: GNU C Library (Debian GLIBC 2.31-3) stable release version 2.31.
gdb --version: GNU gdb (Debian 9.2-1) 9.2

What did you do?

https://play.golang.org/p/FCHj_rCBdiH

What did you expect to see?

Error via return value

What did you see instead?

panic: runtime error: index out of range [17] with length 14

goroutine 1 [running]:
golang.org/x/text/internal/language.Tag.findTypeForKey(0x202000000013e, 0xc00002c070, 0xe, 0x4d8c35, 0x2, 0x2d01104014010d0, 0x21127901ec11a0, 0x14e1470015e1424)
	/tmp/gopath829095260/pkg/mod/golang.org/x/text@v0.3.4/internal/language/language.go:456 +0x366
golang.org/x/text/internal/language.Tag.TypeForKey(0x202000000013e, 0xc00002c070, 0xe, 0x4d8c35, 0x2, 0xe, 0x20)
	/tmp/gopath829095260/pkg/mod/golang.org/x/text@v0.3.4/internal/language/language.go:307 +0x4d
golang.org/x/text/internal/language/compact.Make(0x202000000013e, 0xc00002c070, 0xe, 0xc00002c070, 0xe, 0x4fbf00)
	/tmp/gopath829095260/pkg/mod/golang.org/x/text@v0.3.4/internal/language/compact/language.go:38 +0x65
golang.org/x/text/language.makeTag(...)
	/tmp/gopath829095260/pkg/mod/golang.org/x/text@v0.3.4/language/language.go:25
golang.org/x/text/language.CanonType.Parse(0x17, 0x4da9c2, 0x10, 0x4da9c2, 0x10, 0x0, 0x0, 0x3fc0389239a6386c)
	/tmp/gopath829095260/pkg/mod/golang.org/x/text@v0.3.4/language/parse.go:48 +0x145
golang.org/x/text/language.Parse(...)
	/tmp/gopath829095260/pkg/mod/golang.org/x/text@v0.3.4/language/parse.go:34
golang.org/x/text/language.ParseAcceptLanguage(0x4da9c2, 0x10, 0xc000068f48, 0x442bca, 0x56ed40, 0xc000032778, 0xc000068f78, 0x405e25, 0xc00005e058, 0x0)
	/tmp/gopath829095260/pkg/mod/golang.org/x/text@v0.3.4/language/parse.go:154 +0x165
main.main()
	/tmp/sandbox226474929/prog.go:10 +0x3a

The text was updated successfully, but these errors were encountered:

cagedmantis · 2020-11-13T20:48:50Z

/cc @mpvl

carnil · 2021-01-02T09:53:32Z

CVE-2020-28851 appears to have been assigned for this issue.

andyedwardsibm · 2021-01-07T14:24:43Z

Is a fix available for this, or any info on what version of go it will go into? Or is it just x/text that's affected and the fix will be there (may already be there)?

dvasilen · 2021-01-21T13:42:40Z

The latest text v0.3.5 #42536 (comment) still does not have a fix for this issue.

package main

import (
	"fmt"

	"golang.org/x/text/language"
)

func main() {
//	_, _, err := language.ParseAcceptLanguage("00-t-0o") // fixed CVE-2020-28852 
//	fmt.Println("Error:", err)
// Error: language: tag is not well-formed

	
	_, _, err := language.ParseAcceptLanguage("ES-v-00-u-000-00") // not fixed CVE-2020-28851
	fmt.Println("Error:", err)
// panic: runtime error: index out of range [17] with length 14
}

/cc @rsc

dvasilen · 2021-01-30T18:19:36Z

Any update/ETA on this issue is appreciated.

benjsmi · 2021-02-04T14:18:22Z

+1 -- any word on the progress of this?

rsc · 2021-02-10T20:43:21Z

Discussed with @mpvl - this is in a different part of the code and still needs to be fixed. He will work on it.

gopherbot · 2021-02-18T07:33:28Z

Change https://golang.org/cl/293549 mentions this issue: language: allow variable number of types per key in -u- extension

Jethzabell · 2021-02-26T21:13:50Z

"Any update/ETA on this issue is appreciated."

mpvl · 2021-02-27T11:00:02Z

@Jethzabell: submitted a fix.

zhsj · 2021-02-28T15:57:53Z

A new tag on x/text is appreciated.

dvasilen · 2021-02-28T17:51:11Z

+1

dvasilen · 2021-03-01T16:01:36Z

While we are waiting for the tag ... here is the go.mod update to pick up the fix

golang.org/x/text v0.3.6-0.20210227105805-e3aa4adf54f6

benjsmi · 2021-03-18T13:10:15Z

I'm somewhat new to the Go community, but I have observed that with go1.16.2, when you run go get golang.org/x/text/language, it still pulls version v0.3.5 of this module instead of the newly-fixed 0.3.6. Can someone help me track when that would/will change? So far, I've been watching on https://golang.org/doc/devel/release.html#go1.16 and also https://github.com/golang/go/issues?q=milestone%3AGo1.16.2+label%3ACherryPickApproved as examples.

I should note: I'm aware I can change the version that is used in my go.mod, but my team uses countless Go components that would need to be checked for the existence of golang.org/x/text -- it would be much much easier if the default version installed is the one mentioned by @dvasilen above.

RSAlderman · 2021-03-26T11:05:56Z

Any idea when the tagged v0.3.6 version with the fix will be available?

mpvl · 2021-03-26T11:20:52Z

There is another urgent fix pending and will tag afterwards.

On Fri, Mar 26, 2021 at 12:06 RSAlderman ***@***.***> wrote: Any idea when the tagged v0.3.6 version <https://github.com/golang/text/tags> with the fix will be available? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#42535 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABRFSR6BGN6LTZHUFMVJAL3TFRTCPANCNFSM4TTEOOYA> .

-- Marcel van Lohuizen -- Google Switzerland GmbH -- Identifikationsnummer: CH-020.4.028.116-1

update text package to 0.3.6 to fix CVE CVE-2020-28851 more details at golang/go#42535 and https://bugzilla.redhat.com/show_bug.cgi?id=1922730 Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>

This also fixes CVE-2020-28851. This was an off-by one error, but is fixed by handling all cases according to the spec. These valid case seem to be not used in practice much, if at all, but the main benefit is that it makes all valid BCP 47 language tags also valid -u extensions. Fixing the code to handle BCP 47 results in cleaner and seemingly more robust code. The main difference is as follows. The old impementation assumed a -u- extension of the form: <tag> "-u" { "-" <attr> } { "-" <key> "-" <type> } [ <otherExtensions> ] where <attr> and <type> are of length 3-8 and a <key> is of length 2. According to the spec, though, the format is <tag> "-u" { "-" <attr> } { "-" <key> { "-" <type> } } [ <otherExtensions> ] So every key may be associated with zero or more types, instead of exactly one. The new code now handles this. The language.Tag.TypeForKey method is now defined to only return the first entry or nothing at all. This is for backwards compatibilty reasons. Fixes golang/go#42535 Change-Id: I23aec4e1c4d8807fc2ffc0eb3a08de2d8150219f Reviewed-on: https://go-review.googlesource.com/c/text/+/293549 Trust: Marcel van Lohuizen <mpvl@golang.org> Run-TryBot: Marcel van Lohuizen <mpvl@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>

ph1048 changed the title ~~text/language: panic in language.ParseAcceptLanguage while parsing -u- extension~~ x/text: panic in language.ParseAcceptLanguage while parsing -u- extension Nov 12, 2020

gopherbot added this to the Unreleased milestone Nov 12, 2020

cagedmantis added the NeedsInvestigation label Nov 13, 2020

gopherbot closed this as completed in golang/text@e3aa4ad Feb 27, 2021

Madhu-1 mentioned this issue Jun 4, 2021

rebase: update golang.org/x/text to 0.3.6 ceph/ceph-csi#2132

Closed

golang locked and limited conversation to collaborators Mar 26, 2022

gopherbot added the FrozenDueToAge label Mar 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

x/text: panic in language.ParseAcceptLanguage while parsing -u- extension #42535

x/text: panic in language.ParseAcceptLanguage while parsing -u- extension #42535

ph1048 commented Nov 12, 2020 •

edited

Loading

cagedmantis commented Nov 13, 2020

carnil commented Jan 2, 2021

andyedwardsibm commented Jan 7, 2021 •

edited

Loading

dvasilen commented Jan 21, 2021 •

edited

Loading

dvasilen commented Jan 30, 2021

benjsmi commented Feb 4, 2021

rsc commented Feb 10, 2021

gopherbot commented Feb 18, 2021

Jethzabell commented Feb 26, 2021

mpvl commented Feb 27, 2021

zhsj commented Feb 28, 2021

dvasilen commented Feb 28, 2021

dvasilen commented Mar 1, 2021

benjsmi commented Mar 18, 2021 •

edited

Loading

RSAlderman commented Mar 26, 2021

mpvl commented Mar 26, 2021 via email

x/text: panic in language.ParseAcceptLanguage while parsing -u- extension #42535

x/text: panic in language.ParseAcceptLanguage while parsing -u- extension #42535

Comments

ph1048 commented Nov 12, 2020 • edited Loading

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

cagedmantis commented Nov 13, 2020

carnil commented Jan 2, 2021

andyedwardsibm commented Jan 7, 2021 • edited Loading

dvasilen commented Jan 21, 2021 • edited Loading

dvasilen commented Jan 30, 2021

benjsmi commented Feb 4, 2021

rsc commented Feb 10, 2021

gopherbot commented Feb 18, 2021

Jethzabell commented Feb 26, 2021

mpvl commented Feb 27, 2021

zhsj commented Feb 28, 2021

dvasilen commented Feb 28, 2021

dvasilen commented Mar 1, 2021

benjsmi commented Mar 18, 2021 • edited Loading

RSAlderman commented Mar 26, 2021

mpvl commented Mar 26, 2021 via email

ph1048 commented Nov 12, 2020 •

edited

Loading

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

andyedwardsibm commented Jan 7, 2021 •

edited

Loading

dvasilen commented Jan 21, 2021 •

edited

Loading

benjsmi commented Mar 18, 2021 •

edited

Loading