Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

text/template: generated slice not initialized to zero #34720

Closed
wasmup opened this issue Oct 6, 2019 · 2 comments
Closed

text/template: generated slice not initialized to zero #34720

wasmup opened this issue Oct 6, 2019 · 2 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@wasmup
Copy link

wasmup commented Oct 6, 2019 •
edited

What version of Go are you using (go version)?

$ go version
go version go1.13.1 windows/amd64
and same on:
go version go1.13.1 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

windows/amd64
and
linux/amd64

GOARCH=amd64

go env Output
$ go env

GO111MODULE=""

GOARCH="amd64"

GOBIN="/home/user/go/bin"

GOCACHE="/home/user/.cache/go-build"

GOENV="/home/user/.config/go/env"

GOEXE=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="linux"

GONOPROXY=""

GONOSUMDB=""

GOOS="linux"

GOPATH="/home/user/go"

GOPRIVATE=""

GOPROXY="https://proxy.golang.org,direct"

GOROOT="/usr/local/go"

GOSUMDB="sum.golang.org"

GOTMPDIR="/dev/shm"

GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"

GCCGO="/usr/bin/gccgo"

CC="gcc"

CXX="g++"

CGO_ENABLED="1"

GOMOD=""

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/dev/shm/go-build949846437=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Try this
Then try this

What did you expect to see?

Initialize the new underlying array of the template generated slice to zero.

What did you see instead?

Not zero.

Code one:

package main

import (
    "bytes"
    "fmt"
    "io"
    "text/template"
)

func main() {
    buf := &bytes.Buffer{}
    my := &myWriter{"You", buf}
    template.Must(template.New("my").Parse("Hi{{.Name}}Bye.")).Execute(my, my)
    fmt.Printf("<<%q>>\n", buf.String())
}
func (m *myWriter) Write(p []byte) (n int, err error) {
    fmt.Printf("len=%v cap=%v\t%v %v\n", len(p), cap(p), string(p), p[:cap(p)])
    no++
    fmt.Println("gen:", no, gen())
    m.Writer.Write(p)
    // m.Writer.Write(p[:8])
    return 8, nil
}

type myWriter struct {
    Name string
    io.Writer
}

const genLen = 8

func gen() string {
    b := [genLen]byte{}
    for i := range b {
        b[i] = no
    }
    return string(b[:])
}

var no = byte(49) //'1'

Output:

len=2 cap=8 Hi [72 105 0 0 0 0 0 0]
gen: 50 22222222
len=3 cap=64    You [89 111 117 58 32 53 48 32 50 50 50 50 50 50 50 50 10 50 32 49 48 53 32 48 32 48 32 48 32 48 32 48 32 48 93 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
gen: 51 33333333
len=4 cap=8 Bye. [66 121 101 46 0 0 0 0]
gen: 52 44444444
<<"HiYouBye.">>

Code two:

package main

import (
	"bytes"
	"fmt"
	"io"
	"text/template"
)

func main() {
	buf := &bytes.Buffer{}
	my := &myWriter{"You", buf}
	template.Must(template.New("my").Parse("Hi{{.Name}}Bye.")).Execute(my, my)
	fmt.Printf("<<%q>>\n", buf.String())
}
func (m *myWriter) Write(p []byte) (n int, err error) {
	fmt.Printf("len=%v cap=%v\t%v %v\n", len(p), cap(p), string(p), p[:cap(p)])
	no++
	fmt.Println("gen:", no, gen())
	m.Writer.Write(p)
	// m.Writer.Write(p[:8])
	return 8, nil
}

type myWriter struct {
	Name string
	io.Writer
}

const genLen = 64

func gen() string {
	b := [genLen]byte{}
	for i := range b {
		b[i] = no
	}
	return string(b[:])
}

var no = byte(49) //'1'

Output:

len=2 cap=8 Hi [72 105 0 0 0 0 0 0]
gen: 50 2222222222222222222222222222222222222222222222222222222222222222
len=3 cap=128   You [89 111 117 58 32 53 48 32 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
gen: 51 3333333333333333333333333333333333333333333333333333333333333333
len=4 cap=8 Bye. [66 121 101 46 0 0 0 0]
gen: 52 4444444444444444444444444444444444444444444444444444444444444444
<<"HiYouBye.">>

This should be all zeros except the first three:

[89 111 117 58 32 53 48 32 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 50 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]

The t.Execute(my, my) calls func (m *myWriter) Write(p []byte) so the p with len=3 and cap=128 generated by the tamplate engine.

After debugging the 2nd code inside /usr/local/go/src/fmt/print.go file at line 230, it seems it is fmt.buffer with length=3, and cap=128, here:

func Fprint(w io.Writer, a ...interface{}) (n int, err error) {
	p := newPrinter()
	p.doPrint(a)
	n, err = w.Write(p.buf)
	p.free()
	return
}

The call to p := newPrinter() initialized here p.fmt.init(&p.buf):

// newPrinter allocates a new pp struct or grabs a cached one.
func newPrinter() *pp {
	p := ppFree.Get().(*pp)
	p.panicking = false
	p.erroring = false
	p.wrapErrs = false
	p.fmt.init(&p.buf)
	return p
}

Gets and returns the free memory without initializing it to zero.

Solution:

In /usr/local/go/src/fmt/format.go file at line 58 set the buffer to all zero:

func (f *fmt) init(buf *buffer) {
	b := (*buf)[:cap(*buf)]
	for i := range b {
		b[i] = 0
	}
	f.buf = buf
	f.clearflags()
}

Then Output of the 2nd code is all zero:

len=2 cap=8     Hi [72 105 0 0 0 0 0 0]
gen: 50 2222222222222222222222222222222222222222222222222222222222222222
len=3 cap=128   You [89 111 117 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
gen: 51 3333333333333333333333333333333333333333333333333333333333333333
len=4 cap=8     Bye. [66 121 101 46 0 0 0 0]
gen: 52 4444444444444444444444444444444444444444444444444444444444444444
<<"HiYouBye.">>
@katiehockman katiehockman changed the title template generated slice not initialized to zero text/template: generated slice not initialized to zero Oct 7, 2019
@katiehockman katiehockman added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 7, 2019
@katiehockman
Copy link
Contributor

/cc @robpike

@katiehockman katiehockman added this to the Go1.14 milestone Oct 7, 2019
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@seankhliao
Copy link
Member

This doesn't look like an actual issue, the slices passed have the correct length, and contents in the extra capacity aren't really relevant.

@golang golang locked and limited conversation to collaborators Jun 7, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
5 participants
@rsc @gopherbot @seankhliao @wasmup @katiehockman