Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/compile: unnecessary allocation for []byte->string #14829

Closed
nussjustin opened this issue Mar 15, 2016 · 4 comments
Closed

cmd/compile: unnecessary allocation for []byte->string #14829

nussjustin opened this issue Mar 15, 2016 · 4 comments
Labels
FrozenDueToAge Performance
Milestone
Unplanned

Comments

@nussjustin
Copy link
Contributor

  1. What version of Go are you using (go version)?
    go version go1.6 linux/amd64
    and
    go version devel +ea4b785 Tue Mar 15 08:43:34 2016 +0000 linux/amd64
  2. What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/justinn/Workspace/go"
GORACE=""
GOROOT="/usr/lib/go"
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GO15VENDOREXPERIMENT="1"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"
  1. What did you do?

Benchmarked (a function that calls) strings.Join and looked at the memory allocations with pprof. (With go test -bench Join -benchmem -memprofile mem.pprof -memprofilerate 1 -run ^$)

Benchmarks for strings.Join: http://play.golang.org/p/R7snLsnFmg

Benchmark results:

BenchmarkJoin0-8    500000000            3.21 ns/op        0 B/op          0 allocs/op
BenchmarkJoin1-8    500000000            3.66 ns/op        0 B/op          0 allocs/op
BenchmarkJoin2-8    20000000            80.8 ns/op         6 B/op          2 allocs/op

Memory profile for strings.Join (go tool pprof --alloc_objects <binary> mem.pprof):

   1204055    1204055 (flat, cum)   100% of Total
         .          .    358:   n := len(sep) * (len(a) - 1)
         .          .    359:   for i := 0; i < len(a); i++ {
         .          .    360:       n += len(a[i])
         .          .    361:   }
         .          .    362:
    602028     602028    363:   b := make([]byte, n)
         .          .    364:   bp := copy(b, a[0])
         .          .    365:   for _, s := range a[1:] {
         .          .    366:       bp += copy(b[bp:], sep)
         .          .    367:       bp += copy(b[bp:], s)
         .          .    368:   }
    602027     602027    369:   return string(b)
         .          .    370:}
  1. What did you expect to see?

The compiler reusing the memory of b for the return value.

  1. What did you see instead?

The []byte->string conversion in strings.Join (on line 369) allocates a new string, although the []byte could be reused as there can never be a reference to the []byte after the function returns and the []byte is already on the heap¹.

¹ Calling 'go build -gcflags="-m" in the strings package confirms this:

./strings.go:363: make([]byte, n) escapes to heap
./strings.go:369: string(b) escapes to heap

(Line numbers are from tip)
@bradfitz bradfitz added this to the Unplanned milestone Mar 15, 2016
@bradfitz
Copy link
Contributor

I thought we had a dup of this bug already opened, but I can't find it.

/cc @randall77

@ALTree
Copy link
Member

ALTree commented Mar 15, 2016

There was #11777 (now closed). Similar but not really related.

@alexcesaro
Copy link
Contributor

This is a duplicate of #6714.

@bradfitz
Copy link
Contributor

Thanks, @alexcesaro. I knew I'd seen that before. :)

@golang golang locked and limited conversation to collaborators Mar 19, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge Performance
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
5 participants
@bradfitz @alexcesaro @ALTree @nussjustin @gopherbot