Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/compile: eliminate memory allocation for non-int n in slice extension idiom #29785

Open
valyala opened this issue Jan 17, 2019 · 5 comments
Open

cmd/compile: eliminate memory allocation for non-int n in slice extension idiom #29785

valyala opened this issue Jan 17, 2019 · 5 comments
Assignees
@martisch
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done. Performance
Milestone
Backlog

Comments

@valyala
Copy link
Contributor

valyala commented Jan 17, 2019

The slice extension idiom from #21266 doesn't work in go tip for non-int n in append(a, make([]T, n)...):

func BenchmarkExtendInt(b *testing.B) {
        var buf []byte
        b.ReportAllocs()
        n := int(12345)
        for i := 0; i < b.N; i++ {
                buf = append(buf[:0], make([]byte, n)...)
        }
}

func BenchmarkExtendUint64(b *testing.B) {
        var buf []byte
        b.ReportAllocs()
        n := uint64(12345)
        for i := 0; i < b.N; i++ {
                buf = append(buf[:0], make([]byte, n)...)
        }
}

Benchmark results:

BenchmarkExtendInt-4      	10000000	       145 ns/op	       0 B/op	       0 allocs/op
BenchmarkExtendUint64-4   	 1000000	      1576 ns/op	   13568 B/op	       1 allocs/op

As you can see, if n has a type other than int, go doesn't remove the allocation.
@valyala valyala changed the title Eliminate memory allocation for non-int n in slice extension idiom cmd/compile: eliminate memory allocation for non-int n in slice extension idiom Jan 17, 2019
@mvdan mvdan added Performance NeedsFix The path to resolution is known, but the work has not been done. labels Jan 17, 2019
@mvdan
Copy link
Member

mvdan commented Jan 17, 2019

I'm curious - what happens with other integer types like int16 or int64?

@josharian
Copy link
Contributor

cc @martisch

@martisch
Copy link
Contributor

martisch commented Jan 17, 2019 •
edited

This was a known limitation when writing the compiler optimization and there is a todo in the walk.go code to support more types. Adding all types which values fit in an int shouldnt be hard. For types exceeding the value range of int we need additional runtime checks. E.g. if uint64 seems worth supporting even if only for consistency we can add an extra check to throw the same panic as eg makeslice64 does. I have a look at it once the tree opens for go1.13.

@agnivade agnivade added this to the Go1.13 milestone Jan 17, 2019
@gopherbot
Copy link

Change https://golang.org/cl/182559 mentions this issue: cmd/compile: more optimization for append(x, make([]T, y)...) slice extension

@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
gopherbot pushed a commit that referenced this issue Sep 17, 2019
@cuonglm @randall77
cmd/compile: support more length types for slice extension optimization
ec4e851 
golang.org/cl/109517 optimized the compiler to avoid the allocation for make in
append(x, make([]T, y)...). This was only implemented for the case that y has type int.

This change extends the optimization to trigger for all integer types where the value
is known at compile time to fit into an int.

name             old time/op    new time/op    delta
ExtendInt-12        106ns ± 4%     106ns ± 0%      ~     (p=0.351 n=10+6)
ExtendUint64-12    1.03µs ± 5%    0.10µs ± 4%   -90.01%  (p=0.000 n=9+10)

name             old alloc/op   new alloc/op   delta
ExtendInt-12        0.00B          0.00B           ~     (all equal)
ExtendUint64-12    13.6kB ± 0%     0.0kB       -100.00%  (p=0.000 n=10+10)

name             old allocs/op  new allocs/op  delta
ExtendInt-12         0.00           0.00           ~     (all equal)
ExtendUint64-12      1.00 ± 0%      0.00       -100.00%  (p=0.000 n=10+10)

Updates #29785

Change-Id: Ief7760097c285abd591712da98c5b02bc3961fcd
Reviewed-on: https://go-review.googlesource.com/c/go/+/182559
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@quasilyte
Copy link
Contributor

https://golang.org/cl/182559 fixes the provided benchmarks.
Can this issue be closed or there is something else left to do here?

@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@martisch martisch

Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done. Performance
Projects
Go Compiler / Runtime
Status: Triage Backlog
Milestone
Backlog
Development

No branches or pull requests
9 participants
@josharian @rsc @martisch @valyala @andybons @agnivade @mvdan @quasilyte @gopherbot