Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: use slicebytetostringtmp in concatenation #31993

Open
dvyukov opened this issue May 13, 2019 · 4 comments
Open

cmd/compile: use slicebytetostringtmp in concatenation #31993

dvyukov opened this issue May 13, 2019 · 4 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done. Performance
Milestone

Comments

@dvyukov
Copy link
Member

dvyukov commented May 13, 2019

$ go version
go version devel +2e4edf4697 Sun May 12 07:14:09 2019 +0000 linux/amd64

Test:

package main

var data = []byte("data")
var str = "str"

func main() {
	if (string(data) + str)[1] == 'a' {
		println('a')
	}
}

This currently uses runtime.slicebytetostring followed by runtime.concatstring2. I think this could use runtime.slicebytetostringtmp for slice->string conversion.
This is extracted from a real program where the slice is large, but we need to append a small string to it. Avoiding the second alloc/copy would be useful.

@randall77 randall77 added this to the Go1.14 milestone May 13, 2019
@andybons andybons added the NeedsFix The path to resolution is known, but the work has not been done. label May 13, 2019
@martisch
Copy link
Contributor

IRC the compiler uses slicebytetostringtmp for concats if one of the non converted string concats is known to be non zero as there is an optimization that does not copy the content of the string if all other arguments of concat have zero length.

We could (and I had thought about that too after optimizing it by hand in real code) optimize string concats which are only in comparison. So generally e.g. string(a) + string(b) ...== string(c) + string(d) ... . Is the real program mentioned also using concat in a comparison?

@dvyukov
Copy link
Member Author

dvyukov commented May 14, 2019

IRC the compiler uses slicebytetostringtmp for concats if one of the non converted string concats is known to be non zero as there is an optimization that does not copy the content of the string if all other arguments of concat have zero length.

Good point.

We could (and I had thought about that too after optimizing it by hand in real code) optimize string concats which are only in comparison. So generally e.g. string(a) + string(b) ...== string(c) + string(d) ... . Is the real program mentioned also using concat in a comparison?

No, it wasn't in comparison. The string is parsed further and memorized in an escaping struct.
In my case the other string is usually non-empty, but compiler won't be able to figure it out statically (email subject).

@martisch
Copy link
Contributor

martisch commented May 14, 2019

You could try optimizing using e.g. (“a”+string(data)+str)[1:] to trigger slicebytetostringtmp until we find a nicer way for the compiler to understand that neither data nor str can be empty by e.g. using information from the prove pass for bounds checks and an additional length check in the code.

Alternatively like in this case we could make the compiler use a special concat call that always copies but also avoids all string allocations for string conversions.

@mdempsky
Copy link
Member

One option here is that if the compiler can't prove that any of the strings are non-empty, it could just pass that information on to the runtime, so the runtime knows it can't do the return-without-copying optimization.

E.g., it could provide a uintptr bitset representing which string operands used slicebytetostringtmp. The runtime then just needs to do a single bit test to figure out if the optimization is safe. (And conservatively, we can't use slicebytetostringtmp for arguments past the first 32 or 64, depending on ptr-size, unless we know there's a non-empty string operand.)

@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done. Performance
Projects
Status: Triage Backlog
Development

No branches or pull requests

7 participants