Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: efficient way to compare a byte slice and a string #11777

Closed
ibukanov opened this issue Jul 18, 2015 · 3 comments
Closed

cmd/compile: efficient way to compare a byte slice and a string #11777

ibukanov opened this issue Jul 18, 2015 · 3 comments

Comments

@ibukanov
Copy link

Currently Go does not provide a utility to compare efficiently a byte slice against a string. Doing string(byteSlice) == str is inefficient as it copies the slice first into a new string. It would be nice to have a utility like bytes.EqualString(b []byte, s string) that is just as efficient as bytes.Equal(a, b []byte). Another alternative is to optimize string(byteSlice) compareOp str so compareOp accesses the slice directly. The latter would benefit the existing code even if it is harder to implement.

@nussjustin
Copy link
Contributor

Go 1.5 has what you want. It was implemented in CL #3120.

@ibukanov
Copy link
Author

@nuss-justin Now I see why string(byteSlice) == str became 3 times faster in Go 1.5 according to http://stackoverflow.com/a/31484416/2898400 . But it still 8 times slower than calling bytes.Equal via an unsafe cast. And this is in an ideal case when whole 100 byte long slice has to be read for the answer. So escape analyzes helps, but not that match as bytes still has to be copied.

@bradfitz bradfitz changed the title efficient way to compare a byte slice and a string cmd/compile: efficient way to compare a byte slice and a string Jul 18, 2015
@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Jul 18, 2015
@ALTree
Copy link
Member

ALTree commented Jan 23, 2016

Doing string(byteSlice) == str is inefficient as it copies the slice first into a new string

I don't believe this is true. If you disassemble

var bs []byte = []byte{104, 97, 108, 108, 111}

func main() {
    x := string(bs) == "hello"
    println(x)
}

you'll see that string(bs) is compiled to a call to runtime.slicebytetostringtmp, which, as you can see, does not copy the data:

func slicebytetostringtmp(b []byte) string {
    // Return a "string" referring to the actual []byte bytes.
    // This is only for use by internal compiler optimizations
    // that know that the string form will be discarded before
    // the calling goroutine could possibly modify the original
    // slice or synchronize with another goroutine.
    // First such case is a m[string(k)] lookup where
    // m is a string-keyed map and k is a []byte.
    // Second such case is "<"+string(b)+">" concatenation where b is []byte.
    // Third such case is string(b)=="foo" comparison where b is []byte.

    if raceenabled && len(b) > 0 {
        racereadrangepc(unsafe.Pointer(&b[0]),
            uintptr(len(b)),
            getcallerpc(unsafe.Pointer(&b)),
            funcPC(slicebytetostringtmp))
    }
    if msanenabled && len(b) > 0 {
        msanread(unsafe.Pointer(&b[0]), uintptr(len(b)))
    }
    return *(*string)(unsafe.Pointer(&b))
}

Benchmarking also shows that there's almost no difference between the cast and a call to bytes.Equal. On tip:

var res bool

func BenchmarkStrCast(b *testing.B) {
    s1 := strings.Repeat("a", 1000)
    b2 := []byte(strings.Repeat("a", 999) + "b")
    for n := 0; n < b.N; n++ {
        res = string(b2) == s1
    }
}

func BenchmarkBytesCmp(b *testing.B) {
    b1 := []byte(strings.Repeat("a", 1000))
    b2 := []byte(strings.Repeat("a", 999) + "b")
    for n := 0; n < b.N; n++ {
        res = bytes.Equal(b1, b2)
    }
}

gives

BenchmarkStrCast-4  50000000            28.5 ns/op         0 B/op          0 allocs/op
BenchmarkBytesCmp-4 50000000            27.1 ns/op         0 B/op          0 allocs/op

@golang golang locked and limited conversation to collaborators Jan 23, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

6 participants