I'm not sure I understand the concern about a fault from a vectorized read past the end of the string. Seems like that could happen in Go just as easily as in C.

@ianlancetaylor, because bytes.IndexByte starts with a slice (which contains a length), the assembly for that knows it's safe to do wide loads of many bytes up to that length only (or capacity, at least). C.GoString only takes a *byte but no length, so what if that byte pointer were 2-3 bytes before the end of mapped virtual memory space? If we invented some bogus slice header to give to bytes.IndesByte for GoString, we could be inviting IndexByte to fault on a bogus address.

That's how I read it, at least.

Oh, yeah, OK. For C strings we need to do it slightly differently. Since vector instructions are always aligned, we are fine using vectors as long as we haven't found a zero yet, but we can't read ahead. We could write special purpose code for this, but I doubt it's worth it. We could call a function in runtime/cgo that calls strlen, but we would need a custom asm wrapper to put the arguments in the right place. Is this common enough to make this worth it?

A quick GitHub search says yes, it is used a lot. Whether it is in performance-critical paths, or whether the cost is worth it, I don't know. I'll move to Unplanned, unless/until someone comes forward and says they need it or someone is inspired to dig deeper and implement themselves.

I think using indexbyte with fake slice is safe, if all access is limited to 1 page, so doing something like:

p =  (*slice)(unsafe.Pointer(s))
p.len = int(uintptr(unsafe.Pointer(s)) + uintptr(4093)))/4096*4096 // round up to next pagesize
indexbyte(p,0)
...
for { // page at a time loop
p += 4096*i
p.len = 4096
indexbyte(p,0)
}

@TocarIP That requires fairly strong assumptions about the optimizations present in the rest of the Go compiler and runtime. In general we have no guarantee there isn't some other thread writing to the address just after the string, and the Go memory model does not define behavior in the presence of data races, let alone data races that cross language boundaries.

We could, of course, check the rest of the runtime code to ensure that such races are benign, but it seems very difficult to write a test that would reliably detect violations due to future changes.

I'm not sure I understand what extra assumptions this introduces. IndexByte (at least asm version on amd64) already does out-of-bounds read, as long as they don't cross page for normal go strings.

I think Ilya's strategy should be fine. IndexByte does not depend on data values past the first \0.
Reading data you don't end up looking at doesn't cause a problem on any architecture I know of (caveat: I/O mapped memory, but I'm pretty sure that's not at issue here). The only problem is making sure we don't accidentally read from unmapped memory, which Ilya's code solves.

Change https://golang.org/cl/97523 mentions this issue: runtime: use bytes.IndexByte in findnull

Reverted in https://go-review.googlesource.com/c/go/+/97995

/cc @TocarIP

Change https://golang.org/cl/98015 mentions this issue: runtime: use bytes.IndexByte in findnull