It may not be clear and it may not be right, but this behavior is how strings work in Go. Invalid UTF-8 gets turned, one byte at a time, into U+FFFD. Here is text from the spec about ranges over strings, which is as good a description as any of how Go handles invalid UTF-8. It's part of the language itself to do it this way:

If the iteration encounters an invalid UTF-8 sequence, the second value will be 0xFFFD, the Unicode replacement character, and the next iteration will advance a single byte in the string.

From the point of view of the matching algorithm, the regexp compiler has already overwritten all the invalid UTF-8 when it built the engine using Go's rules to interpret the string.

There is no way to fix this compatibly other than to provide a flag or other mechanism to avoid this interpretation. Given that the code is all runes inside, though, even that may be infeasible.

Working as intended, and unfortunate.

You are right that your best bet is likely to validate the string ahead of time, or else elide the invalid UTF-8 altogether.

Invalid UTF-8 gets turned, one byte at a time, into U+FFFD

This does not explain why a simple regex explicitly looking for U+FFFD does NOT match on invalid utf8. Maybe there's some optimization going on for regexes such as \ufffd or \x{fffd} that turns them into simple string.Contains or similar?

Anyway, thanks for the quick response. I just wanted to save someone some time hairpulling like I did today :)

Based on @ComaVN and @robpike's conversation, I'm going to close this issue.

I think there is a real bug here for some cases. Reopening.

The literals (lines 13-18) are buggy in https://play.golang.org/p/j-jsteknY0M and should be fixed.
The literal search needs to not kick in for U+FFFD.

This is a duplicate of #38006, which I've merged into this issue because this issue had more commentary. That issue was marked as just needing a documentation update.

I started to look into fixing this, but it's fairly complex to get all the cases in all the matching engines.

For the record, the coherent behavior options are:

1. Invalid UTF-8 does not match any character classes, nor a U+FFFD literal (nor \x{fffd}).
2. Each byte of invalid UTF-8 is treated identically to a U+FFFD in the input, as a utf8.DecodeRune loop might.

RE2 uses Rule 1. Because it works byte at a time it can also provide \C to match any single byte of input, which matches invalid UTF-8 as well. This provides the nice property that a match for a regexp without \C is guaranteed to be valid UTF-8.

Unfortunately, today Go has an incoherent mix of these two, although mostly Rule 2. This is a deviation from RE2, and it gives up the nice property, but we probably can't correct that at this point. In particular .* already matches entire inputs today, valid UTF-8 or not, and I doubt we can break that. The right solution for Go is probably to adopt Rule 2 officially, fixing the few places that deviate from Rule 2.

Change https://golang.org/cl/354569 mentions this issue: regexp: document and implement that invalid UTF-8 bytes are the same as U+FFFD