I can take this on.

See also #24513

Change https://golang.org/cl/107295 mentions this issue: http2: don't sniff Content-type in Server when X-Content-Type-Options:nosniff

Needs to be bundled back into std.

Change https://golang.org/cl/111655 mentions this issue: net/http: update bundled http2

This change is probably incompatible with ever doing #24513 (automatically add X-Content-Type-Options:nosniff), since adding the header automatically is now equivalent to disabling server-side content sniffing for everyone. Do we want to close off that avenue?

I'm reopening this for confirmation that we actually want this change.

To my understanding, it's generally accepted that it's always a good idea to set the X-Content-Type-Options:nosniff header. #24513 proposes automatically adding this header; I believe the only reason we haven't followed through on it is that it causes failures in tests which have unreasonable expectations about the precise set of headers returned. Inside Google, our standard http package wrapper always adds this header.

Changes https://golang.org/cl/89275 and https://golang.org/cl/107295 disable server-side Content-Type sniffing when X-Content-Type-Options:nosniff is set.

If we tie client-side and server-side Content-Type sniffing in this fashion, then #24513 becomes equivalent to disabling the automatic setting of the Content-Type by default. This will break more than just tests; I'm certain plenty of users are depending on this ability.

I think we're better off leaving these decoupled so that we can go ahead with setting X-Content-Type-Options:nosniff by default. That change is, so far as I know, a strict security upgrade for all users and shouldn't break anything other than overly-restrictive tests. In contrast, the change here only applies to users who 1. set X-Content-Type-Options:nosniff, and 2. don't set Content-Type explicitly; even if it's a security benefit in this case, it's one which applies to fewer users than #24513 will.

I'm fine reverting it. Was that decided? I wasn't the one who initiated this change, though. Are we waiting on opinions or do you want me to just revert it?

I think you're the decider on whether to revert or not; I'm just presenting my opinion here.

That change is, so far as I know, a strict security upgrade for all users and shouldn't break anything other than overly-restrictive tests.

This is not true, otherwise modern browsers would always behave as if it were set. It still breaks loading script and style tags with the wrong Content-Type, or HTML without Content-Type. https://fetch.spec.whatwg.org/#x-content-type-options-header

The problem with server-side sniffing is that it's less defined and less context-aware than browser sniffing, will probably be more unexpected, and not as well scrutinized as time passes. Sniffing is dizzily complex: https://mimesniff.spec.whatwg.org

The rationale of this change is that if nosniff is explicitly set we shouldn't defuse it and override it by doing server-side sniffing. It's true that always setting nosniff would be a bigger win, but I don't think it's something we can get away with in the open ecosystem yet. Note how even #24513 was restricted to only a few Content-Types, until it was explained to me that the world is more complex than I thought and it would be useless.

Inside Google I think we can manage to keep both default nosniff and this change, by landing it well, while on the outside I think this is the best we can do. So I'm -1 on reverting.

I think we should revert it for now and figure it out all at once if/when #24513 happens. I agree with @neild that we shouldn't close off any avenues.

Also, I just verified that the net/http server won't mimesniff if the response "Content-Type" header's []string value is set to nil. (that is, in the map, but with no values)

So if Google wants such behavior internally, its middleware that sets nosniff can also set the Content-Type to nil.

Okay, I definitely think this should be reverted now. It turns out that hoisting the automatic setting of Content-Type into user code is quite a bit more complicated than I'd realized.

If the Content-Type isn't set by the user, the http package sets one based on the first 512 bytes of data written to the ResponseWriter. If a user wants to set X-Content-Type-Options:nosniff but preserve the server-side sniffing, they need to manage the buffering of the first 512 bytes of response themselves. This is more complicated than just wrapping a custom ResponseWriter around the usual one, since the ResponseWriter interface doesn't have a Close method. (End-of-response being signaled by the handler function returning rather than an explicit close.)

Change https://golang.org/cl/126895 mentions this issue: http2: revert CL 107295 (don't sniff Content-type in Server when nosniff)

@neild, I sent two CLs. PTAL.

Change https://golang.org/cl/126896 mentions this issue: net/http: revert CL 89275 (don't sniff Content-Type when nosniff set)