CC @tombergan

Thanks for the report. I will have to do a bit more code archaeology to see if this was intentional. Per this comment, whitespace at the end of a key is intentionally stripped despite being forbidden by the RFC. It's unclear so far if whitespace at the beginning of the first key is intentionally ignored, or if that was an oversight.

The trailing whitespace was intentional, as seen from this test from 31e9429 from a security camera I used to have:

// Test that we read slightly-bogus MIME headers seen in the wild,                                                                                 
// with spaces before colons, and spaces in keys.                                                                                                  
func TestReadMIMEHeaderNonCompliant(t *testing.T) {
        // Invalid HTTP response header as sent by an Axis security                                                                                
        // camera: (this is handled by IE, Firefox, Chrome, curl, etc.)                                                                            
        r := reader("Foo: bar\r\n" +
                "Content-Language: en\r\n" +
                "SID : 0\r\n" +
                "Audio Mode : None\r\n" +
                "Privilege : 127\r\n\r\n")
        m, err := r.ReadMIMEHeader()
        want := MIMEHeader{
                "Foo":              {"bar"},
                "Content-Language": {"en"},
                "Sid":              {"0"},
                "Audio Mode":       {"None"},
                "Privilege":        {"127"},
        }
        if !reflect.DeepEqual(m, want) || err != nil {
                t.Fatalf("ReadMIMEHeader =\n%v, %v; want:\n%v", m, err, want)
        }
}               

I know of no reason to accept leading whitespace, though, and I'm surprised we do.

Change https://golang.org/cl/75350 mentions this issue: net/textproto: retain leading whitespace in the first header key

We fixed the parser to ignore headers like this:

GET / HTTP/1.1
  Host: foo.com
Accept-Encoding: gzip

However, headers like the following used to return an error, but don't anymore (note the lack of colon in the second line):

GET / HTTP/1.1
  Host foo.com
Accept-Encoding: gzip

For Go 1.10, I will fix the above issue. For Go 1.11, we will reconsider if textproto.Reader needs multiple modes to more precisely handle the various RFCs it implements and to allow the caller to control whether we ignore invalid header lines or fail loudly.

I'd be in favor of reverting the net/textproto changes entirely for Go 1.10 and instead writing the few line header parser in net/http that net/http needs (also for Go 1.10).

All this effort working across two packages and differing needs isn't worth it.

I'd be in favor of reverting the net/textproto changes entirely for Go 1.10 and instead writing the few line header parser in net/http that net/http needs (also for Go 1.10).

I agree that net/texproto is doing too much, but I don't think that's a complete solution. It solves the problem for net/http but no one else. net/textproto is still wrong: it should not accept headers where the first line begins with a space. This holds for both RFC 5322 (smtp) and RFC 7230 (http). And, there are people outside net/http using net/textproto to parse HTTP headers (I know of one fairly major use inside Google). Those parsers will be broken if we don't fix net/textproto.

Change https://golang.org/cl/80055 mentions this issue: net/textproto: reject all headers with a leading space

@tombergan, what remains here?

Nothing remains for 1.10. What remains is deciding whether or not to make a larger change for Go 1.11, which could include one or both of:

• Copy ReadMIMEHeader into a private implementation of net/http and fine tune the behavior for HTTP clients and servers. Perhaps also do the same for net/mail and net/smtp.

• Add a "mode" enum to textproto.Reader that would add loose/strict modes and specialize ReadMIMEHeader for each RFC. e.g., ModeSMTP, ModeLooseHTTP, ModeStrictHTTP.

SGTM. Thanks for the milestone update.