code review 47770043: go.net/html: Expose data read from the input reader but...

go.net/html: Expose data read from the input reader but not yet tokenized in Tokenizer.

This allows clients to efficiently reconstruct the original input in the case of ErrBufferExceeded. TestMaxBufferReconstruction now properly verifies this.

[piatek, 2014-01-04 05:54:48]

Hello bradfitz (cc: golang-codereviews@googlegroups.com),

I'd like you to review this change to
http://code.google.com/p/go.net

[bradfitz, 2014-01-06 18:30:24]

https://codereview.appspot.com/47770043/diff/80001/html/token.go
File html/token.go (right):

https://codereview.appspot.com/47770043/diff/80001/html/token.go#newcode291
html/token.go:291: return z.buf[z.raw.end:]
I was going to complain that this leaks the private z.buf outside this package,
but it already does elsewhere, so whatever.

https://codereview.appspot.com/47770043/diff/80001/html/token_test.go
File html/token_test.go (right):

https://codereview.appspot.com/47770043/diff/80001/html/token_test.go#newcode509
html/token_test.go:509: assembled, err :=
ioutil.ReadAll(io.MultiReader(&tokenized, bytes.NewBuffer(z.Buffered()), r))
bytes.NewReader instead of NewBuffer

[piatek, 2014-01-06 18:43:49]

https://codereview.appspot.com/47770043/diff/80001/html/token.go
File html/token.go (right):

https://codereview.appspot.com/47770043/diff/80001/html/token.go#newcode291
html/token.go:291: return z.buf[z.raw.end:]
On 2014/01/06 18:30:24, bradfitz wrote:
> I was going to complain that this leaks the private z.buf outside this
package,
> but it already does elsewhere, so whatever.

ack

https://codereview.appspot.com/47770043/diff/80001/html/token_test.go
File html/token_test.go (right):

https://codereview.appspot.com/47770043/diff/80001/html/token_test.go#newcode509
html/token_test.go:509: assembled, err :=
ioutil.ReadAll(io.MultiReader(&tokenized, bytes.NewBuffer(z.Buffered()), r))
On 2014/01/06 18:30:24, bradfitz wrote:
> bytes.NewReader instead of NewBuffer

Done.

[bradfitz, 2014-01-06 18:49:56]

LGTM

[bradfitz, 2014-01-06 18:51:25]

*** Submitted as
https://code.google.com/p/go/source/detail?r=2b669bbfb7ba&repo=net ***

go.net/html: Expose data read from the input reader but not yet tokenized in
Tokenizer.

This allows clients to efficiently reconstruct the original input in the case of
ErrBufferExceeded. TestMaxBufferReconstruction now properly verifies this.

R=bradfitz
CC=golang-codereviews
https://codereview.appspot.com/47770043

Committer: Brad Fitzpatrick <bradfitz@golang.org>