code review 170030043: bufio: fix reading of many blank lines in a row

bufio: fix reading of many blank lines in a row

Fixes issue 9020.

[rsc, 2014-11-06 00:53:49]

Hello r (cc: golang-codereviews@googlegroups.com),

I'd like you to review this change to
https://code.google.com/p/go/

[bradfitz, 2014-11-06 01:04:41]

LGTM but leaving for r.
 On Nov 5, 2014 9:53 PM, <rsc@golang.org> wrote:

> Reviewers: r,
>
> Message:
> Hello r (cc: golang-codereviews@googlegroups.com),
>
> I'd like you to review this change to
> https://code.google.com/p/go/
>
>
> Description:
> bufio: fix reading of many blank lines in a row
>
> Fixes issue 9020.
>
> Please review this at https://codereview.appspot.com/170030043/
>
> Affected files (+14, -1 lines):
>   M src/bufio/scan.go
>   M src/bufio/scan_test.go
>
>
> Index: src/bufio/scan.go
> ===================================================================
> --- a/src/bufio/scan.go
> +++ b/src/bufio/scan.go
> @@ -128,9 +128,10 @@
>                         }
>                         s.token = token
>                         if token != nil {
> -                               if len(token) > 0 {
> +                               if s.err == nil || advance > 0 {
>                                         s.empties = 0
>                                 } else {
> +                                       // Returning tokens not advancing
> input at EOF.
>                                         s.empties++
>                                         if s.empties > 100 {
>                                                 panic("bufio.Scan: 100
> empty tokens without progressing")
> Index: src/bufio/scan_test.go
> ===================================================================
> --- a/src/bufio/scan_test.go
> +++ b/src/bufio/scan_test.go
> @@ -489,6 +489,18 @@
>         }
>  }
>
> +func TestBlankLines(t *testing.T) {
> +       s := NewScanner(strings.NewReader(strings.Repeat("\n", 1000)))
> +       for count := 0; s.Scan(); count++ {
> +               if count > 2000 {
> +                       t.Fatal("looping")
> +               }
> +       }
> +       if s.Err() != nil {
> +               t.Fatal("after scan:", s.Err())
> +       }
> +}
> +
>  type countdown int
>
>  func (c *countdown) split(data []byte, atEOF bool) (advance int, token
> []byte, err error) {
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-codereviews" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-codereviews+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

[rsc, 2014-11-06 01:11:46]

An alternative would be to use if advance > 0 alone, so that getting stuck
anywhere in the input counts as a failure, not just getting stuck at EOF
(or other file-ending error). But getting stuck at EOF is the specific new
problem made easy by 1.4, so I only try to detect that one case. Getting
stuck in the middle was just as easy in 1.3 as in 1.4.

Russ

[r, 2014-11-06 02:31:08]

LGTM

[rsc, 2014-11-06 03:50:32]

*** Submitted as https://code.google.com/p/go/source/detail?r=c3de8d47b282 ***

bufio: fix reading of many blank lines in a row

Fixes issue 9020.

LGTM=bradfitz, r
R=r, bradfitz
CC=golang-codereviews
https://codereview.appspot.com/170030043