Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bufio: bug in the emptyFinalToken example for Scanner #25909

Closed
pam4 opened this issue Jun 15, 2018 · 2 comments
Closed

bufio: bug in the emptyFinalToken example for Scanner #25909

pam4 opened this issue Jun 15, 2018 · 2 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Unplanned

Comments

@pam4
Copy link

pam4 commented Jun 15, 2018

The SplitFunc in the emptyFinalToken example for bufio.Scanner doesn't handle the case of requesting more data if necessary.

The example only happens to work because the whole input is returned by a single call to Read (it uses a strings.Reader).

This should be enough to fix it:

@@ -21,7 +21,10 @@
        // There is one final token to be delivered, which may be the empty string.
        // Returning bufio.ErrFinalToken here tells Scan there are no more tokens after this
        // but does not trigger an error to be returned from Scan itself.
-       return 0, data, bufio.ErrFinalToken
+       if atEOF {
+           return 0, data, bufio.ErrFinalToken
+       }
+       return 0, nil, nil
    }
    scanner.Split(onComma)
    // Scan.

The problem is demonstrated here:
https://play.golang.org/p/CM-sWovg8mg
@meirf
Copy link
Contributor

meirf commented Jun 16, 2018

I think we should include logic like the OP includes since that's an important general pattern for the Scanner and none of the other examples directly include logic for requesting more data - all the other examples (transitively) have similar checks but only via their usage of the Scan* SplitFuncs.

cc: @ianlancetaylor

@andybons andybons added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 18, 2018
@andybons andybons added this to the Unplanned milestone Jun 18, 2018
@gopherbot
Copy link

Change https://golang.org/cl/157758 mentions this issue: bufio: fix emptyFinalToken example to handle multiple Reads

@golang golang locked and limited conversation to collaborators Mar 1, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
4 participants
@andybons @meirf @gopherbot @pam4