New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bufio: Scanner considers a buffer of length = len(token) as not enough #43183
Comments
I don't see an easy fix here. The code would have to do a look ahead when it reaches the end of the buffer, which would be complicated, and would help approximately nobody. This should probably just be mentioned in the documentation. |
You are using change to or add |
|
I know how to workaround with |
replace string to string with EOL to simulate reading file |
The issue is not about how to workaround or simulate something. I can make it work. The issue emphasizes how EOL and EOF are treated differently in terms of how big your buffer should be for the scanner. This behavior is not immediately obvious (and of course it doesn't matter what we read, a file can also not have EOL). |
scanline without EOF and EOL is not working , take a look here I think you would like to add additional "if" to scanline function to read string without EOF and EOL |
I'm not sure why you say "without EOF and EOL" there is EOF in my case. In this case EOF is just a contract supported by the Reader interface. No more data to read it will return EOF. |
I investigated this issue and you are right something is bad. |
The code at scan.go indicate that if
|
Fixes golang#43183 Change-Id: I50d99ef8ed513bba47166a25ea5c7c80cd8bd799
Change https://go.dev/cl/518860 mentions this issue: |
Fixes golang#43183 Change-Id: I50d99ef8ed513bba47166a25ea5c7c80cd8bd799
Fixes golang#43183 Change-Id: I50d99ef8ed513bba47166a25ea5c7c80cd8bd799
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Haven't tried
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
Golang bufio.Scanner considers a buffer of length = len(token) as not enough, please see example below:
(full code is here https://play.golang.org/p/3e95HG3j4H-)
What did you expect to see?
No error (output:
Token=12, Err=<nil>
)What did you see instead?
Error (output:
Token=, Err=bufio.Scanner: token too long
)The text was updated successfully, but these errors were encountered: