-
Notifications
You must be signed in to change notification settings - Fork 18k
bufio: Read reads at most once #42332
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
|
You should make block with two triple-backquote ```.
|
@davecheney If I'm using a bufio.Reader to read a large data file I will get some truncated reads. To work around this, I'd need to check for the bufio.Read output for truncated read and then call the function again... In C I wouldn't have to do this myself, the Read function would do it for me. It would be a straight forward addition to the package |
@JordanStoker i'm sorry, my initial comment was incorrect. I deleted it, I'm sorry. |
The short version is, io.Reader doesn't guarentee to read anything more than 1 character, in fact, it can read 0 characters without error. |
@mattn Thanks for reminding |
I am not sure how this could be changed with the compatibility promise. I might be missing something, could you clarify how the documented suggestion of using io.ReadFull doesn't help in your scenario? |
@toothrot In my scenario, I am unpacking small data structures from a large file. Here is a workaround I've used where I would recursively call the bufio.Reader until my data has been fully read. Recursion is overkill in my scenario (my structures are much much smaller than the bufio buffer), a single bufio.Reader.Read would have sufficed.
For the opposite scenario where I want to pack data in a file, the bufio.Writer.Write (https://golang.org/src/bufio/bufio.go) would iteratively write the data if the data is larger than the buffer available. I would have expected the same behaviour from bufio.Reader. |
@JordanStoker it looks like you can replace your |
What version of Go are you using (
go version
)?go version go1.15.3 linux/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env
)?Ubuntu 18.04 amd64
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jordan/.cache/go-build"
GOENV="/home/jordan/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/jordan/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/jordan/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/snap/go/6633"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/snap/go/6633/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build344922599=/tmp/go-build -gno-record-gcc-switches"
What did you do?
Called bufio.Read multiple times until the next read required more bytes than was available i.e. len(buf) > bufio.Available(). When this happened, bufio copied as much as was available in the current buffer but did not attempt to read the remaining bytes requested.
https://play.golang.org/p/M953MGK136s
What did you expect to see?
I expected bufio.Read to recursively perform an io.Read until the requested buffer length was fully read.
Although the documentation does suggest this behavior it is inconsistent with its C-equivalent and the rest of this package - e.g. Discard, write (would write at most twice, w)
What did you see instead?
An incomplete read with no attempt to read the remaining buffer
The text was updated successfully, but these errors were encountered: