Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

io: improve Reader documentation relating to EOF and zero-length []byte #10182

Closed
kortschak opened this issue Mar 17, 2015 · 9 comments
Closed

Comments

@kortschak
Copy link
Contributor

The documentation for io.Reader states that a Read following a Read that has returned a non-zero number of bytes at the end of the input stream "should return 0, EOF regardless."

This is not what {bytes,strings}.Reader do:

http://play.golang.org/p/b7X-Te_ltg

If the "regardless" were weakened to specify "for non-zero length []byte" the documentation would match the current behaviour. (I think allowing a return of 0, io.EOF for zero-length []byte should be retained).

See https://groups.google.com/d/topic/golang-nuts/hdVJF0hU2o8/discussion for discussion.

@minux
Copy link
Member

minux commented Mar 17, 2015 via email

@peterGo
Copy link
Contributor

peterGo commented Mar 17, 2015

The documentaion is correct. The example is wrong.

In the example (http://play.golang.org/p/b7X-Te_ltg), the first Read does not encounter EOF:

r.Read(make([]byte, len(text)))

EOF will not happen until a Read with len(p) > 0.

In the example, the regardless condition has not been met; it's not "at the end of the input stream."

@minux
Copy link
Member

minux commented Mar 17, 2015 via email

@peterGo
Copy link
Contributor

peterGo commented Mar 17, 2015

A convenient IO model is:

while (Read(buf) == nil) {
// process buf
}

For example, bufio.Scanner.

EOF is not signaled until after all data has been read.

For example, I have a read/write file or buffer that contains 10 bytes and I read 10 bytes, I don't have EOF. I'll append some more data before the next read. Don't read ahead to determine EOF.

Go has a complicated io.Reader model which has two allowable EOF behaviours.

"When Read encounters an error or end-of-file condition after successfully reading n > 0 bytes, it returns the number of bytes read. It may return the (non-nil) error from the same call or return the error (and n == 0) from a subsequent call. An instance of this general case is that a Reader returning a non-zero number of bytes at the end of the input stream may return either err == EOF or err == nil. The next Read should return 0, EOF regardless."

Packages strings, bytes.Buffer, and bufio Readers don't return EOF until after there are no more bytes left to read. To see this behaviour, see DanK's earlier example, not the abbreviated one he posted here:

http://play.golang.org/p/JwhBtBPbcM

const text = "Hello, playground" // len(text) == 17
r := strings.NewReader(text)
fmt.Println(r.Read(make([]byte, len(text))))
fmt.Println(r.Read([]byte{}))
fmt.Println(r.Read([]byte{0}))

Output:

17
0
0 EOF

Which works as documented (http://golang.org/pkg/io/#Reader).

@kortschak
Copy link
Contributor Author

At which point in the set of reads below is the point time there are no more bytes to read?

I would claim it is after the first read.

On 17/03/2015, at 6:55 PM, "peterGo" <notifications@github.commailto:notifications@github.com> wrote:

Packages strings, bytes.Buffer, and bufio Readers don't return EOF until after there are no more bytes left to read. To see this behaviour, see DanK's earlier example, not the abbreviated one he posted here:

http://play.golang.org/p/JwhBtBPbcM

const text = "Hello, playground" // len(text) == 17
r := strings.NewReader(text)
fmt.Println(r.Read(make([]byte, len(text))))
fmt.Println(r.Read([]byte{}))
fmt.Println(r.Read([]byte{0}))

@bmea
Copy link

bmea commented Mar 17, 2015

If the Reader know that it only has len(p) bytes left, why forbid it to return EOF when it could potentially save another call to Read?

You yourself do so with in the CL. Consider the case of len(text)==len(p)==0.

Further, where the following case was once covered by the docs, it is now undefined:

r.Read(make([]byte, len(text)+1))  // len(text), EOF
r.Read(r.Read([]byte{}))           // 0, EOF

@minux
Copy link
Member

minux commented Mar 17, 2015

On Tue, Mar 17, 2015 at 8:57 AM, sabroad notifications@github.com wrote:

If the Reader know that it only has len(p) bytes left, why forbid it to
return EOF when it could potentially save another call to Read?

You yourself do so with in the CL. Consider the case of len(p)==0.

Further, where the following case was once covered by the docs, it is now
undefined:

r.Read(make([]byte, len(text)+1)) // len(text), EOF
r.Read(r.Read([]byte{})) // 0, EOF

I don't understand. My CL clarifies that it's always valid to return 0,
nil for len(p) == 0.

In your example, the first line can still return len(text), EOF, and that
haven't been changed. (Note "can", not "must")

the change is that the 2nd call is allowed to return 0, EOF or 0, nil.
Depending how the Reader handles len(p) == 0 case.

Ideally, you should never pass a zero-length []byte to Read. It's all.

@kortschak
Copy link
Contributor Author

Since the addition of text has been rejected, an alternative would be to remove the regardless clause to become, "The next Read should return 0, EOF."

I take @robpike's point that the final paragraph allows returning 0, nil when len(p) == 0. However, given that the regardless is unnecessary and so easily read as an emphatic (I actually still find it hard to read otherwise even with the external commentaries), its presence muddies the clarity of the documentation by making the reading "The next Read should return 0, EOF regardless [of all other conditions]."

@gopherbot
Copy link

CL https://golang.org/cl/11674 mentions this issue.

@rsc rsc closed this as completed in 643ef15 Jun 29, 2015
@mikioh mikioh modified the milestones: Go1.5, Go1.5Maybe Jun 30, 2015
@golang golang locked and limited conversation to collaborators Jun 29, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

7 participants