When does this come up in real code? We don't want to add new API just to avoid a hole. We want to add it if it is useful.

Also, see https://golang.org/doc/faq#x_in_std .

Thanks.

When does this come up in real code? We don't want to add new API just to avoid a hole. We want to add it if it is useful.

Also, see https://golang.org/doc/faq#x_in_std .

Thanks.

@ianlancetaylor I admit that I ran into this mostly in writing tests, but I decided to open this issue when I was writing a sentinel value for type that embedded a reader. The project is proprietary but it looked something similar to the following:

// loose example, not actually related to file systems
type FileLoader interface {
    LoadFile(name, path string) (CustomFileType, err)
}

// actually in use, the type is implemented over a reader the contains various
// data segments together. (Yes a ReadSeaker is more efficient, ignore that for now)
type fileLoader io.Reader

func (fl fileLoader) LoadFile(name, path string) (file CustomFileType, err error) {
    // does stuff, pulls specified data from reader
}

// ok here's where io.Empty comes in

// example 1, use standard input, this is actually lifted verbatim from the project
var CommonUseCase1 FileLoader = fileLoader(os.Sdtin)

// example 2, get a reference to a common use case from another internal package,
// in the actual project it creates a reader with data that is pulled from a database
var CommonUseCase2 FileLoader = fileLoader(internalPackage.GetCommonUseCase())

// example 3, this is where the missing io.Empty is noticeably weird, in the project,
// this value is used when the client package has failed to setup the details for
// CommonUseCase2, since it's possible that the necessary details aren't available
// for a legitimate reason, rather than return an error we give a safe instance
// (which might never be read from depending on the user)
var CommonUseCase3 FileLoader = fileLoader(strings.NewReader(""))

// compare to:
var CommonUseCase4 FileLoader = fileLoader(io.Empty))

EDIT: typo, I wrote this on my phone

A value like io.Empty that implements io.Reader, where every call returns (n = 0, err = EOF) and does nothing to the passed byte slice.

You can use iotest.ErrReader.

package main

import (
	"fmt"
	"io"
	"testing/iotest"
)

func main() {
	r := iotest.ErrReader(io.EOF)
	n, err := r.Read(nil)
	fmt.Printf("n:   %d\nerr: %q\n", n, err)
}

You can use iotest.ErrReader.

package main

import (
	"fmt"
	"io"
	"testing/iotest"
)

func main() {
	r := iotest.ErrReader(io.EOF)
	n, err := r.Read(nil)
	fmt.Printf("n:   %d\nerr: %q\n", n, err)
}

@tdakkota that's a good alternative for writing tests, thanks! I think that importing "testing/*" is probably not a good idea in production code though?

As noted, this is already possible with testing/iotest, as well as bytes.NewReader(nil) (and others).
It doesn't seem like we need even more ways to do this?

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

If #47487 is accepted, it will also be possible to write func(_ []byte) (int, error) { return 0, io.EOF } inline if not wanting to import testing/iotest.

As noted, this is already possible with testing/iotest, as well as bytes.NewReader(nil) (and others).
It doesn't seem like we need even more ways to do this?

@rsc It seems like the actual problem is discoverability? It is not obvious that either iotest.ErrReader(io.EOF) or strings.NewReader("") is the correct idiom for a zero reader.

If #47487 is accepted, it will also be possible to write func(_ []byte) (int, error) { return 0, io.EOF } inline if not wanting to import testing/iotest.

@kortschak I think what @yuri-norwood is trying to solve is the discoverability problem. How does the author find out what the correct idiom is? And how does someone new to Go understand what has been written?

@kortschak I think what @yuri-norwood is trying to solve is the discoverability problem. How does the author find out what the correct idiom is? And how does someone new to Go understand what has been written?

@sfllaw Yes, my main concern is that there is an obvious idiom for one of the most common interfaces in the language, but not another, equally important and possibly even more common interface. If io.Discard didn't exist, I would have probably used something like bytes.NewBuffer(nil).Write(fooBytes), and not thought much more of it than strings.NewReader("").Read(barBytes).

I don't understand why this even rises to the level of idiom.
How often do you need to process zero bytes of input?

Put another way, lots of shell scripts contain > /dev/null.
Many fewer contain < /dev/null. Why is that case so important in Go?

It seems like a much more common idiom would be echo data | program,
or in Go, how to provide a specific piece of data as a Reader.
For that, bytes.NewReader(data) or strings.NewReader(data) are the answer.
Providing no data at all is just a special case of that, and a relatively uncommon one.
Why not rely on the general purpose solution?

Note that bytes.NewReader(nil) is not the same as io.Discard:
it accumulates the data instead of discarding it.

 ... bytes.NewReader(nil) is not the same as io.Discard ...

@rsc, I feel like this is the main problem, perhaps expanding the io package isn't the right way to address the idiomatic gap, but maybe a note should be added to the documentation of io.Discard, I'm open to alternatives. My proposal here is to help people new to writing Go, or new to reading and understanding Go written by others. To use your /dev/null example, while it isn't common to read from /dev/null, it does occur and when it does its meaning is clear, and nobody would suggest creating and empty file to read from as an alternative. As I said, if adding io.Empty is not a viable option, then there should be clarification on A) why, and B) what to use instead.

@yuri-norwood Perhaps you should change your proposal to just improve the standard library’s documentation, so that when someone is reading about io.Reader, and want to know how to test their function against empty readers, they will read about strings.NewReader("")? (Or whatever idiomatic option we want to endorse.)

This mirrors how io.Discard is mentioned right after io.Writer.

If you want to create your own EmptyReader, you could just write:

var EmptyReader io.Reader = strings.NewReader("")

@rsc From all the examples that I’ve seen, testing that your function handles an immediate io.EOF is the most common use-case for this request.

For research purposes, here are the Sourcegraph searches for a few possible recommendations:

• strings.NewReader("") (Go 22 results) (Public 500+ results)
• bytes.NewReader(nil) (Go 7 results) (Public 500+ results)
• iotest.ErrReader(io.EOF) (Go: 0 results) (Public: 2 results)

@yuri-norwood Perhaps you should change your proposal to just improve the standard library’s documentation, so that when someone is reading about io.Reader, and want to know how to test their function against empty readers, they will read about strings.NewReader("")? (Or whatever idiomatic option we want to endorse.)

This mirrors how io.Discard is mentioned right after io.Writer.

If you want to create your own EmptyReader, you could just write:

var EmptyReader io.Reader = strings.NewReader("")

@sfllaw, good idea, I've update the proposal title, but I'll leave the original request description for historical purposes.

@rsc I think that adding this note to the documentation of io.Reader would at least make discovery easier. Are there any objections to using strings.NewReader("")?

Documentation changes don't need to be proposals, so taking this out of the proposal process.

Ok so I've left this proposal for a while, but I keep coming back to the same problem, and the same gut feeling that there should be an empty reader.

I think my original example was a little too vague and too abstract, so here's a more "real world" example, a brainf*ck interpreter I've been working on. I was hesitant to use this as an example for fear that the solution would be "just pass a nil reader" which isn't what I want as a nil reader should be an error, just as a nil writer is.

In this example, I have a bfk.Program type with a method Execute(input io.Reader, output io.Writer) error. The most common way this method would be called is by passing os.Stdin and os.Stdout, which is pretty straightforward, but not all programs process input, especially in BF, where most programs are along the lines of a Hello World or 99 Bottles of Beer on the Wall. In these cases passing standard input is fine, it never gets read, but only if the output is to standard output.

Another use case is to write a program that generates and image in PNG, in this case, the output parameter would just be a file that the caller creates and closes afterwards with defer. But this is where the issue comes in, passing standard input into a program and redirecting the output into a file is a common pattern both in Go and in shell scripts, however only when the input is actually being processed into the output.

Compare:

var file os.File
var program *bfk.Program

// ...
 
err := program.Execute(os.Stdin, file)

To:

var file os.File
var program *bfk.Program

// ...
 
err := program.Execute(io.Empty, file)

I think this more clearly demonstrates my proposal here, in the first example the intent is implied to be reading standard input and writing to the file, which is a valid intent but not what is actually needed here. The former is also not safe if a change is made to the program (which is parsed from a string) such that it attempts to read its input stream, now it is consuming standard input (which may affect later use of the stream) and may be reading garbage data.

I hope that this clears up the intention of this proposal. I think that adding a note to the documentation of io.Discard may help mitigate some of the confusion in these cases, but I also believe that the use cases are valid enough to justify also providing the solution, not just saying that one exists if you make it yourself.

Edit: Typos

Thanks, but we've made a decision here. One example is not enough to change that decision. You can write, in your own code,

var Empty = strings.NewReader("")

and use that. Let's document this as suggested and move on. Thanks.

@yuri-norwood I’m having a lot of trouble understanding how your example makes this more clear?

For instance, I would write this as:

var filename = flag.String("o", "", "output filename")

func main() {
	// ...

	var input io.Reader
	var output io.Writer
	if filename != "" {
		input = os.Stdin

		f, err := os.OpenFile(filename, os.O_RWDR|os.O_CREATE, 0o755)
		if err != nil {
			log.Fatal(err)
		}
		defer f.Close()
		output = f
	}

	// ...

	if err := program.Execute(input, output); err != nil {
		log.Fatal(err)
	}
}

In this case, your program wouldn’t do any work if there was nothing to read or write:

func Execute(input io.Reader, output io.Reader) error {
	if input == nil || output == nil {
		return nil
	}

	// ...
}

This kind of early exit is very common in Go programs and would make your package robust against nil arguments.

This kind of early exit is very common in Go programs and would make your package robust against nil arguments.

@sfllaw In the case of this particular package, nil readers will result in an ErrBadReader, similarly a nil writer would result in an ErrBadWriter. Try to image that there is no io.Discard, your argument would be that if someone wanted an equivalent to /dev/null they should just use a nil writer. One could even argue that since the null device is even called "null", a nil (null in most other languages) would be the "correct" way of discarding input.

But, since io.Discard does exist, we provide syntactic sugar for the end user rather than telling the user to create their own, or to use nil values.

@ianlancetaylor, I understand that I could write my own var Empty io.Reader = ... but since that would be specific to that one package it would only solve that one problem in that one package. This isn't a one off problem, I provided two independent "real code" examples, and I fully expect to encounter this again.

My goal isn't to just add boilerplate to every package that needs it, because this seams like pointless repetition. Especially since what I'm suggesting is that instead of me, a single person having the responsibility of providing one off instances of Empty, the standard library fills the hole that it itself created. Instead of me writing mypackage1.EmptyReader and mypackage2.EmptyStream etc, the standard library just provides a single io.Empty value for everyone, not just my code.

Change https://golang.org/cl/363666 mentions this issue: io: Document how to create an empty io.Reader

In the case of this particular package, nil readers will result in an ErrBadReader, similarly a nil writer would result in an ErrBadWriter.

Now I’m really confused! Why would you want to define those errors? How would you be able to tell that an io.Reader or io.Writer are bad? The point of these interfaces is that your program shouldn’t need to worry about the type of io.Reader, it should only try to read from a non-nil implementation.

If you are trying to define nil as invalid, I’d ask you to reconsider. I have a suggested an implementation above that makes the zero values useful: when input is nil, there is nothing to read; and when output is nil, there is nothing to write.

@sfllaw I've just had a chance to read your snippet, in your example a nil reader will result in the Execute method returning nil early, if this happened the program is not actually executed, nothing will be written to the output.

@yuri-norwood What else would an interpreter do if there is no input?

sfllaw@laptop:~$ sh -c ''
sfllaw@laptop:~$ js --eval ''
sfllaw@laptop:~$ python3 -c ''

@sfllaw you misunderstand, the input is for the program, not the compiler.

The program is created with a parser function https://pkg.go.dev/github.com/yuri-norwood/bfk#Parse

@sfllaw See the examples in the package for usage, e.g.

func main() {
	program, err := bfk.Parse(HelloWorld) // HelloWorld is the text of the program, omitted for brevity

	if err != nil {
		log.Fatalln(err)
	}

	err = program.Execute(os.Stdin, os.Stdout)

	if err != nil {
		log.Fatalln(err)
	}
}

@sfllaw this proposal would be to make it look like this instead.

func main() {
	program, err := bfk.Parse(HelloWorld) // HelloWorld is the text of the program, omitted for brevity

	if err != nil {
		log.Fatalln(err)
	}

	err = program.Execute(io.Empty, os.Stdout)

	if err != nil {
		log.Fatalln(err)
	}
}

you misunderstand, the input is for the program, not the compiler.

Sorry, I did misunderstand. I apologize.

I did not expect that bfk.(*Program).Execute has parameters for essentially standard input and output.

This reminds me of os/exec.Cmd.Stdin and os/exec.Cmd.Stdout. If you look, it seems like the nil Stdin and Stdout cases are handled, in the way that you expect:

I revise my suggestion so that (*Program).Execute reads like this:

func (p *Program) Execute(stdin io.Reader, stdout io.Reader) error {
	if stdin == nil {
		stdin = strings.NewReader("")
	}
	if stdout == nil {
		stdout = io.Discard
	}

	// Execute the program here.

	return nil
}

This way, your callers will not have to think about it:

func main() {
	program, err := bfk.Parse(HelloWorld) // HelloWorld is the text of the program, omitted for brevity
	if err != nil {
		log.Fatalln(err)
	}

	err = program.Execute(nil, os.Stdout)
	if err != nil {
		log.Fatalln(err)
	}
}

There is no need to expose the empty reader to your caller and the zero-values for your arguments have valid meanings.

Yes, this means that the library author, i.e. you, have to know that strings.NewReader works this way. Yes, that means there is a conceptual hole in the io package. But practically, I don’t think that having an io.Empty makes anything more clear.

This reminds me of os/exec.Cmd.Stdin and os/exec.Cmd.Stdout.

@sfllaw I hadn't thought of looking at these, I'll take your modification to my package under advisement. I've also taken a look at the documentation changes you've written, it looks nice and clear to me so thank you for making those changes 👍. It's a shame that there will be a hole in the standard library but I think it will at least be clearer going forward. Thank you for your time.

@ianlancetaylor, would you mind weighing in on https://go-review.googlesource.com/c/go/+/363666/4/src/io/io.go#585? I believe this issue has come to a resolution, so would like to resolve the CL. As stated in the comment, I am not tied to this particular resolution.

I don't feel strongly on the matter so I'm fine with Rob's call to not add this documentation to the io package. Sorry.

I have abandoned https://go-review.googlesource.com/c/go/+/363666