Skip to content

proposal: io: change Copy error to distinguish reader from writer #44571

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rockmenjack opened this issue Feb 24, 2021 · 14 comments
Closed

proposal: io: change Copy error to distinguish reader from writer #44571

rockmenjack opened this issue Feb 24, 2021 · 14 comments

Comments

@rockmenjack
Copy link

I encountered a need where different operation is performed based on the error from io.Copy is a read error or write error.

Currently I can only guess the following:

  1. err != nil && n > 0 likely be a write error, but can be a successful write followed by a failed read
  2. err != nil && n == 0 can be either a failed read or failed write

It would be good such check as below is valid.

_, err := io.Copy(dst, src)

if errors.Is(err, io.WriteError) {
  // do some stuff
} else if errors.Is(err, io.ReadError) {
  // do other stuff
}

I am happy to contribute is if this is nice to have.

@gopherbot gopherbot added this to the Proposal milestone Feb 24, 2021
@davecheney
Copy link
Contributor

If you were able to determine that the reader, or the writer, was the cause of the error, what would that let you do?

@ianlancetaylor
Copy link
Member

Can you give an example of the case which you need to distinguish? In many cases I would expect the error would be an os.PathError, so you could look at the Path field.

@rsc
Copy link
Contributor

rsc commented Feb 24, 2021

At this point I'm not at all sure we can change the errors being returned here.

We could potentially provide Reader and Writer wrappers that remember errors so that you can check them after the call, but they'd have to forward ReadFrom etc checks, which would require other changes to Copy.

@rsc rsc changed the title proposal: make error returned from io.Copy distinguishable from reader or writer proposal: io: change Copy error to distinguish reader from writer Feb 24, 2021
@rsc
Copy link
Contributor

rsc commented Feb 24, 2021

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rockmenjack
Copy link
Author

Can you give an example of the case which you need to distinguish? In many cases I would expect the error would be an os.PathError, so you could look at the Path field.

                    tcp svr           vsock
                    +-----+          +-----+
            <-------+     +<---------+     |  // io.Copy(tcp_conn, vsock_conn) call it copy1
some client         |     |          |     |
            ------->+     +--------->+     |  // io.Copy(vsock_conn, tcp_conn) call it copy2
                    +-----+          +-----+

                | -----my go program ----- |

Condiser below scenarios:

  1. in copy1, if client actively closed receive side, then it is a write error, so I won't close vsock_conn, because client may still sending in copy2
  2. in copy2, for some reason, write to vsock_conn failed, still a write error, so I won't close tcp_conn, as client may still receiving in copy1
  3. in both copy1 and copy2, if it is a read error, I just close the connections.

@rockmenjack
Copy link
Author

rockmenjack commented Feb 25, 2021

At this point I'm not at all sure we can change the errors being returned here.

We could potentially provide Reader and Writer wrappers that remember errors so that you can check them after the call, but they'd have to forward ReadFrom etc checks, which would require other changes to Copy.

Is it feasible to change in below way?

type ErrWrite struct{ err error }
func (e ErrWrite) Unwrap() error {return e.err}
type ErrRead struct{ err error }
func (e ErrRead) Unwrap() error {return e.err}

// ...  code snippets from copyBuffer
		nr, er := src.Read(buf)
		if nr > 0 {
			nw, ew := dst.Write(buf[0:nr])
			if nw < 0 || nr < nw {
				nw = 0
				if ew == nil {
					ew = ErrWrite{errInvalidWrite} // wrap
				}
			}
			written += int64(nw)
			if ew != nil {
				err = ErrWrite{ew} // wrap
				break
			}
			if nr != nw {
				err = ErrWrite{ErrShortWrite} // wrap
				break
			}
		}
		if er != nil {
			if er != EOF {
				err = ErrRead{er} // wrap
			}
			break
		}
	}
// ...

@rsc
Copy link
Contributor

rsc commented Mar 10, 2021

This seems like something that does not come up a lot. It is also specific to io.Copy.
However, the more general problem of "was the failure due to I/O or something else" comes up more often.
For example https://go.googlesource.com/tools/+/refs/heads/master/godoc/server.go#512 has an error-saving writer wrapper so that it can tell whether a template execution failed due to a real template problem or a (not very important) I/O problem writing to an HTTP client.
If we provided general error-saving wrappers for Reader and Writer then that could be more useful.
And then when you wanted to answer the question for Copy, you'd just add one or both.

We just need names for the wrappers. I wrote one of these a few weeks ago and called it a WriteErrorSaver, which I'm not wild about. Any better, shorter names?

@rsc
Copy link
Contributor

rsc commented Mar 24, 2021

OK, in the absence of better names, it sounds like the proposal is

type WriteErrorSaver struct { W io.Writer; Err error }
func (w *WriteErrorSaver) Write(b []byte) (int, error) {
    n, err := w.W.Write(b)
    if err != nil {
        w.Err = err
    }
    return n, err
}

and similarly for Read. This has the property of breaking the io.Copy optimizations for ReadFrom and WriteTo.
(I suppose io.Copy could special case this implementation.)

Thoughts?

@kortschak
Copy link
Contributor

Given the impact on the optimisations, the size of the code required, and the frequency of need; would it not be better to have this as an example showing how specific io.Reader/io.Writer error capture can be implemented?

@rockmenjack
Copy link
Author

rockmenjack commented Mar 25, 2021

(I suppose io.Copy could special case this implementation.)

In this way we can still have the optimizations.

@rsc
Copy link
Contributor

rsc commented Mar 31, 2021

People can write the wrappers I listed above themselves. There's no clear reason they must be in the standard library.
Given the lack of enthusiasm here, it sounds like maybe we shouldn't make any changes.

@rockmenjack
Copy link
Author

Maybe just append some guide in the doca?

@rsc
Copy link
Contributor

rsc commented Apr 7, 2021

Based on the discussion above, this proposal seems like a likely decline.
— rsc for the proposal review group

@rsc
Copy link
Contributor

rsc commented Apr 14, 2021

No change in consensus, so declined.
— rsc for the proposal review group

@rsc rsc closed this as completed Apr 14, 2021
@golang golang locked and limited conversation to collaborators Apr 14, 2022
@rsc rsc moved this to Declined in Proposals Aug 10, 2022
@rsc rsc added this to Proposals Aug 10, 2022
@rsc rsc removed this from Proposals Oct 19, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

6 participants