Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: io: CopyReader to the io package #64274

Closed
lkeix opened this issue Nov 20, 2023 · 4 comments
Closed

proposal: io: CopyReader to the io package #64274

lkeix opened this issue Nov 20, 2023 · 4 comments
Milestone

Comments

@lkeix
Copy link

lkeix commented Nov 20, 2023

proposal: io: CopyReader to the io package

Abstract

This proposal suggests the addition of the CopyReader function to the io package in the Go standard library.

Background

When processing HTTP requests, both middleware and handlers often need to read the request body. However, as the io.Reader interface, from which the body is read, is a forward-only stream, you run into an issue if you need to read it more than once. Once the body is read in the handler (endpoint), it has been consumed and cannot be read again. This characteristic of io.Reader causes difficulty because there's no capability for re-reading a stream that's already been read.

Proposal

  • CopyReader is a function in io package.
  • CopyReader is got 2 parameters src / dst that is io.Reader type.
  • CopyReader return error

CopyReader example is below.

  var dst io.Reader
  src := strings.NewReader("hogehoge")
  err := io.CopyReader(src, dst) // dst has hogehoge

Rationale

Adding an io.CopyReader function to the io package would address a common pain point in working with io.Reader interfaces: the inability to re-read an already consumed stream. This is particularly troublesome when developing middleware for HTTP servers where both the middleware and the handler want to read the request body. Once the body is consumed by the io.Reader in the handler, it cannot be read again in the middleware, which limits the ability to process request data in middleware functionality such as logging or data validation.

It encourages developers to read from the io.Reader freely, without having to excessively plan ahead or manage temporary workaround solutions that is implemented by bytes/io.NopCloser. This gives more freedom to write more readable and maintainable code.

@lkeix lkeix added the Proposal label Nov 20, 2023
@gopherbot gopherbot added this to the Proposal milestone Nov 20, 2023
@gophun
Copy link

gophun commented Nov 20, 2023

CopyReader example is below.

  var dst io.Reader
  src := strings.NewReader("hogehoge")
  err := io.CopyReader(src, dst) // dst has hogehoge

How is this different from

var dst bytes.Buffer
src := strings.NewReader("hogehoge")
_, err := dst.ReadFrom(src)

?

@panjf2000
Copy link
Member

I think you can do this in your middleware:

var buf bytes.Buffer
tee := io.TeeReader(c.Request.Body, &buf)
body, _ := io.ReadAll(tee)
c.Request.Body = io.NopCloser(&buf)

@panjf2000
Copy link
Member

panjf2000 commented Nov 20, 2023

Besides, I have no idea how we're going to repopulate an arbitrary src Reader in io.CopyReader, let alone the fact that src might be read-only.

This proposal doesn't look feasible to me.

@lkeix
Copy link
Author

lkeix commented Nov 20, 2023

@panjf2000
Thank you, this was informative.
Indeed, io.CopyReader being a wrapper around io.TeeReader does not enhance its extensibility.

@lkeix lkeix closed this as completed Nov 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants