Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

io/ioutil: nopCloser should be public #23118

Closed
kegsay opened this issue Dec 13, 2017 · 11 comments
Closed

io/ioutil: nopCloser should be public #23118

kegsay opened this issue Dec 13, 2017 · 11 comments
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. Suggested Issues that may be good for new contributors looking for work to do.

Comments

@kegsay
Copy link

kegsay commented Dec 13, 2017

As of Go 1.9.2, ioutil.nopCloser remains private, meaning you cannot type assert on it. This was brought up years ago but a valid use case was never proposed.

I am implementing a RoundTripper which may need to retry sending its HTTP request. This requires either:

  • A new req.Body to read from (aka copying)
  • A way to seek the already-read req.Body back to the beginning.

Seeking is preferable but not everything will support that, so I fall back to copying as a last resort. My RoundTripper would attempt to type assert on Seeker.

In the calling code, I purposefully made a bytes.Reader which supports seeking and fed that to http.NewRequest, expecting my seeking code to fire. To my surprise, the seeking code failed to execute because it turns out that http.NewRequest will wrap up readers in nopCloser if they do not implement the Closer interface, and nopCloser does not implement the Seeker interface. I am unable to resolve this because ioutil.nopCloser is private, so I cannot type assert on it.

I can obviously work around this by implementing my own nopCloser but this severely limits the usefulness of the RoundTripper since consumers will expect to be able to just call http.NewRequest with a Seeker and have it Just Work™

@mvdan mvdan changed the title io/ioutil : nopCloser should be public io/ioutil: nopCloser should be public Dec 13, 2017
@mvdan mvdan added NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. Suggested Issues that may be good for new contributors looking for work to do. labels Dec 13, 2017
@davecheney
Copy link
Contributor

davecheney commented Dec 13, 2017 via email

@kegsay
Copy link
Author

kegsay commented Dec 13, 2017

The use case is to supply an http.Client which automatically retries requests without having to copy the request body.

In order for users to 'enable' this, they pass a reader which implements io.Seeker to NewRequest. If ioutil.nopCloser were public, this would be it and the rest would just work as the RoundTripper could inspect the Body and determine that it is a io.Seeker. However, because it is private, the RoundTripper cannot introspect in this way.

@davecheney
Copy link
Contributor

davecheney commented Dec 13, 2017 via email

@kegsay
Copy link
Author

kegsay commented Dec 13, 2017

This is for use in an internal project whereby I know more about the requests than just things like the method. As an aside, PUT should be regarded as idempotent, but the others are not.

@ianlancetaylor
Copy link
Contributor

I'm not sure I fully grasp the problem, but exporting ioutil.nopCloser so that other code can type assert to it, pull out the internal io.Reader, and type assert on that, seems like heaping Pelion upon Ossa.

I think what you are getting at is a language/library issue: there is no way to add a method to an existing value without obscuring the existing methods. Can we think of a way to solve that general problem?

In the meantime, I think that all net/http cares about is having a ReadCloser, so it seems to me that the simpler solution is to give it that, rather than try to undo the choices it makes when it sees something else. For example, write an adapter type just like io.nopCloser but use io.ReadSeeker rather than io.Reader, and use that adapter before going into net/http.

@bradfitz
Copy link
Contributor

https://golang.org/pkg/net/http/#Request.GetBody was added recently explicitly for request retries. You should use that instead.

We won't be exporting weird internals.

@kegsay
Copy link
Author

kegsay commented Dec 13, 2017

For example, write an adapter type just like io.nopCloser but use io.ReadSeeker rather than io.Reader, and use that adapter before going into net/http.

I did something similar to this in order to work around this issue.

https://golang.org/pkg/net/http/#Request.GetBody was added recently explicitly for request retries. You should use that instead.

I noticed this was recently added for the purposes of retrying, but was under the impression that it was used in the guts of net/http and not intended to be used by custom Tripper implementations. If we're happy to let custom Tripper implementations make use of this then I'm happy for this to be closed.

We won't be exporting weird internals.

I don't find this very helpful because you never state why it's weird. It satisfies an interface which is used throughout the standard library, the implementation is clear and obvious to any Go developer, and the intent of the struct is clear, I fail to see it as a weird internal IMHO.

@bradfitz
Copy link
Contributor

I don't find this very helpful because you never state why it's weird.

Because it's not the API we want people using. Anything we export becomes a mental burden for people reading the godoc, distracting them from the parts we want them to use.

@kegsay
Copy link
Author

kegsay commented Dec 14, 2017

I wholeheartedly agree that gratuitous exports can pollute the docs / API surface and incurs a mental cost on the developer. I don't feel that this applies in this scenario however.

If nopCloser were to be public, it would obviate the need for func NopCloser(r) as it would be possible to:

rc := ioutil.NopCloser{r}

But because the function form existed first, it'd need to be kept for backwards-compat. I concede that this may not be the API shape you wish to expose, as there now exists 2 ways of doing the same thing, and it's not clear if they are equivalent (if the function form does anything extra).

My concern is that this is taking precedence over adding the ability for developers to introspect their structs however they so wish to. This gets exacerbated by the use of this struct in net/http, making it more likely developers will encounter this now and in the future.

Either way, if a decision has been made (No: it's not the API we want people to use) then this issue can be closed. I don't particularly have my heart set on this as workarounds exist for my particular use case, but I suspect this will come up again in the future. Thanks for considering it anyway.

@mvdan
Copy link
Member

mvdan commented Dec 14, 2017

rc := ioutil.NopCloser{r}

Small point - note that then you'd have to use ioutil.NopCloser{Reader: r}, which is more verbose than the current ioutil.NopCloser(r). The reason is backwards compatibility (with imported packages), and there even is a vet warning about it:

https://golang.org/pkg/cmd/vet/#hdr-Unkeyed_composite_literals

@bradfitz
Copy link
Contributor

If we're happy to let custom Tripper implementations make use of this then I'm happy for this to be closed.

SGTM :)

@golang golang locked and limited conversation to collaborators Dec 14, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. Suggested Issues that may be good for new contributors looking for work to do.
Projects
None yet
Development

No branches or pull requests

6 participants