Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http/httputil: make it easier for ReverseProxy.ErrorHandler to preserve the request body #33726

Closed
fovecifer opened this issue Aug 20, 2019 · 3 comments
Labels
FeatureRequest FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@fovecifer
Copy link

fovecifer commented Aug 20, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.7 darwin/amd64

We could use ErrorHandler func(http.ResponseWriter, *http.Request, error) to handle errors when backend timeout, but one important fact is the Body io.ReadCloser
in http.Request of ErrorHandler func is no "readable", because the interface io.ReadClose
just like a stream, you could not read twice from it.
So when I add some "retry function" in the ErrorHandler, the Body always empty, we could
not retry this request to another health backend, to let my project work I do something like
this in func (p *ReverseProxy) ServeHTTP(rw http.ResponseWriter, req *http.Request):

	if outreq.Body != nil && outreq.ContentLength != 0 {
		tee := io.TeeReader(outreq.Body, &buf)
		outreq.Body = ioutil.NopCloser(tee)
	}
	res, err := transport.RoundTrip(outreq)
	if err != nil {
		if outreq.Body != nil && outreq.ContentLength != 0 {
			outreq.Body = ioutil.NopCloser(&buf)
		}
		p.getErrorHandler()(rw, outreq, err)
		return
	}

Maybe we have 2 options:

  1. Add Comments to ErrorHandler like this:
        // ErrorHandler is an optional function that handles errors
	// reaching the backend or errors from ModifyResponse.
        // *the Body of http.Request in this function maybe unreadable*
        // *because it readed by RoundTrip function*
	//
	// If nil, the default is to log the provided error and return
	// a 502 Status Bad Gateway response.
	ErrorHandler func(http.ResponseWriter, *http.Request, error)
  1. Add a option to ReverseProxy, let the call to choose storing the body into buffer or not
@bcmills bcmills changed the title net/http/httputil: ReverseProxy.ErrorHandler enhancement net/http/httputil: make it easier for ReverseProxy.ErrorHandler to read the request body Aug 20, 2019
@bcmills bcmills added FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Aug 20, 2019
@bcmills bcmills added this to the Unplanned milestone Aug 20, 2019
@bcmills
Copy link
Contributor

bcmills commented Aug 20, 2019

Add a option to ReverseProxy, let the call to choose storing the body into buffer or not

As far as I can tell, you can already use the Director field to inject the io.TeeReader. The difficult part is how to pass the resulting buffer to the ErrorHandler function.

On the other hand, if you're assigning to outreq.Body anyway, then you could provide your own implementation than injects your own custom struct type, and use a type-switch to unpack that type within the ErrorHandler function. So it's not obvious to me that anything is needed here beyond the existing Director function.

@bcmills
Copy link
Contributor

bcmills commented Aug 20, 2019

CC @bradfitz (on leave) for httputil.

@bcmills bcmills changed the title net/http/httputil: make it easier for ReverseProxy.ErrorHandler to read the request body net/http/httputil: make it easier for ReverseProxy.ErrorHandler to preserve the request body Aug 20, 2019
@fovecifer
Copy link
Author

Add a option to ReverseProxy, let the call to choose storing the body into buffer or not

As far as I can tell, you can already use the Director field to inject the io.TeeReader. The difficult part is how to pass the resulting buffer to the ErrorHandler function.

Thank you!

Your way is much better, I should create a new "init" function like this:

func NewSingleHostBodyBufReverseProxy(target *url.URL, key string) *httputil.ReverseProxy {
	director := func(req *http.Request) {
		httputil.NewSingleHostReverseProxy(target).Director(req)
		if req.Body != nil && req.ContentLength != 0 {
			var buf bytes.Buffer
			tee := io.TeeReader(req.Body, &buf)
			req.Body = ioutil.NopCloser(tee)
			ctx := context.WithValue(req.Context(), key, &buf)
			r2 := req.WithContext(ctx)
			*req = *r2
		}
	}
	return &httputil.ReverseProxy{Director: director}
}

I inject the "buf" into context of request with the "key", so in my ErrorHandler, it could be

if buf, ok := r.Context().Value("bodybuf").(*bytes.Buffer); ok {
	r.Body = ioutil.NopCloser(buf)
}

the existing Director and ReverseProxy are good enough

@golang golang locked and limited conversation to collaborators Aug 20, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FeatureRequest FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

3 participants