Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: net/http: thoughts for Go 2.0 #5465

Open
bradfitz opened this issue May 14, 2013 · 43 comments
Open

proposal: net/http: thoughts for Go 2.0 #5465

bradfitz opened this issue May 14, 2013 · 43 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. v2 A language change or incompatible library change
Milestone

Comments

@bradfitz
Copy link
Contributor

Wishlist bug for what net/http would look like if we could make backwards-incompatible
changes.

Anything goes.

I'll start:

* Handler (http://golang.org/pkg/net/http/#Handler) currently takes a pointer to a 208
byte Request struct.  

* The Request struct (http://golang.org/pkg/net/http/#Request) has all its fields
publicly exposed, most of which require memory allocation:

  -- Method
  -- URL (itself 104 bytes, with a bunch of strings requiring allocation)
  -- Header map + slices + strings (even if never accessed)
  -- TransferEncoding slice (even if not accessed)
  -- Host string (even if not accessed)
  -- RemoteAddr string (even if not accessed)
  -- RequestURI (even if not accessed)
  -- TLS state struct (even if not accessed)

For a lightweight handler that doesn't touch anything (say, serves some static content
from memory), this means we can't do any better than generating ~1KB of garbage per
request.

I'd prefer to make a ServerRequest interface with accessor methods which can generate
needed data on demand.

This would also simplify our docs on our doubly-abused-in-different-ways *Request, which
contains documentation gems like:

    // PostForm contains the parsed form data from POST or PUT
    // body parameters.
    // This field is only available after ParseForm is called.
    // The HTTP client ignores PostForm and uses Body instead.
    PostForm url.Values

If we had byte views (issue #5376), I would also say that most fields in the
ServerRequest, URL, Header, and FormValues are all byte views with validity scoped to
the duration of the http.Handler call, instead of strings.
@nigeltao
Copy link
Contributor

nigeltao commented Jun 4, 2013

Comment 1:

A CookieJar's methods should return error, and maybe be renamed from http.CookieJar to
cookiejar.Interface.

@gopherbot
Copy link

Comment 2 by robryk:

In http.Client, there should be a uniform way of specifying request headers on the
initial request and subsequent requests when following a redirect. Current the
Request.Header field is useless if the request might get redirected. See issue #5782.
Maybe something like a func(*Request) field in the request, that gets called on this
request and each redirected one before it gets sent?

@gopherbot
Copy link

Comment 3 by jeff.allen:

The fact that headers are parsed into a map is a mega bummer for garbage reduction. If
the main way headers were exposed was more opaque, the headers map could be made up of
offsets into the buffer that came straight from Read.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 4:

Labels changed: added repo-main.

@nigeltao
Copy link
Contributor

nigeltao commented Jan 9, 2014

Comment 5:

Related to comment #1, net/http/cookiejar.PublicSuffixList's PublicSuffix method should
perhaps return (string, error) instead of (string), in case the PSL implementation is
passed non-canonicalized input such as upper-cased "EXAMPLE.COM.AU". Returning an
explicit error is probably safer than silently returning a "" that would mean a liberal
cookie policy.

@alberts
Copy link
Contributor

alberts commented Feb 15, 2014

Comment 6:

It would be nice to have a handle on when all the goroutines that handle requests have
terminated after closing the listener (thereby breaking out of the accept loop) for
cases where they are using a shared resource that needs to be cleaned up (e.g. an
in-process database that needs to be closed). And maybe a way to force-close all their
connections. Especially useful for tests and SIGTERM situations.

@gopherbot
Copy link

Comment 7 by robryk:

You can do that by counting open connections returned by Listener's
Accept: http://play.golang.org/p/sPmvXaR-2M
(Disclaimer: I didn't test that code.)
Robert

@alberts
Copy link
Contributor

alberts commented Feb 15, 2014

Comment 8:

@Robert thanks, I did something similar in the end.

@alberts
Copy link
Contributor

alberts commented Feb 15, 2014

Comment 9:

@Robert For what it's worth, I think your listener breaks in the more general case where
you hand off the socket to both a reader and a writer goroutine. There a "done" channel
might be better.

@gopherbot
Copy link

Comment 10 by robryk:

Summarizing a discussion with fullung@: That version didn't handle multiple calls to
Close and had a (likely irrelevant) ReadWrite/Close race condition. The corrected
version is here: http://play.golang.org/p/WTKa021ipJ

@gopherbot
Copy link

Comment 11 by robryk:

Summarizing a discussion with fullung@: That version didn't handle multiple calls to
Close and had a (likely irrelevant) ReadWrite/Close race condition. The corrected
version is here: http://play.golang.org/p/WTKa021ipJ

@rsc
Copy link
Contributor

rsc commented Mar 3, 2014

Comment 12:

Labels changed: added release-none.

@gopherbot
Copy link

Comment 13 by robryk:

We have http.StripPrefix which modifies Request.URL.Path and passes the modified request
to a handler. We also have http.Redirect, which assumes that Request.URL.Path is
absolute.
Thus, if one tries to use http.Redirect within a handler that is wrapped in
http.StripPrefix the redirect will point to a wrong URL. In fact, we don't use
http.Redirect in http.FileServer for this very reason:
https://code.google.com/p/go/source/browse/src/pkg/net/http/fs.go#339
We could abandon StripPrefix and have another way of passing the prefix to handlers. We
could also keep the original path and the possibly-shortened path in request, so that
redirect could use the original path.

@gopherbot
Copy link

Comment 14 by nigel.tao.gnome:

Make http.Server recovering from panics opt-in instead of opt-out.
https://groups.google.com/forum/#!msg/golang-dev/rXs4TG1gdXw/7BQ29S4NPrgJ

@bradfitz bradfitz added accepted Performance GarbageCollector v2 A language change or incompatible library change labels Aug 27, 2014
@bradfitz bradfitz self-assigned this Aug 27, 2014
@bradfitz bradfitz removed their assignment Dec 13, 2014
@rsc rsc added this to the Unplanned milestone Apr 10, 2015
@tv42
Copy link

tv42 commented Aug 30, 2015

I wish there was a way to delegate URL path subtrees to another handler in a way where recipient doesn't have to specifically understand the whole path, but can still easily do redirects and such. A convention for "this is the unprocessed part of the path".

For example, try using the same handler for /foo/ and /bar/quux/, having the handler extract the immediate child segment as a numeric ID, doing a database lookup, and creating a redirect to a canonical non-numerically named child (e.g. http://macworld.com/article/2367748 )

Right now, mutating Request.URL.Path (like StripPrefix) destroys knowledge of the requested URL (and re-parsing Request.RequestURI seems wrong).

@CAFxX
Copy link
Contributor

CAFxX commented Dec 15, 2015

Only partially related (it belongs more in the scheduler/netpoller) but since net/http has ListenAndServe* I'll add it here: integrate support for REUSEPORT in the netpoller (i.e. open multiple listening sockets on the same port -one for proc?- and accept() on multiple procs)

@nhooyr
Copy link
Contributor

nhooyr commented Aug 12, 2018

It'd be neat if we could just use *httputil.ReverseProxy and support WebSocket connection upgrades transparently.

@bradfitz
Copy link
Contributor Author

@nhooyr, there's no reason that needs to wait for Go 2. I'll do it for Go 1.12. I filed #26937.

@nhooyr
Copy link
Contributor

nhooyr commented Aug 12, 2018

Awesome! I figured we'd need an API change to do this correctly as we have to hijack the client connection. Curious to see how you managed to do it.

@nhooyr
Copy link
Contributor

nhooyr commented Aug 12, 2018

Nvm, I see now. We just don't use the transport.

@jasonmf
Copy link

jasonmf commented Aug 20, 2018

The http/Server.ErrorLog is *log.Logger.

If you want to log server errors in a custom way you have to create a custom io.Writer that assumes each call to Write() is a single, complete log message or you have to buffer, parse, and restructure a stream of calls to Write().

It would be much more flexible to have ErrorLog be an interface such as:

type ErrorLog interface {
    Printf(string, ...interface{})
}

@bradfitz
Copy link
Contributor Author

@AgentZombie, there's another bug open about rethinking log/log interfaces. Whatever we did across std would also be done in net/http.

@mklencke
Copy link

How about explicit context support?

ServeHTTP(context.Context, ResponseWriter, *Request)

and then get rid of (*Request).Context(). That way, middleware can also more easily augment the context without suffering #28737.

@nhooyr
Copy link
Contributor

nhooyr commented Mar 16, 2019

Would be nice to rename http.Header to http.Headers as it is a map of headers and not a single header.

@RalphCorderoy
Copy link

Hi @nhooyr, but Header["foo"] gives the single Foo header so it reads okay. I agree another style is to use a plural for an array or map so it really depends on the normal style within http and stdlib as a whole.

@nhooyr
Copy link
Contributor

nhooyr commented Mar 17, 2019

It’s not the worst thing but most usages of maps and array are plural in the stdlib.

@shinny-chengzhi
Copy link

The auto recovering of http.Server didn't generate core dump even if GOTRACEBACK=crash, which make it hard to debug. All the information I get is the call stack which is not enough as I need to examine the value of various variable.

@guyskk
Copy link

guyskk commented Dec 13, 2021

Consider http client DO NOT following redirects by default. FYI: encode/httpx#1785

@bcmills
Copy link
Contributor

bcmills commented May 16, 2022

One more thought via #52727. Especially now that we have errors.Is, a new net/http client should produce errors with more structure.

For example, on a Content-Length mismatch the current client returns a bare io.ErrUnexpectedEOF, rather than (say) a more structured error in the style produced by the os package: perhaps one that includes the URL and/or the expected Content-Length.

@jchadwick-buf
Copy link

One thought I have: it seems like it's somewhat common to forget to implement http.Flusher when wrapping an http.ResponseWriter, which can cause some really nasty issues. Would it perhaps be better if the Flush method was simply on http.ResponseWriter, and canonically a no-op if the underlying implementation is unbuffered?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. v2 A language change or incompatible library change
Projects
None yet
Development

No branches or pull requests