Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http/cookiejar: implement basic in-memory Jar #1960

Closed
gopherbot opened this issue Jun 15, 2011 · 23 comments
Closed

net/http/cookiejar: implement basic in-memory Jar #1960

gopherbot opened this issue Jun 15, 2011 · 23 comments
Milestone

Comments

@gopherbot
Copy link

by czapkofan:

What steps will reproduce the problem?
The default handler of redirects in http.Client.Do() doesn't pass the contents of the
Cookie table to subsequent pages (pages it's redirected to). I've also found it fairly
unintuitive to work around this behaviour without rewriting the whole Client (the
solution I found was to pass a redirect-intercepting callback returning a special error
value).
(Sorry, I don't have time to carve "A Minimal Code Example".)

Which compiler are you using (5g, 6g, 8g, gccgo)?
8g

Which operating system are you using?
$ uname -srvmpio
Linux 2.6.18-92.1.18.el5 #1 SMP Wed Nov 12 09:30:27 EST 2008 i686 i686 i386 GNU/Linux

Which revision are you using?  (hg identify)
8818ac606e92 tip
@rsc
Copy link
Contributor

rsc commented Jun 15, 2011

Comment 1:

Owner changed to @bradfitz.

Status changed to Accepted.

@bradfitz
Copy link
Contributor

Comment 2:

The existing TODO in the code:
// TODO: if/when we add cookie support, the redirected request shouldn't
// necessarily supply the same cookies as the original.
... because Cookies should only be sent if their host + path match, which we don't
necessarily know.  We can see if the host changes, but we don't know the path
restrictions on the original cookies.
So really we need a cookie jar interface on the Client object, which are notoriously
complicated and security-bug-prone.
See: http://lcamtuf.blogspot.com/2010/10/http-cookies-or-how-not-to-design.html
I suppose we could punt on this and only offer the minimum interface in the standard
http package:  just the capability to add Cookies to a request before it's sent out.
That'd probably unblock you, right?

@rsc
Copy link
Contributor

rsc commented Jun 15, 2011

Comment 3:

I think the original poster is not blocked already.
But we should probably do a cookie jar, once, right.

@bradfitz
Copy link
Contributor

Comment 4:

Labels changed: added pkg-http.

@gopherbot
Copy link
Author

Comment 5 by TylerEgeto:

I'm not sure if its exactly the same issue, but it definitely looks like its related,
I'm not getting cookies passed back using the ReverseProxy. This is not blocking
anything on my end either, but I came across this ticket and figured I would make a note
so its on the record.
Cookies go from the browser through the proxy and reach the end server. The end server
sends back a cookie but it gets lost. The response of transport.RoundTrip in
ReverseProxy's ServeHTTP function never receives it.

@bradfitz
Copy link
Contributor

Comment 6:

Tyler, that bug was fixed awhile back. What version are you using? Unless I broke it
again, but I added tests:
http://code.google.com/p/go/source/detail?r=9fba246966e7

@gopherbot
Copy link
Author

Comment 7 by TylerEgeto:

Brad, you are correct, it is working correctly. The fix you linked to is not present in
the source linked to on the main site (here: http://code.google.com/p/go/source/browse/)
but it is in the hg pull. I was using the linked src as a starting point for an
alternate reverse proxy. I didn't realize they were different. Sorry to waste your time
on this.

@rsc
Copy link
Contributor

rsc commented Dec 9, 2011

Comment 8:

Labels changed: added priority-later.

@rsc
Copy link
Contributor

rsc commented Dec 12, 2011

Comment 9:

Labels changed: added priority-go1.

@bradfitz
Copy link
Contributor

Comment 10:

For Go 1, I'd be at least happy to have this in:
http://golang.org/cl/5399043/
I think it's ready for submission but I'd like Russ to have a look first.
Then the community can do their own cookie jar implementations with the standard http
Client.
If we get dr.volker.dobler's actual implementation of the Jar interface in too, that'd
be even better, but I want our cookie security experts on the Chrome team to review it
first.

@bradfitz
Copy link
Contributor

Comment 11:

That CL is now in as http://code.google.com/p/go/source/detail?r=06e9bc8641 , so I'm
reasonably happy closing this bug.
We can also keep it open, though, and see if we can get an actual implementation of
CookieJar in before Go 1 too.

@vdobler
Copy link
Contributor

vdobler commented Jan 10, 2012

Comment 12:

I am using some spare time for a CookieJar implementation
and got stuck with three issues:
A) Are my own test cases correct? 
I am currently considering using the test from either
http://src.chromium.org/viewvc/chrome/trunk/src/net/base/cookie_monster_unittest.cc
or 
https://github.com/abarth/http-state/tree/master/tests
Any suggestions?
   
B) To what pattern of use should the underlying data structures be
optimized?  Taking to 5 different urls and dealing with 3 session
cookies could be implemented much faster than a general purpose
storage capable of storing and retrieving 50 cookies for 4000 
domains.
Currently I think a general purpose CookieJar should allow lots
of cookies and limit them in number to prevent a DoS and try to
achieve almost constant retrieval time in Cookies().
C) Should the default implementation of CookieJar export functions
to sanitize/judge/set-up recieved cookies so that different
implementations could use these?

@rsc
Copy link
Contributor

rsc commented Jan 10, 2012

Comment 13:

(Sorry, code hosting drops quoted text.)
(A) I would use both.
(B) The data structures should be as simple as possible.
Start with the low-load case, get it working right with a
good test suite, and then worry about crazy scaling in
a different CL.
(C) Not at first.

@vdobler
Copy link
Contributor

vdobler commented Jan 12, 2012

Comment 14:

Thanks for the answer.
I came up with more issues, sorry...
D) Currently CookieJar is used by net/http only.
Should a CookieJar implementation allow non-http
request, both in SetCookies() and Cookies()?
Or should this behaviour be switchable?
Or rephrased: Does Go think cookies non-http URLs 
are a good idea? (RFC 6265 allows it, I'm not too
fond of the idea).
E) Where should a CookieJar implementation reside?
Inside "net/http"? As a subpackage "net/http/jar"?
Below net directly "net/jar"?  Under exp/jar to
experiment with it? And how should it be
named (Default[Cookie]Jar? Http[Cookie]Jar)?  That
depends a bit on answer to D)
F) Currently the cookie handling in package http
does not process the Domain and Path attribute of
a cookie. See
http://tip.goneat.org/src/pkg/net/http/cookie.go#L95
http://tip.goneat.org/src/pkg/net/http/cookie.go#L122
Should I prepare a CL for these two TODOs first?
G) How to handle the public suffices/effective TLDs
whatever you call them:  Domains like co.uk where you
should not be able to set a domain cookie as not all
subdomains are under one control only?
(See http://publicsuffix.org)
 - Ignore public suffixes completely?
 - Read them once during the build of Go and use 
   that fixed list
 - Read that list during init() and fallback to
   fixed list if e.g. no network connection?

@bradfitz
Copy link
Contributor

Comment 15:

(D) I only care about HTTP cookies. What other type are there in reality? None, right?
(E) net/http works for me, considering (D).  But see (G), below.
(F) They're saved. I'm not sure what those TODOs mean.  Validate that they're sane
formats?  Sure, you could do that separately first if you'd like.
(G) I don't want all Go binaries using HTTP to have a huge public suffix list compiled
in, but it should be available for people who do want to use cookies, and probably by
default.  That perhaps argues that the cookie jar implementation should live in its own
package, so only users who care about cookies get the public suffix list linked in.   I
don't think fetching it from the net at runtime is a good option.  That wouldn't work
for many corp/production firewalls anyway.
Let's go with exp/.../jar for now:
package jar
type Jar struct {
    // ... optional future limits / interfaces for storage?
}
func New() *Jar {
   ...
}
Russ might have opinions on all this too, though.

@bradfitz
Copy link
Contributor

Comment 16:

Talked with Russ.  Decision:
net/http/cookiejar
package cookiejar
type Jar ...

@robpike
Copy link
Contributor

robpike commented Jan 13, 2012

Comment 17:

Owner changed to builder@golang.org.

@rsc
Copy link
Contributor

rsc commented Jan 25, 2012

Comment 19:

Volker Dobler is working on this.  codereview.appspot.com/5544082.
Postponing until after Go 1, to give us more time to consider the API.

Labels changed: added priority-later, removed pkg-http, priority-go1.

@rsc
Copy link
Contributor

rsc commented Sep 12, 2012

Comment 20:

Labels changed: added go1.1.

@rsc
Copy link
Contributor

rsc commented Dec 10, 2012

Comment 21:

Labels changed: added size-xl.

@nigeltao
Copy link
Contributor

nigeltao commented Mar 1, 2013

Comment 22:

This issue was closed by revision f29b0cf.

Status changed to Fixed.

@dborowitz
Copy link
Contributor

Comment 23:

Am I correct in understanding this implementation does not include either a default
public suffix list, or support for parsing a publicly-available list like at
publicsuffix.org? IMHO this is not very useful until such support exists. (I may offer
to put my money where my mouth is and write it myself, though not sure exactly when :)

@bradfitz
Copy link
Contributor

bradfitz commented Mar 1, 2013

Comment 24:

Dave, that's split off into http://godoc.org/code.google.com/p/go.net/publicsuffix

@rsc rsc added this to the Go1.1 milestone Apr 14, 2015
@rsc rsc removed the go1.1 label Apr 14, 2015
@golang golang locked and limited conversation to collaborators Jun 24, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

7 participants