Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: client does not send cookies if an intermediary request hits a different domain #25465

Closed
aeneasr opened this issue May 19, 2018 · 18 comments
Labels
FrozenDueToAge help wanted NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@aeneasr
Copy link

aeneasr commented May 19, 2018

What version of Go are you using (go version)?

1.10.2

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Windows 10 and OSX

What did you do?

I have the following scenario:

  1. HTTP Client makes a request to http://domain-a/set
  2. HTTP Handler at http://domain-a/set sets a Cookie and redirects to http://domain-b/foo
  3. HTTP Handler at http://domain-b/foo does nothing except redirecting back to http://domain-a/check
  4. HTTP Header at http://domain-a/check is missing the cookie

The cookie is set if we do not redirect to http://domain-b but directly to http://domain-a:

  1. HTTP Client makes a request to http://domain-a/set
  2. HTTP Handler at http://domain-a/set sets a Cookie and redirects to http://domain-a/check
  3. HTTP Header at http://domain-a/check has the cookie set

A reproducible demo is available on gist.

I believe that this is caused by some logic that is tied to:

// • when forwarding sensitive headers like "Authorization",
// "WWW-Authenticate", and "Cookie" to untrusted targets.
// These headers will be ignored when following a redirect to a domain
// that is not a subdomain match or exact match of the initial domain.
// For example, a redirect from "foo.com" to either "foo.com" or "sub.foo.com"
// will forward the sensitive headers, but a redirect to "bar.com" will not.

My intuition tells me that the cookie header is deleted if the client encounters a domain that does not match the original domain, and does not recover the cookie in subsequent redirects if the domain is set back to the original one.

From what I can tell, this behaviour could be caused by shouldCopyHeaderOnRedirect which strips all the info and thus makes it unavailable in subsequent requests.

What did you expect to see?

I expected the cookie to be available at localhost:9000 (this is not about the intermediary domain localhost:8000), even if one of the redirects (in the middle) is a different domain. This is the default behavior of browsers too.

What did you see instead?

The cookie was not set.


edit://

While this is an edge case, it is kind of common in scenarios with federation where the browser does a lot of redirects between domains. That's actually how I found this issue.

@aeneasr
Copy link
Author

aeneasr commented May 19, 2018

After further investigation I noticed that this can be handled by overwriting the CheckRedirect function of the HTTP client in such a manner, that it uses the cookies from earlier requests (using the via argument of CheckRedirect). Of course, that's kinda hacky and error prone so I think this should be handled by the stdlib instead.

@agnivade agnivade changed the title HTTP client does not send cookies if an intermediary request hits a different domain net/http: client does not send cookies if an intermediary request hits a different domain May 20, 2018
@agnivade agnivade added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label May 20, 2018
@agnivade
Copy link
Contributor

@arekkas - As per the documentation, it seems to me that this is intended, and changing this is possibly going to break behavior.

/cc @bradfitz /@tombergan

@agnivade agnivade added this to the Go1.11 milestone May 20, 2018
@aeneasr
Copy link
Author

aeneasr commented May 20, 2018

We discussed this in the Gopher slack too. While it is (obviously) required for the client to not send any confidential data across domains, the client should be able to send confidential data to the domain that data belongs to, regardless of prior redirects.

Let's assume I have cookies for domain a and domain b. If I call domain a which redirects me to domain b, then domain a should get the proper credentials as should domain b. To me it appears to be nonsensical to strip cookies for domain b just because there was a prior redirect. You will not find a single browser that implements this behvaiour (stripping cookies due to redirects).

@aeneasr
Copy link
Author

aeneasr commented May 20, 2018

And it is possible to do that, the following code has serious security issues as it sends confidential data across domains and has unexpected behaviour too. However, I want to prove my point that the data is there and it can actually be implemented without a lot of refactoring (imo):

	resp, err := (&http.Client{
		Jar: c,
		// Hack to fix cookie across domains
		CheckRedirect: func(req *http.Request, via []*http.Request) error {
			if len(via) > 0 && req.Header.Get("cookie") == "" {
				req.Header.Set("Cookie", via[len(via)-1].Header.Get("Cookie"))
			}

			return nil
		},
	})

@aeneasr
Copy link
Author

aeneasr commented May 20, 2018

If it turns out to be not accepted due to concerns of being a breaking change, we could probably supply another DefaultClient or rather another CheckRedirect which implements this behavior.

@bradfitz bradfitz modified the milestones: Go1.11, Go1.12 May 20, 2018
@bradfitz bradfitz added the NeedsFix The path to resolution is known, but the work has not been done. label May 20, 2018
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label May 20, 2018
@bradfitz
Copy link
Contributor

Seems fixable (making sure we set/get cookies from the Jar per request across redirects), but not for Go 1.11.

@odeke-em
Copy link
Member

/cc @vdobler too

@vdobler
Copy link
Contributor

vdobler commented May 22, 2018

The gist provided to recreate the issue uses both localhost and
127.0.0.1 as the host.

For a cookiejar.Jar these are two different domains.
If I use either localhost or 127.0.0.1 consistently throughout the
whole code (as the problem description does) I cannot reproduce
the issue.

@arekkas Could you provide a working example of the issue
using only "localhost" ?

@aeneasr
Copy link
Author

aeneasr commented May 22, 2018

For a cookiejar.Jar these are two different domains.
If I use either localhost or 127.0.0.1 consistently throughout the
whole code (as the problem description does) I cannot reproduce
the issue.

127.0.0.1:80 is different to localhost:80 as localhost:8000 is different to localhost:9000. All of those are considered different domains as they run on different ports and/or hostnames, so the provided example illustrates the issue described in the opening post as intended. I agree though that it's a bit confusing and I should have used consistent naming in the issue description and the example.

@vdobler
Copy link
Contributor

vdobler commented May 22, 2018

@arekkas

127.0.0.1:80 is different to localhost:80 as localhost:8000 is different to localhost:9000. All of those are considered different domains as they run on different ports and/or hostnames

No they are not. See RFC 6265 section 1 third paragraph:
https://tools.ietf.org/html/rfc6265#section-1 :

Similarly, cookies for a given host are shared across all the ports on that host, even though the usual "same-origin policy" used by web browsers isolates content retrieved via different ports.

But localhost and 127.0.0.1 are considered different domains.
(See https://tools.ietf.org/html/rfc6265#section-5.1.3,
https://tools.ietf.org/html/rfc6265#section-5.3 no. 6 and
https://tools.ietf.org/html/rfc6265#section-5.4).

Can you provide sample code which shows the claimed issue?

@aeneasr
Copy link
Author

aeneasr commented May 22, 2018

No they are not. See RFC 6265 section 1 third paragraph:

I didn't know that! It doesn't affect the issue though and the issue in Go actually affects different ports as well.

Can you provide sample code which shows the claimed issue?

I updated the opening post to better reflect the issue. The code represents the issue as intended.

@aeneasr
Copy link
Author

aeneasr commented May 22, 2018

By the way, I'm also pretty sure that the following will happen as well:

  1. Initialize a cookie jar with cookies for http://domain-a/
  2. Make a HTTP request to http://domain-b/hello
  3. HTTP handler at http://domain-b/hello redirects to http://domain-a/foo
  4. HTTP handler at http://domain-a/foo misses cookies set for http://domain-a/ although there is no reason why they should be missing.

I will have to confirm this with code, though.

This works as expected, see code example at https://play.golang.org/p/p7io33sGrOa

@vdobler
Copy link
Contributor

vdobler commented May 22, 2018

@arekkas

I think everything expected to work actually does so.
All this is totally unrelated to ports.
You can test it with domain-a == "localhost" and domain-b == "127.0.0.1".
Feel free to use whatever ports you like but make sure that you do not
expect cookies for localhost to appear on 127.0.0.1

@aeneasr
Copy link
Author

aeneasr commented May 22, 2018

Sorry for the post spam.

I didn't know that! It doesn't affect the issue though and the issue in Go actually affects different ports as well.

I'm rolling back this statement, the issue only affects if the domains are different - not the ports. Sorry for the confusion here!

@aeneasr
Copy link
Author

aeneasr commented May 22, 2018

Feel free to use whatever ports you like but make sure that you do not
expect cookies for localhost to appear on 127.0.0.1

I think there is a misunderstanding here. I'll try to be as concise as possible in the description.

I have a cookie jar that has cookies for domain-a. I expect that these cookies are always sent to domain-a when making a request to domain-a, regardless of any prior redirections.

What happens instead is that, if I have a cookie jar with cookies for domain-a, the cookies will be stripped if - somewhere inbetween - a request to another domain (e.g. domain-b) was made.

From the opening post:

  1. HTTP Client makes a request to http://domain-a/set. (Here, a cookie for domain-a will be set in the cookiejar).
  2. HTTP Handler at http://domain-a/set sets a Cookie and redirects to http://domain-b/foo
  3. HTTP Request is sent to http://domain-b/foo. No cookies are included. This is expected and required and ok.
  4. HTTP Handler at http://domain-b/foo does nothing except redirecting back to http://domain-a/check
  5. HTTP Header at http://domain-a/check is missing the the cookie from the cookie jar for domain-a. (Here, we'd expect the cookies for domain-a to be included because the prior redirection should not affect whether or not legitimate cookies are sent to domain-a or not. The proper cookies should always be included in requests to domain-a if the cookies are in fact set for domain-a)

I hope this clarifies the confusion.

@vdobler
Copy link
Contributor

vdobler commented May 22, 2018

@arekkas

I do understand what you are trying so say and I do understand the scenario
you are describing. There is no confusion here.

You claim that in step 5 the cookies for domain-a are missing and this claim
is factual wrong. I checked. I run your scenario in code and I cannot
reproduce your claim that there is a bug in step 5 simply because
my code does send the appropriate cookie.

Now it can happen that I made a mistake and that is why I ask you to
provide sample code (not a description, the description is
absolutely clear) which demonstrates that your claim (no cookies in
step 5) is true.

The gist you provided is not a suitable evidence for your claim because
in the gist you set the cookies for 127.0.0.1 but redirect to localhost!
And the problem description is very good and clear (it is just wrong).

Please try it out. Everything works exactly like you think it should, including
the proper cookies being sent to domain-a after redirect in step 5.
There is no need to argue. Try it out and show the code. Either you are
wrong or I am wrong. Only an experiment will tell.

@aeneasr
Copy link
Author

aeneasr commented May 22, 2018

You are right, please excuse that I didn't properly check the messy code I wrote. I refactored, cleaned and used stdlib only in a new example (here) and it works as expected (the cookie is being set). I tried it in different configurations (one mux on the same port but different domains, two muxs on different ports on different domains) without luck (wrt reproducing the initial issue).

I'll close this for now and will come back to it if I can figure out what initially caused this mess. Please accept my apologies for spamming your (probably already very busy) inboxes with a non-issue and thank you all for the time put into trying to resolve this.

@aeneasr aeneasr closed this as completed May 22, 2018
@vdobler
Copy link
Contributor

vdobler commented May 22, 2018

@arekkas No need to apologize, this happens :-) I'm glad it is working now.

@golang golang locked and limited conversation to collaborators May 22, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge help wanted NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

6 participants