Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net/http: cookies not set on redirections (302) by the default client #64464

Open
Amodio opened this issue Nov 30, 2023 · 4 comments
Open

net/http: cookies not set on redirections (302) by the default client #64464

Amodio opened this issue Nov 30, 2023 · 4 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@Amodio
Copy link

Amodio commented Nov 30, 2023 •
edited

Go version

go version go1.19.8 linux/amd64

What operating system and processor architecture are you using (go env)?

GOVERSION="go1.19.8"
GOHOSTARCH="amd64"
GOHOSTOS="linux"

What did you do?

First case, I did a GET onto a webpage that sets a cookie and redirects users: https://go.dev/play/p/ecskkzjvHRy
Second case, I did a POST onto a webpage that sets a cookie and redirects users: https://go.dev/play/p/a4xWVv0ZVqT

What did you expect to see?

In both cases, I expected a redirection to a webpage with cookies set.
It turns out that a Cookie Jar has to be defined/initialized for this behaviour to happen (see here for example).
Corrected code for each case: GET & POST.
Output in the first case:

$ go run main.go
Response: Cookie was: SET<br><a href="?a=1701347614.2247">try again</a>
Cookie: test=deleted; Expires=Thu, 01 Jan 1970 00:00:01 GMT; Max-Age=0

In the second case, I got a Wrong Credentials error.

What did you see instead?

First case:

$ go run main.go
Response: Cookie was: NOT SET<br><a href="?a=1701347518.7983">try again</a>
Cookie: test=deleted; Expires=Thu, 01 Jan 1970 00:00:01 GMT; Max-Age=0

Second case detailed log (censored):

$ GODEBUG=http2debug=2 go run main.go
2023/11/30 12:28:37 http2: Transport failed to get client conn for tictoc2.chall.malicecyber.com:443: http2: no cached connection was available
2023/11/30 12:28:37 http2: Transport creating client conn 0xc00010c180 to 46.30.202.223:443
2023/11/30 12:28:37 http2: Framer 0xc000068e00: wrote SETTINGS len=18, settings: ENABLE_PUSH=0, INITIAL_WINDOW_SIZE=4194304, MAX_HEADER_LIST_SIZE=10485760
2023/11/30 12:28:37 http2: Framer 0xc000068e00: wrote WINDOW_UPDATE len=4 (conn) incr=1073741824
2023/11/30 12:28:37 http2: Transport encoding header ":authority" = "tictoc2.chall.malicecyber.com"
2023/11/30 12:28:37 http2: Transport encoding header ":method" = "POST"
2023/11/30 12:28:37 http2: Transport encoding header ":path" = "/login.php"
2023/11/30 12:28:37 http2: Transport encoding header ":scheme" = "https"
2023/11/30 12:28:37 http2: Transport encoding header "content-type" = "application/x-www-form-urlencoded"
2023/11/30 12:28:37 http2: Transport encoding header "content-length" = "81"
2023/11/30 12:28:37 http2: Transport encoding header "accept-encoding" = "gzip"
2023/11/30 12:28:37 http2: Transport encoding header "user-agent" = "Go-http-client/2.0"
2023/11/30 12:28:37 http2: Framer 0xc000068e00: wrote HEADERS flags=END_HEADERS stream=1 len=84
2023/11/30 12:28:37 http2: Framer 0xc000068e00: wrote DATA flags=END_STREAM stream=1 len=81 data="password=X&username=admin"
2023/11/30 12:28:37 http2: Framer 0xc000068e00: read SETTINGS len=24, settings: MAX_FRAME_SIZE=1048576, MAX_CONCURRENT_STREAMS=250, MAX_HEADER_LIST_SIZE=1048896, INITIAL_WINDOW_SIZE=1048576
2023/11/30 12:28:37 http2: Transport received SETTINGS len=24, settings: MAX_FRAME_SIZE=1048576, MAX_CONCURRENT_STREAMS=250, MAX_HEADER_LIST_SIZE=1048896, INITIAL_WINDOW_SIZE=1048576
2023/11/30 12:28:37 http2: Framer 0xc000068e00: wrote SETTINGS flags=ACK len=0
2023/11/30 12:28:37 http2: Framer 0xc000068e00: read WINDOW_UPDATE len=4 (conn) incr=983041
2023/11/30 12:28:37 http2: Transport received WINDOW_UPDATE len=4 (conn) incr=983041
2023/11/30 12:28:37 http2: Framer 0xc000068e00: read SETTINGS flags=ACK len=0
2023/11/30 12:28:37 http2: Transport received SETTINGS flags=ACK len=0
2023/11/30 12:28:37 http2: Framer 0xc000068e00: read WINDOW_UPDATE len=4 (conn) incr=81
2023/11/30 12:28:37 http2: Transport received WINDOW_UPDATE len=4 (conn) incr=81
2023/11/30 12:28:48 http2: Framer 0xc000068e00: read HEADERS flags=END_STREAM|END_HEADERS stream=1 len=205
2023/11/30 12:28:48 http2: decoded hpack field header field ":status" = "302"
2023/11/30 12:28:48 http2: decoded hpack field header field "cache-control" = "no-store, no-cache, must-revalidate"
2023/11/30 12:28:48 http2: decoded hpack field header field "content-type" = "text/html; charset=UTF-8"
2023/11/30 12:28:48 http2: decoded hpack field header field "date" = "Thu, 30 Nov 2023 11:28:37 GMT"
2023/11/30 12:28:48 http2: decoded hpack field header field "expires" = "Thu, 19 Nov 1981 08:52:00 GMT"
2023/11/30 12:28:48 http2: decoded hpack field header field "location" = "/account.php"
2023/11/30 12:28:48 http2: decoded hpack field header field "pragma" = "no-cache"
2023/11/30 12:28:48 http2: decoded hpack field header field "server" = "Apache/2.4.56 (Debian)"
2023/11/30 12:28:48 http2: decoded hpack field header field "set-cookie" = "PHPSESSID=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX; path=/"
2023/11/30 12:28:48 http2: decoded hpack field header field "x-powered-by" = "PHP/8.2.6"
2023/11/30 12:28:48 http2: decoded hpack field header field "content-length" = "0"
2023/11/30 12:28:48 http2: Transport received HEADERS flags=END_STREAM|END_HEADERS stream=1 len=205
2023/11/30 12:28:48 http2: Transport encoding header ":authority" = "tictoc2.chall.malicecyber.com"
2023/11/30 12:28:48 http2: Transport encoding header ":method" = "GET"
2023/11/30 12:28:48 http2: Transport encoding header ":path" = "/account.php"
2023/11/30 12:28:48 http2: Transport encoding header ":scheme" = "https"
2023/11/30 12:28:48 http2: Transport encoding header "referer" = "https://tictoc2.chall.malicecyber.com/login.php"
2023/11/30 12:28:48 http2: Transport encoding header "content-type" = "application/x-www-form-urlencoded"
2023/11/30 12:28:48 http2: Transport encoding header "accept-encoding" = "gzip"
2023/11/30 12:28:48 http2: Transport encoding header "user-agent" = "Go-http-client/2.0"
2023/11/30 12:28:48 http2: Framer 0xc000068e00: wrote HEADERS flags=END_STREAM|END_HEADERS stream=3 len=52
2023/11/30 12:28:49 http2: Framer 0xc000068e00: read HEADERS flags=END_STREAM|END_HEADERS stream=3 len=98
2023/11/30 12:28:49 http2: decoded hpack field header field ":status" = "302"
2023/11/30 12:28:49 http2: decoded hpack field header field "cache-control" = "no-store, no-cache, must-revalidate"
2023/11/30 12:28:49 http2: decoded hpack field header field "content-type" = "text/html; charset=UTF-8"
2023/11/30 12:28:49 http2: decoded hpack field header field "date" = "Thu, 30 Nov 2023 11:28:48 GMT"
2023/11/30 12:28:49 http2: decoded hpack field header field "expires" = "Thu, 19 Nov 1981 08:52:00 GMT"
2023/11/30 12:28:49 http2: decoded hpack field header field "location" = "index.php?error=wrong_credentials"
2023/11/30 12:28:49 http2: decoded hpack field header field "pragma" = "no-cache"
2023/11/30 12:28:49 http2: decoded hpack field header field "server" = "Apache/2.4.56 (Debian)"
2023/11/30 12:28:49 http2: decoded hpack field header field "set-cookie" = "PHPSESSID=68879433fa243e2e9cfae6f7a8fed841; path=/"
2023/11/30 12:28:49 http2: decoded hpack field header field "x-powered-by" = "PHP/8.2.6"
2023/11/30 12:28:49 http2: decoded hpack field header field "content-length" = "0"
2023/11/30 12:28:49 http2: Transport received HEADERS flags=END_STREAM|END_HEADERS stream=3 len=98
2023/11/30 12:28:49 http2: Transport encoding header ":authority" = "tictoc2.chall.malicecyber.com"
2023/11/30 12:28:49 http2: Transport encoding header ":method" = "GET"
2023/11/30 12:28:49 http2: Transport encoding header ":path" = "/index.php?error=wrong_credentials"
2023/11/30 12:28:49 http2: Transport encoding header ":scheme" = "https"
2023/11/30 12:28:49 http2: Transport encoding header "content-type" = "application/x-www-form-urlencoded"
2023/11/30 12:28:49 http2: Transport encoding header "referer" = "https://tictoc2.chall.malicecyber.com/account.php"
2023/11/30 12:28:49 http2: Transport encoding header "accept-encoding" = "gzip"
2023/11/30 12:28:49 http2: Transport encoding header "user-agent" = "Go-http-client/2.0"
2023/11/30 12:28:49 http2: Framer 0xc000068e00: wrote HEADERS flags=END_STREAM|END_HEADERS stream=5 len=69
2023/11/30 12:28:49 http2: Framer 0xc000068e00: read HEADERS flags=END_HEADERS stream=5 len=28
2023/11/30 12:28:49 http2: decoded hpack field header field ":status" = "200"
2023/11/30 12:28:49 http2: decoded hpack field header field "content-encoding" = "gzip"
2023/11/30 12:28:49 http2: decoded hpack field header field "content-type" = "text/html; charset=UTF-8"
2023/11/30 12:28:49 http2: decoded hpack field header field "date" = "Thu, 30 Nov 2023 11:28:48 GMT"
2023/11/30 12:28:49 http2: decoded hpack field header field "server" = "Apache/2.4.56 (Debian)"
2023/11/30 12:28:49 http2: decoded hpack field header field "vary" = "Accept-Encoding"
2023/11/30 12:28:49 http2: decoded hpack field header field "x-powered-by" = "PHP/8.2.6"
2023/11/30 12:28:49 http2: decoded hpack field header field "content-length" = "972"
2023/11/30 12:28:49 http2: Transport received HEADERS flags=END_HEADERS stream=5 len=28
2023/11/30 12:28:49 http2: Framer 0xc000068e00: read DATA flags=END_STREAM stream=5 len=972 data="\x1f\x8b\b\x00\x00\x00\x00\x00\x00\x03\x9dV\xdbn\xdb8\x10}\xcfWp\x05\xecS!Ɏ\xedn\x9b\xda\xc6f\xdbf\x91\xc0\xf1&\xc8}\xdf(\x8a\x92\x98R$A\x8e\xec(_\xbfCIN\x9d\xf8\x82t\rX\xe6\x90\xc33s\xe6f\x1d\x102.\xa0\x94SB\xc8\x01~\xc9\xd8A-\xf9\xf4\xa0\x15\xe2Nj\x84\xdf\u0090\x9c2\xadHfuI\n\x00\xe3\x8e\xe2x\xb9\\F\x99\xa4 \xf0$b\xba\x8c3\xcby\xe8%\x17疪4\xa3Pp\x1b2\xa9ُ\xa0\xb3\xe2?I\xfe\n\xc3\x15\x15\x00\xb7\x0eP\xad\xc1\x11%\xcdyh\n\r:^\b\x05^ZR)[$\x17.\xa8\x15\x1c\xea\xb0qх\xc3a\xefp8\x18\xf5\x0f\x1b\x03a\xd89]p\x9ar;}\xb1:\x96B\xfd \x85\xe5\xd9$X\x99g\xa9\x8a\x1e]ʥX\xd8Hq\x88\x95)\xe3Dkp`\xa9\xf9s\x14\xf5\xa2\xc38\x15\x0eb\xe6\xdcσ\xa8\x14\xc8ع\x80X.'A" (716 bytes omitted)
2023/11/30 12:28:49 http2: Transport received DATA flags=END_STREAM stream=5 len=972 data="\x1f\x8b\b\x00\x00\x00\x00\x00\x00\x03\x9dV\xdbn\xdb8\x10}\xcfWp\x05\xecS!Ɏ\xedn\x9b\xda\xc6f\xdbf\x91\xc0\xf1&\xc8}\xdf(\x8a\x92\x98R$A\x8e\xec(_\xbfCIN\x9d\xf8\x82t\rX\xe6\x90\xc33s\xe6f\x1d\x102.\xa0\x94SB\xc8\x01~\xc9\xd8A-\xf9\xf4\xa0\x15\xe2Nj\x84\xdf\u0090\x9c2\xadHfuI\n\x00\xe3\x8e\xe2x\xb9\\F\x99\xa4 \xf0$b\xba\x8c3\xcby\xe8%\x17疪4\xa3Pp\x1b2\xa9ُ\xa0\xb3\xe2?I\xfe\n\xc3\x15\x15\x00\xb7\x0eP\xad\xc1\x11%\xcdyh\n\r:^\b\x05^ZR)[$\x17.\xa8\x15\x1c\xea\xb0qх\xc3a\xefp8\x18\xf5\x0f\x1b\x03a\xd89]p\x9ar;}\xb1:\x96B\xfd \x85\xe5\xd9$X\x99g\xa9\x8a\x1e]ʥX\xd8Hq\x88\x95)\xe3Dkp`\xa9\xf9s\x14\xf5\xa2\xc38\x15\x0eb\xe6\xdcσ\xa8\x14\xc8ع\x80X.'A" (716 bytes omitted)
Response: 
  <html>   
    <style>

    </style>
    <!-- Icon from https://www.flaticon.com/free-icons/grandfather-clock"  
        bg https://www.shutterstock.com/image-photo/vintage-wall-clocks-variety-styles-440243512
    -->
    <header>
        <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.0.2/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-EVSTQN3/azprG1Anm3QDgpJLIm9Nao0Yz1ztcQTwFspd3yD65VohhpuuCOmLASjC" crossorigin="anonymous">
        <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.0.2/dist/js/bootstrap.bundle.min.js" integrity="sha384-MrcW6ZMFYlzcLA8Nl+NtUVF0sA7MsXsP1UyJoMp4YLEuNSfAP+JcXn/tWtIaxVXM" crossorigin="anonymous"></script>
        <link href="./Styles/style.css" rel="stylesheet" ></link>
    </header>
    <body >
        <nav class="navbar navbar-light bg-light bannerTop">
             <span class="navbar-brand mb-0 h1"><img src="./images/clock.png" style="width:1em;"> TicToc</span>
             <span class="navbar-brand mb-0 h1"> <a href="contact.php"> <button type="button" class="btn btn-light"> Join Us </btn> </a> </span>
        </nav>

        <div style="height:auto">
            <div class="d-flex justify-content-center align-self-center">
                <fieldset>
                <div>
                    <div>
                        <h3> <i> No more time waster... </i></h3> 
                        <hr>
                        <h5>Use your connection Information to access your clocks</h4>
                        <br>
                    </div>

            <p ><h5 class="text-danger">Wrong credentials</h5></p>                <form method="post" action="login.php">
                    <div><h4>Login</h4></div>                 
                    <div><input type="text" name="username" /></div>
                    <div><h4> Password</h4></div>
                    <div><input type="password" name="password"/></div>
                    <div><input type="submit" class="btn btn-primary btn-sm mt-2" value="Login"></div>
                </form>
                </div>
                </fieldset>
            </div>
        </div>


    </body>
            
        <footer class="bannerBot">
                <div class="stage animate" style="height:100%">
                <div class="stage" style="height:100%;left:-100%">
                </div>
            </div>
        </footer>
    <html>

As pointed out in this 2012 blog post, most browsers redirect users by setting cookies and Golang does only half of this job by default. Please make this default, or do not redirect by default.
@Amodio Amodio closed this as completed Nov 30, 2023
@Amodio Amodio reopened this Nov 30, 2023
@Amodio
Copy link
Author

Amodio commented Nov 30, 2023

It works as expected in Python:

$ cat test.py
import requests

response = requests.get("http://dubbelboer.com/302cookie.php")
if response.status_code == 200:
    print(response.text)
else:
    print(f"Error: {response.status_code}")
final_url = response.url
print(f"Final URL: {final_url}")

$ python3 test.py 
Cookie was: SET<br><a href="?a=1701349447.3061">try again</a>
Final URL: https://dubbelboer.com/302cookie.php?show=1701349447.2922

@mauri870 mauri870 changed the title Cookies not set on redirections (302) by the default client net/http: Cookies not set on redirections (302) by the default client Nov 30, 2023
@mauri870 mauri870 changed the title net/http: Cookies not set on redirections (302) by the default client net/http: cookies not set on redirections (302) by the default client Nov 30, 2023
@mauri870 mauri870 added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 30, 2023
@bcmills
Copy link
Contributor

bcmills commented Nov 30, 2023

This behavior is as documented. Per https://pkg.go.dev/net/http#Client:

When following redirects, the Client will forward all headers set on the initial Request except:
…
when forwarding the "Cookie" header with a non-nil cookie Jar. …

@bcmills bcmills closed this as not planned Won't fix, can't repro, duplicate, stale Nov 30, 2023
@Amodio
Copy link
Author

Amodio commented Nov 30, 2023

Even if it is documented, Python and browsers handle it differently so, can that evolve please?

Furthermore, the documentation says the Golang HTTP (default) client forwards all headers set on the initial request (with exceptions), but here the cookie is set on the initial response, isn't it?
I understand this: If Jar is nil, the initial cookies are forwarded without change. as: by default, cookies are forwarded (= it should work), is it wrong?

@bcmills bcmills reopened this Nov 30, 2023
@seankhliao
Copy link
Member

There'd be an import cycle if net/http tried to import net/http/cookiejar. It would also make the default client much easier to track.

cc @neild

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Amodio @bcmills @mauri870 @seankhliao