net/http: Transport doesn't support NTLM challenge authentication #20053

chen-keinan · 2017-04-20T17:18:51Z

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

Go 1.8

What operating system and processor architecture are you using (`go env`)?

  OSX darwin-amd64

What did you do?

I have send an https request to a proxy (ntlm) below request and initial response
(via wireshark)

Request:
CONNECT www.endpoint.com:443 HTTP/1.1
Host: www.endpoint.com:443
User-Agent: Go-http-client/1.1
Location: https://www.endpoint.com
Proxy-Authorization: NTLM TlRMTVNTUAABAAAAB4IAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAMAA=
------------------------------------------------------------------------------------------
Response
HTTP/1.1 407 Proxy Authentication Required
Server: FreeProxy/4.50
Date: Thu, 20 Apr 2017 15:20:10 GMT
Content-Type: text/html
Transfer-Encoding: Chunked
Proxy-Authenticate: NTLM
TlRMTVNTUAACAAAADAAMADgAAAAFgoECloLVra5EaVAAAAAAAAAAA
A9KAEYAUgBPAEcAMAACAAwASgBGAFIATwBHADAAAQAOAFcASQBOA
ZgByAG8AZwAuAGwAbwBjAGEAbAADACYAdwBpAG4AMgAwADEAMgAu
wAbwBjAGEAbAAFABYAagBmAHIAbwBnAC4AbABvAGMAYQBsAAcACAD
Proxy-Connection: Keep-Alive
------------------------------------------------------------------------------------------------
The response above never reach the client, on transport.dialConn the response return status code 407 for challenge , because the response code != 200 the persist connection become nil

  -------------------------------------------------------------------------------------------
	br := bufio.NewReader(conn)
	resp, err := ReadResponse(br, connectReq) // resp.StatusCode =407
	if err != nil {
		conn.Close()
		return nil, err
	}
	if resp.StatusCode != 200 {
		f := strings.SplitN(resp.Status, " ", 2)
		conn.Close()
		return nil, errors.New(f[1]) // persist connection become nil 
	}

since the persist connection return nil then request is cancelled and response return as nil
with error Proxy Authentication Required
see --> transport.RoundTrip

 --------------------------------------------------------------------------------------------
     pconn, err := t.getConn(treq, cm) // pconn = nil
	if err != nil {
		t.setReqCanceler(req, nil)
		req.closeBody()
		return nil, err
	}
-------------------------------------------------------------------------------------------------

What did you expect to see?

I expect the response to return is it send from the proxy with status code 407

What did you see instead?

I got nil response with error: Proxy Authentication Required

Note: if I use http instead of https it works OK

This issue is blocking us from developing support to NTLM Proxy , as requests https endpoint do not return challenge from proxy

The text was updated successfully, but these errors were encountered:

bradfitz · 2017-04-20T17:37:20Z

I believe this is all working as intended.

If your proxy returns an error, we don't want to return its HTTP response to the user, as that would imply the origin server replied with that.

You can use ProxyConnectHeader to authenticate to your proxy.

Let me know if I misunderstand something.

chen-keinan · 2017-04-20T17:42:27Z

I did used ProxyConnectHeader to send the Proxy-Authorization which is OK , at that point the CONNECT NEGOTIATE started , then proxy return 407 with Challenge header which I do not received in the client due to issue describe above.
407 is good , it is part of the NTLM proto it will enable me to continue with negotiation , same as its done with http.

since the response come as nil I cannot process the response , if the response would return with 407 and challenge (as proxy send it) it will help me, on the client side to decide on how to continue

bradfitz · 2017-04-20T17:46:54Z

I see. It's true we don't support authentication that takes multiple rounds.

chen-keinan · 2017-04-20T17:49:41Z

the same NTLM negotiation works in http , why is https different ?

bradfitz · 2017-04-20T17:56:03Z

Because HTTPS does CONNECT and authenticates to that to give your channel for future requests.

With HTTP you're kinda getting lucky and happen to be using the same TCP connection I suppose, but there was never explicit design or support for what you're trying to do.

chen-keinan · 2017-04-20T18:04:15Z

do you have any suggested workaround for this issue in the meantime ?

bradfitz · 2017-04-20T18:18:06Z

Implement Transport.DialContext (but leave Transport.Proxy nil) and do your own CONNECT setup before giving the Conn back to the http package?

gogolok · 2017-07-10T15:59:06Z

hey @chenkjfrog

I've built a modified Go version that supports NTLM proxies.
I'm using it to build a Cloud Foundry CLI with NTLM support.
It's WIP and I'm not sure what solution someone should target to support this in the Go base/master in the future but I hope it can somehow help you :-)

https://gist.github.com/gogolok/018443687392ea4682bd82ac8712b363

gogolok · 2017-09-08T13:28:17Z

I'm currently working on extending http standard library to support authentication that takes multiple rounds.

Hopefully I can present something in the near future.

bhendo · 2017-09-25T19:10:02Z

@gogolok Thanks for this!

I want to bring one thing to your attention.

I've been doing some testing with your patch and have found one scenario that does not work.
When a request is made with the http scheme through an NTLM proxy that also intercepts the request, the proxy does not forward the request to the end server. My understanding is that the CONNECT method is meant for tunneling https requests only.

I think the reason the patch works for proxies that don't intercept is because the proxy "blindly" establishes a connection from the client to the server. But when a proxy is in intercept mode it actually maintains two connections: client<->proxy & proxy<->server

I captured some packets with WireShark to see how chrome handles http requests through an NTLM proxy. It seems to use the original request, method (e.g. GET, POST, etc), and body for each part of the handshake.

After a quick review of transport.go I think the NTLM handshake specifically for http requests might need to take place in RoundTrip rather than dialConn

I will admit this all is a bit out of my wheelhouse, so take my last statement with a grain of salt :)

gogolok · 2017-10-12T21:26:52Z

@bhendo Thanks for your feedback. I'm currently testing and extending my new implementation and will try to consider the intercept mode. I might come back to you then :-)

Rots · 2019-01-17T15:44:21Z

There's an implementation for NTLM transport at: https://github.com/Azure/go-ntlmssp

bhendo · 2019-01-18T19:25:04Z

It looks like this issue should be closed for the same reasons as #22288

The dial context solution recommended in that issue: #22288 (comment) still has the specific failing I described above #20053 (comment)

A while back I ended up writing my own transport to deal with this scenario . If anyone is looking for a transport that can handle proxies that require NTLM Proxy Authentication they are welcome to use it.

mholt · 2019-11-05T20:11:20Z

This might be a little different from what you're doing, but for the record: I was able to proxy requests from a web browser to a backend server that requires NTLM authentication by using a distinct *http.Transport for each downstream/client connection. This works, I think, because it forces each client connection into its own connection (or connection pool) which has a distinct auth context. Basically it does what nginx's docs say for their commercial ntlm module:

Allows proxying requests with NTLM Authentication. The upstream connection is bound to the client connection once the client sends a request with the “Authorization” header field value starting with “Negotiate” or “NTLM”. Further client requests will be proxied through the same upstream connection, keeping the authentication context.

Just thought I'd post that here in case it was helpful to anyone.

(We'll be rolling it out in Caddy 2 pretty soon, so you'll be able to look at the source code at that point. Edit: Here's the code.)

chen-keinan changed the title ~~https request via proxy which return 407 return nil response~~ https request via proxy return nil response Apr 20, 2017

bradfitz closed this as completed Apr 20, 2017

bradfitz reopened this Apr 20, 2017

bradfitz added this to the Unplanned milestone Apr 20, 2017

bradfitz changed the title ~~https request via proxy return nil response~~ net/http: Transport doesn't support NTLM challenge authentication Apr 20, 2017

bradfitz added FeatureRequest help wanted labels Jun 15, 2017

gogolok mentioned this issue Oct 16, 2017

proposal: net/http: Proxy: Support authentication that takes multiple rounds #22288

Closed

bhendo mentioned this issue Oct 4, 2019

Intercept Proxy Edge Case bdwyertech/gontlm-proxy#1

Closed

akrantz01 mentioned this issue Aug 2, 2021

[input.http] Fails on NTLM authorization influxdata/telegraf#9192

Closed

sjwang90 mentioned this issue May 24, 2022

NTLM authentication support influxdata/telegraf#11184

Closed

3 tasks

n1hility mentioned this issue Nov 10, 2023

Investigate NTLM proxy support containers/podman#20656

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net/http: Transport doesn't support NTLM challenge authentication #20053

net/http: Transport doesn't support NTLM challenge authentication #20053

chen-keinan commented Apr 20, 2017 •

edited

bradfitz commented Apr 20, 2017

chen-keinan commented Apr 20, 2017 •

edited

bradfitz commented Apr 20, 2017

chen-keinan commented Apr 20, 2017 •

edited

bradfitz commented Apr 20, 2017

chen-keinan commented Apr 20, 2017

bradfitz commented Apr 20, 2017

gogolok commented Jul 10, 2017 •

edited

gogolok commented Sep 8, 2017

bhendo commented Sep 25, 2017 •

edited

gogolok commented Oct 12, 2017

Rots commented Jan 17, 2019

bhendo commented Jan 18, 2019

mholt commented Nov 5, 2019 •

edited

net/http: Transport doesn't support NTLM challenge authentication #20053

net/http: Transport doesn't support NTLM challenge authentication #20053

Comments

chen-keinan commented Apr 20, 2017 • edited

What version of Go are you using (go version)?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

bradfitz commented Apr 20, 2017

chen-keinan commented Apr 20, 2017 • edited

bradfitz commented Apr 20, 2017

chen-keinan commented Apr 20, 2017 • edited

bradfitz commented Apr 20, 2017

chen-keinan commented Apr 20, 2017

bradfitz commented Apr 20, 2017

gogolok commented Jul 10, 2017 • edited

gogolok commented Sep 8, 2017

bhendo commented Sep 25, 2017 • edited

gogolok commented Oct 12, 2017

Rots commented Jan 17, 2019

bhendo commented Jan 18, 2019

mholt commented Nov 5, 2019 • edited

chen-keinan commented Apr 20, 2017 •

edited

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

chen-keinan commented Apr 20, 2017 •

edited

chen-keinan commented Apr 20, 2017 •

edited

gogolok commented Jul 10, 2017 •

edited

bhendo commented Sep 25, 2017 •

edited

mholt commented Nov 5, 2019 •

edited