Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: configurable error message for Client sent an HTTP request to an HTTPS server. #49310

Open
mzky opened this issue Nov 3, 2021 · 29 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@mzky
Copy link

mzky commented Nov 3, 2021

What version of Go are you using (go version)?

$ go version
go version go1.17.2 linux/amd64

Does this issue reproduce with the latest release?

yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/root/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn"
GOROOT="/var/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/var/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17.2"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/root/go/src/net/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1909543103=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Start the HTTPS service written in golang, and the user accesses the HTTP address.

What did you expect to see?

Display localized language information or customize html page or automatically jump to https address.

What did you see instead?

  1. Chrome and Firefox:
    display the following information
    image

  2. Other browsers:
    The IE browser prompts a 400 error, and the instructional content is not displayed.
    Browsers using the Chromium kernel do not display any visible content.
    image
    Users will not be able to understand the current situation.

P.S.
Adding custom options to net/http can solve this problem:
https://github.com/mzky/go/blob/dev.boringcrypto.go1.17/src/net/http/server.go
./make.bash verification is normal

P.P.S.
Use reference:

var tlsBadRequest := `HTTP/1.1 301 Moved Permanently

<!DOCTYPE html>
<html><head><meta charset="UTF-8">
<title>Automatically jump to HTTPS</title
<script type="text/javascript">url = window.location.href.replace("http:", "https:");window.location.replace(url);</script>
</head>
<body></body>
</html>`

http.ListenAndServeTLS2(addr, serverPem, serverKey, tlsBadRequest, Handler)

@mzky mzky changed the title net/http It is recommended to add an optional https downgrade access prompt net/http: It is recommended to add an optional https downgrade access prompt Nov 3, 2021
@thanm thanm added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 3, 2021
@thanm thanm added this to the Backlog milestone Nov 3, 2021
@thanm
Copy link
Contributor

thanm commented Nov 3, 2021

Thanks for raising the issue.

From my read this sounds more like a feature request as opposed to a bug, correct?

@neild

@neild
Copy link
Contributor

neild commented Nov 3, 2021

The feature request is to respond to an HTTP request on an HTTPS port with a configurable error message.

Testing several major websites (www.google.com, www.amazon.com, www.microsoft.com), none of them respond with an error message to an HTTP request on port 443. Two close the connection without response, one leaves the connection open but does not respond.

I have not checked any other HTTPS server implementations (Apache, nginx, etc.) to see how they handle this condition. It would be interesting to know if any attempt to report an error to the peer in this case.

In the absence of evidence that this is a common feature, I don't think we should add this.

@mzky
Copy link
Author

mzky commented Nov 4, 2021

Hi ! This is a feature request, not a bug.

I found some examples of Nginx configuring HTTP redirection to HTTPS:https://linuxize.com/post/redirect-http-to-https-in-nginx/

  1. Old version Parameters:
    ···
    rewrite ^(.*)$ https://$host$1 permanent;
    ···
  2. New version Parameters:
    ···
    return 301 https://$server_name$request_uri;
    ···

I want to replace nginx's auto-jump feature with Golang.
Of course, this is just a suggestion, the changes will not affect the original functions.

@seankhliao
Copy link
Member

HTTP redirection is different from the filed issue. It's already possible by listening on both HTTP and HTTPS ports and handling the redirection within the handlers

go http.ListenAndServe(":80", http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
        // handle redirect here
}))
http.ListenAndServeTLS(":443", "...", "...", nil)

@mzky
Copy link
Author

mzky commented Nov 5, 2021

HTTP redirection is different from the filed issue. It's already possible by listening on both HTTP and HTTPS ports and handling the redirection within the handlers

go http.ListenAndServe(":80", http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
        // handle redirect here
}))
http.ListenAndServeTLS(":443", "...", "...", nil)

Yes, different port forwarding is also possible, but I need a forwarding on the same port

@mzky
Copy link
Author

mzky commented Nov 15, 2021

They all jump to https with the same port:
image
image
image
image

@mzky
Copy link
Author

mzky commented Nov 15, 2021

Go language simulates google:

image

image

@neild
Copy link
Contributor

neild commented Nov 15, 2021

They all jump to https with the same port:

An https:// URL with no port uses a default port of 443, not 80. All these are redirecting from HTTP on port 80 to HTTPS on port 443.

@seankhliao seankhliao changed the title net/http: It is recommended to add an optional https downgrade access prompt net/http: configurable error message for Client sent an HTTP request to an HTTPS server. Feb 8, 2022
@hktalent
Copy link

hktalent commented Mar 6, 2023

I would also like to make some comments on this issue

  • First of all, since the server can output "Client send an HTTP request to an HTTPS server", it should give the control of the output to the developer instead of burying him deeply

  • Secondly, to solve this problem, we should not use nginx or listen to port 80 more, which are very cumbersome

@hktalent
Copy link

Yes, any core modifications don't seem to be standard-compliant, but, that's the truth, we need a way to intercept when "Client send an HTTP request to an HTTPS server" happens, or have a kind of friendly wrapper that allows him to Controllable output, because "Client send an HTTP request to an HTTPS server" directly output to the browser is very unfriendly, and it is uncontrollable and unexpected

@mitar
Copy link
Contributor

mitar commented Nov 17, 2023

I would also like that this is configurable. Maybe http.Server could get the following field:

TLSHandshakeBadRequest func(conn net.Conn)

And if not defined, current:

io.WriteString(re.Conn, "HTTP/1.0 400 Bad Request\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")

is called. Otherwise TLSHandshakeBadRequest(re.Conn) is called.

Open to other field names.

@martin0104
Copy link

i have same problem that response content not friendly, i wish can configurable the content

@bddjr
Copy link

bddjr commented Feb 16, 2024

The nginx example you provided is incorrect, it should be like this:

# port 443
error_page 497 https://$host$request_uri;

# other ports
error_page 497 https://$host:$server_port$request_uri;

Then, attempting to access the HTTPS port using HTTP will return a 302 redirect.
For example:
http://q8p.cc:443


For this, I wrote a script to implement browser redirection without affecting the HTTP 400 status code.
https://github.com/bddjr/go-https-port-auto-redirect-for-http

HTTP/1.1 400 Bad Request
Content-Type: text/html
Connection: close

<!-- Client sent an HTTP request to an HTTPS server. -->
<script> location.protocol = 'https:' </script>

@bddjr
Copy link

bddjr commented Feb 16, 2024

Maybe we can read the "Host" request header, then return 307 status code

The following is a number of debugging screenshots, not yet implemented the above functions

src/crypto/tls/conn.go
image

src/net/http/server.go
image

test.go
image

@bddjr
Copy link

bddjr commented Feb 17, 2024

I did it!
Next, I will submit a pull request.

image

image

image

@gopherbot
Copy link

Change https://go.dev/cl/564997 mentions this issue: net/http: configurable error message for Client sent an HTTP request …

@neild
Copy link
Contributor

neild commented Feb 27, 2024

It looks like nginx responds to an HTTP request on an HTTPS port with an error 497 (an nginx-specific code), and permits code-specific error handler overrides. The example nginx configuration from above is interesting, because it seems to indicate that nginx is parsing the full HTTP request in this case:

error_page 497 https://$host$request_uri;

This points at a question: Is the desired feature request to customize the static message we send when responding to a misdirected HTTP request, or to respond with a redirect to the correct location? The latter is substantially more complicated to implement, because it requires effectively restarting the connection.

If were were to add a feature, I could imagine something like:

type Server struct {
  // HTTPOnHTTPSPortErrorResponseString is sent on an HTTPS connection
  // which receives what looks like an HTTP request.
  HTTPOnHTTPSPortErrorResponse string
}

But that's static, and it sounds like a common use case is a redirect. So maybe what people want is:

type Server struct {
  // HTTPOnHTTPSPortErrorHandler handles HTTP requests sent to an HTTPS port.
  HTTPOnHTTPSPortErrorHandler Handler
}

Although in that case, what happens when someone inevitably sets Server.Handler and Server.HTTPOnHTTPSPortErrorHandler to the same thing? Does that work? Should that work? Are there edge cases where it fails? I think implementation for this would also be complicated; unless I'm wrong, I don't think this is something we'd want to do.

A middle ground, suggested by @mitar above, is essentially a roll-your-own handler where we hand the net.Conn to the user:

type Server struct {
  TLSHandshakeBadRequest func(conn net.Conn)
}

You're going to want the already-consumed bytes from the net.Conn as well though, no?

I don't have any clear answers here. There's been a fair bit of discussion on this issue over time, but half of it is about cases other than an HTTP request sent to an HTTPS port. (For example, redirecting from HTTP on port 80 to HTTPS on port 443, which of course is something you can do today.) Some of the example use cases (such as that nginx config) involve more than just customizing the error string.

Would a statically customizable response string be enough? That's at least implementable without adding a ton of complexity.

@bddjr
Copy link

bddjr commented Feb 27, 2024

A static string can be redirected in the browser using JavaScript,
but it doesn't fit the 301 status code redirect that some people want

"HTTP/1.1 400 Bad Request\r\nContent-Type: text/html\r\nConnection: close\r\n\r\n<!-- Client sent an HTTP request to an HTTPS server. -->\n<script> location.protocol = 'https:' </script>\n"
HTTP/1.1 400 Bad Request
Content-Type: text/html
Connection: close

<!-- Client sent an HTTP request to an HTTPS server. -->
<script> location.protocol = 'https:' </script>

@mitar
Copy link
Contributor

mitar commented Feb 27, 2024

@neild Yea, I realized this in https://go-review.googlesource.com/c/go/+/564997 as well. I think we could have:

type Server struct {
  TLSHandshakeBadRequest func(conn net.Conn, recordedBytes []byte)
}

But I think those recorded bytes are really optional and it might be hard for later on for implementation to change its behavior. If we look at current code, it does not use those bytes, it just writes to the connection:

io.WriteString(re.Conn, "HTTP/1.0 400 Bad Request\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")

So I was envisioning that the TLSHandshakeBadRequest would do the same, only maybe write a Location header redirect.

I am not sure how much utility is in inspecting recordedBytes?

@bddjr
Copy link

bddjr commented Feb 28, 2024

I think it's possible to provide multiple interfaces for different use cases, like TLSConfig's NameToCertificate and GetCertificate

(I think the relevant configuration should be written in TLSConfig)

type Config struct {
  // HttpOnHttpsPortErrorResponseString is sent on an HTTPS connection
  // which receives what looks like an HTTP request.
  HttpOnHttpsPortErrorResponse string

  // HttpOnHttpsPortErrorRedirect is sent on an HTTPS connection
  // which receives what looks like an HTTP request.
  // If true, 307 redirect will be sent
  HttpOnHttpsPortErrorRedirect bool

  // HttpOnHttpsPortErrorHandler handles HTTP requests sent to an HTTPS port.
  HttpOnHttpsPortErrorHandler func(conn net.Conn, requestBytes []byte)

Priority is from bottom to top

@bddjr
Copy link

bddjr commented Feb 28, 2024

I am not sure how much utility is in inspecting recordedBytes?

307 redirect

var compiledRegExp_httpHost = regexp.MustCompile(`\r\nHost: \S+`) // "\r\nHost: local.q8p.cc:5678"
var compiledRegExp_httpPath = regexp.MustCompile(`/\S*`)          // "/index.html"

func httpOnHttpsPortErrorHandler (conn net.Conn, requestBytes []byte) {
    requestString := string(requestBytes)
    Host := compiledRegExp_httpHost.FindString(requestString) // "\r\nHost: local.q8p.cc:5678"
    if Host == "" {
        io.WriteString(conn, "HTTP/1.1 400 Bad Request\r\n"+
            "\r\n"+
            "Missing required Host header.\n",
        )
        return
    }
    Host = Host[8:]                                             // "local.q8p.cc:5678"
    Path := compiledRegExp_httpPath.FindString(requestString) // "/index.html"
    io.WriteString(conn, "HTTP/1.1 307 Temporary Redirect\r\n"+
        "Location: https://"+Host+Path+"\r\n"+
        "Connection: close\r\n"+
        "\r\n"+
        "Client sent an HTTP request to an HTTPS server.\n",
    )
}
HTTP/1.1 307 Temporary Redirect
Location: https://local.q8p.cc:5678/index.html
Connection: close

Client sent an HTTP request to an HTTPS server.

@bddjr
Copy link

bddjr commented Feb 28, 2024

I just found a more standard request parsing method

req, err := http.ReadRequest(bufio.NewReader(bytes.NewReader(requestBytes)))
if err != nil {
	fmt.Println(err)
	return
}
io.WriteString(conn, fmt.Sprintf(
	"HTTP/1.1 307 Temporary Redirect\r\nLocation: https://%s%s\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n",
	req.Host, req.URL.Path,
))

@mitar
Copy link
Contributor

mitar commented Feb 28, 2024

I see. To allow virtual hosts redirects. And if requestBytes does not contain enough data (not yet all headers), you can still read more from the connection?

@bddjr
Copy link

bddjr commented Feb 28, 2024

🤔I just tried, it doesn't work, I'll see if there's an interface to continue reading

The request content cannot exceed 576 bytes, otherwise the request cannot be parsed

@bddjr
Copy link

bddjr commented Feb 29, 2024

I successfully read the remaining bytes from conn

server.go

// net/http: configurable error message for Client sent an HTTP request to an HTTPS server.
// https://go.dev/issue/49310
func (c *conn) httpOnHttpsPortErrorHandler(conn net.Conn, recondBytes []byte) {
	// Read Response string
	badRequestResponse := c.server.TLSConfig.HttpOnHttpsPortErrorResponse
	if badRequestResponse == "" {
		badRequestResponse = "HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n"
	}

	// Handler
	if handler := c.server.TLSConfig.HttpOnHttpsPortErrorHandler; handler != nil {
		handler(conn, recondBytes, badRequestResponse)
		return
	}

	// Redirect
	if c.server.TLSConfig.HttpOnHttpsPortErrorRedirect {
		// Read Header
		req, _, err := ReadRequestForHttpOnHttpsPortErrorHandler(conn, recondBytes)
		if err != nil {
			io.WriteString(conn, badRequestResponse)
			return
		}

		// Send Redirect
		io.WriteString(conn, fmt.Sprintf(
			"HTTP/1.1 307 Temporary Redirect\r\nLocation: https://%s%s\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n",
			req.Host, req.URL.Path,
		))
		return
	}

	// Send Response string
	io.WriteString(conn, badRequestResponse)
}

func ReadRequestForHttpOnHttpsPortErrorHandler(conn net.Conn, recondBytes []byte) (req *Request, reqBytes []byte, err error) {
	// Max 4KB
	if len(recondBytes) > 4096 {
		return nil, recondBytes, errors.New("recondBytes too long")
	}

	// Content length may be insufficient, continue reading.
	if len(recondBytes) < 4096 && !bytes.Contains(recondBytes, []byte("\r\n\r\n")) {
		b := make([]byte, 4096-len(recondBytes))
		// Set Timeout 1s
		conn.SetReadDeadline(time.Now().Add(time.Duration(1 * time.Second)))
		n, err := conn.Read(b)
		if err != nil {
			return nil, recondBytes, err
		}
		recondBytes = append(recondBytes, b[:n]...)
	}

	// Read Request
	req, err = ReadRequest(bufio.NewReader(bytes.NewReader(recondBytes)))
	if err != nil {
		return nil, recondBytes, err
	}
	if req.Host == "" {
		return nil, recondBytes, errors.New("missing required Host header")
	}
	return req, recondBytes, nil
}

image

image

@bddjr
Copy link

bddjr commented Feb 29, 2024

If this is enough, I will submit a PullRequest
@mitar

TLSConfig

type Config struct {
	// HttpOnHttpsPortErrorResponseString is sent on an HTTPS connection
	// which receives what looks like an HTTP request.
	//   "HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n"
	HttpOnHttpsPortErrorResponse string

	// HttpOnHttpsPortErrorRedirect is sent on an HTTPS connection
	// which receives what looks like an HTTP request.
	// If true, 307 redirect will be sent
	HttpOnHttpsPortErrorRedirect bool

	// HttpOnHttpsPortErrorHandler handles HTTP requests sent to an HTTPS port.
	//
	// WriteString use:
	//   io.WriteString(conn, "HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")
	// Parse the request header use:
	//   http.ReadRequestForHttpOnHttpsPortErrorHandler(conn, recondBytes)
	HttpOnHttpsPortErrorHandler func(conn net.Conn, recondBytes []byte, badRequestResponse string)

@mitar
Copy link
Contributor

mitar commented Feb 29, 2024

@bddjr Please read contributing guide. I think this proposal has not yet been accepted. Making a pull request is fine to be able to discuss code, but until the proposal is accepted it cannot be merged.

And about your last proposal: personally, I think the Config is overkill, I think we should have only one function callback and not so many options.

Also, what is badRequestResponse in HttpOnHttpsPortErrorHandler?

@bddjr
Copy link

bddjr commented Feb 29, 2024

Also, what is badRequestResponse in HttpOnHttpsPortErrorHandler?

It will be TLSConfig.HttpOnHttpsPortErrorResponse

	// Read Response string
	badRequestResponse := c.server.TLSConfig.HttpOnHttpsPortErrorResponse
	if badRequestResponse == "" {
		badRequestResponse = "HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n"
	}

	// Handler
	if handler := c.server.TLSConfig.HttpOnHttpsPortErrorHandler; handler != nil {
		handler(conn, recondBytes, badRequestResponse)
		return
	}

@bddjr
Copy link

bddjr commented Mar 30, 2024

I made a go mod hijacking net.Conn to implement this feature

https://github.com/bddjr/hlfhr

var srv *hlfhr.Server

func main() {
	srv = hlfhr.New(&http.Server{
		Addr:              ":5678",
		Handler:           http.HandlerFunc(httpResponseHandle),
		ReadHeaderTimeout: 10 * time.Second,
		IdleTimeout:       10 * time.Second,
	})
	// Then just use it like &http.Server .

	err := srv.ListenAndServeTLS("localhost.crt", "localhost.key")
	if err != nil && err != http.ErrServerClosed {
		fmt.Println(err)
	}
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

9 participants