This is either necessary and correct per HTTP spec, or a bug/missing feature. This issue needs investigation to determine which.

CC @bradfitz, @empijei.

I fear this might be neither. RFC 2396 allows for the single slash to be a path separator. This means that a URL with two consecutive slashes implies an empty path segment.

Now, while that spec is clear on the sintactic meaning of an empty path segment, it's not very clear what the semantics of that should be. That is left to the implementation. Filesystem implementations all ignore empty path segments, while other implementations differ.

So this is the point about URL specifications and server implementations: the server (origin) can decide what meaning to attribute to an empty path segment. Go aligns with the Unix interpretation and ignores empty path components on server multiplexers and routers.

So now we have to look into the proxy side of specs: is a hop in HTTP a receiving end for the statement above or not?

In this case we are looking at gateways (aka reverse proxies). This paragraph says that requirements for a gateway are the same for origin servers on one side, and user agents for the other side and adds no restrictions. I think the best approach here would be to make a proxy rely a message as transparently as possible, so not cleaning empty path segments.

So the question is not on what the spec says, but on what we decide might be better to do.

That said Go's ReverseProxy does not strip empty path components. The logic linked here is that of a ServeMux. If you run a ReverseProxy as Handler on an http.Server the paths will be forwarded as-is.

I don't think there is any change needed here as ReverseProxy already behaves in the best possible way. Your issue here is that you are using a ServeMux and when you do so you are not asserting that you are just a gateway, you also become an authoritative origin and, as such, you have to attribute meaning to paths.

Long story short

• httputil.ReverseProxy does not strip empty path components, and this is in line with the best possible interpretation of the spec
• http.ServeMux behaves as an origin, and as such it attributes meaning to path components. In this case empty ones are discarded a-la-UNIX
• Combining a reverse proxy with a server multiplexer makes the behavior of a server multiplexer prevail, which seems reasonable to me (for consistency with other paths that might be registered on the mux).

So, before we close this I have a couple of questions for @iiiusky:

1. The code you linked is in ServeMux, not in ReverseProxy. Are you using a ReverseProxy inside a ServeMux for another service or are those separate things? If you just need to proxy you can do so without a ServeMux and maybe get around the issue? (Note: you can set any http handler as Handler in a Server, not just muxes).
2. What is the technology used for the other server? Go should accept empty path segments without issues.

@empijei
My code is roughly as follows:

func handleRequestAndRedirect(res http.ResponseWriter, req *http.Request) {
	var xDefaultTransport http.RoundTripper = &http.Transport{
		DialContext: (&net.Dialer{
			Timeout:   30 * time.Second,
			KeepAlive: 30 * time.Second,
			DualStack: true,
		}).DialContext,
		ForceAttemptHTTP2:     true,
		MaxIdleConns:          100,
		IdleConnTimeout:       90 * time.Second,
		TLSHandshakeTimeout:   10 * time.Second,
		ExpectContinueTimeout: 1 * time.Second,
		TLSClientConfig: &tls.Config{
			InsecureSkipVerify: true, // test server certificate is not trusted.
		},
	}

	url, _ := url.Parse(target)

	proxy := httputil.NewSingleHostReverseProxy(url)
	proxy.Transport = xDefaultTransport
	proxy.ModifyResponse = ModifyResponse

	req.URL.Host = url.Host
	req.URL.Scheme = url.Scheme
	req.Header.Set("X-Forwarded-Host", req.Header.Get("Host"))
	req.Header.Set("Referer", strings.ReplaceAll(req.Header.Get("Referer"),
		domain, target))
	proxy.ServeHTTP(res, req)
}

http.HandleFunc("/", handleRequestAndRedirect)
if err := http.ListenAndServe(fmt.Sprintf("%s:%d", bindIP, bindPort), nil); err != nil {
		panic(err)
}

The main realization is that I use http.ListenAndServe to open a web service, and then add a handle to it, this handle uses anti-generation, and can modify the response content.

http.HandleFunc registers your handler on DefaultServeMux, which is a Mux as I suspected.

Can you please change your last 4 lines to be:

s := http.Server{
  Addr: fmt.Sprintf("%s:%d", bindIP, bindPort),
  Handler: http.HandlerFunc(handleRequestAndRedirect),
}
if err := s.ListenAndServe(fmt.Sprintf("%s:%d", bindIP, bindPort), nil); err != nil {
		panic(err)
}

and check if it works?

It works fine instead

s := http.Server{
		Addr: fmt.Sprintf("%s:%d", bindIP, bindPort),
		Handler: http.HandlerFunc(handleRequestAndRedirect),
	}
	if err := s.ListenAndServe(); err != nil {
		panic(err)
	}

thinks.