net/http: ServeFile panics when StripPrefix over-strips and results in empty path #30165

ggicci · 2019-02-11T09:48:36Z

What version of Go are you using (`go version`)?

$ go version
go version go1.11.4 darwin/amd64

Does this issue reproduce with the latest release?

YES

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ggicci/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/ggicci/workspace"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.4/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ht/hdwl47w16m10r45w89drkkph0000gn/T/go-build233590303=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I did use http.StripPrefix accompanied with http.ServeFile to strip URL prefix and serve a folder. And the HTTP server will panic and the web page will receive nothing if I opened http://localhost:8080/download/.

Here's a code snippet to reproduce the problem:

package main

import "net/http"

var myHandler = http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
	http.ServeFile(rw, r, "./files")
})

func main() {
	http.Handle("/download/", http.StripPrefix("/download/", myHandler))
	http.ListenAndServe(":8080", nil)
}

What did you expect to see?

I should see contents from my files folder. File list or index page content.

What did you see instead?

No HTTP response, but panics information from stderr.

2019/02/11 17:43:17 http: panic serving [::1]:56154: runtime error: index out of range
goroutine 19 [running]:
net/http.(*conn).serve.func1(0xc00008a960)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00011a000, 0xc000122000, 0x12fdce0, 0xc000010030, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00011a000, 0xc000122000, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00011a000, 0xc000122000)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00011a000, 0xc000122000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00008a960, 0x1300100, 0xc000096200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:43:17 http: panic serving [::1]:56174: runtime error: index out of range
goroutine 20 [running]:
net/http.(*conn).serve.func1(0xc00008aa00)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00015e000, 0xc00016c000, 0x12fdce0, 0xc00015a010, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00015e000, 0xc00016c000, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00015e000, 0xc00016c000)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00015e000, 0xc00016c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00008aa00, 0x1300100, 0xc000096300)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:43:17 http: panic serving [::1]:56175: runtime error: index out of range
goroutine 21 [running]:
net/http.(*conn).serve.func1(0xc00008aaa0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc000194000, 0xc00010c300, 0x12fdce0, 0xc000088d50, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc000194000, 0xc00010c300, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc000194000, 0xc00010c300)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc000194000, 0xc00010c300)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00008aaa0, 0x1300100, 0xc000096480)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:43:17 http: panic serving [::1]:56176: runtime error: index out of range
goroutine 5 [running]:
net/http.(*conn).serve.func1(0xc0001ae000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00011a0e0, 0xc000122300, 0x12fdce0, 0xc000010080, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00011a0e0, 0xc000122300, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00011a0e0, 0xc000122300)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00011a0e0, 0xc000122300)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc0001ae000, 0x1300100, 0xc00005e200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:44:47 http: panic serving [::1]:56544: runtime error: index out of range
goroutine 35 [running]:
net/http.(*conn).serve.func1(0xc0001c2000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00015e0e0, 0xc00016c200, 0x12fdce0, 0xc00015a070, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00015e0e0, 0xc00016c200, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00015e0e0, 0xc00016c200)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00015e0e0, 0xc00016c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc0001c2000, 0x1300100, 0xc0001580c0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5

The text was updated successfully, but these errors were encountered:

gopherbot · 2019-02-11T10:20:49Z

Change https://golang.org/cl/161738 mentions this issue: net/http: StipPrefix clean URL path

gopherbot · 2019-06-04T21:23:43Z

Change https://golang.org/cl/180498 mentions this issue: net/http: roll back "clean the path of the stripped URL by StripPrefix"

dmitshur · 2019-06-04T22:02:11Z

I investigated this and here's some more information about why this happens.

The URL path for a GET /download/ request currently ends up being the empty string after http.StripPrefix("/download/", myHandler) strips the prefix.

Calling http.ServeFile on a *http.Request with the empty URL path panics in http.serveFile here:

// redirect if the directory name doesn't end in a slash
if d.IsDir() {
	url := r.URL.Path
	if url[len(url)-1] != '/' { // panic here, since url is empty string
		localRedirect(w, r, path.Base(url)+"/")
		return
	}
}

/cc @bradfitz

gopherbot · 2019-06-04T22:27:27Z

Change https://golang.org/cl/180499 mentions this issue: net/http: don't panic serving dir in ServeFile with empty Request.URL.Path

Roll back CL 161738. That fix changed StripPrefix behavior in the general case, not just in the situation where where stripping the prefix from path resulted in the empty string, causing issue #31622. That kind of change to StripPrefix behavior is not backwards compatible, and there can be a smaller, more targeted fix for the original issue. Fixes #31622 Updates #30165 Change-Id: Ie2fcfe6787a32e44f71d564d8f9c9d580fc6f704 Reviewed-on: https://go-review.googlesource.com/c/go/+/180498 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

dmitshur · 2019-06-04T22:43:57Z

Re-opening this because the fix was rolled back in CL 180498.

For (my) reference, the code where panic happens was added in CL 20128 to fix #13996. It didn't take into account that r.URL.Path can be empty string when http.StripPrefix is used in a way that over-strips the prefix.

When fixing this, need to make sure it won't introduce an infinite or otherwise unwanted redirects. The analysis in the comment #13996 (comment) is good and should be taken into account.

odeke-em · 2019-07-31T13:49:47Z

I see this issue was moved to Go1.14 but since it is a regression perhaps for Go1.13, but also Brad's CL https://golang.org/cl/180499 LGTM, just that I don't have access to update his CL with a code review comment that was requested to use strings.HasPrefix instead of a plain comparison to ""

@dmitshur @andybons can we perhaps make that happen and submit that CL? Thank you.

andybons · 2019-07-31T16:27:21Z

@odeke-em it’s too late for this to go into 1.13 at this point. Sorry.

odeke-em · 2019-08-02T03:08:15Z

Got it! We can wait until Go.1.14 then :)

….Path Updates #30165 Updates #31622 Change-Id: I7a4b91aa7c5c3af8c0b1273cbb42046feddf7d78 Reviewed-on: https://go-review.googlesource.com/c/go/+/180499 Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>

….Path Updates golang#30165 Updates golang#31622 Change-Id: I7a4b91aa7c5c3af8c0b1273cbb42046feddf7d78 Reviewed-on: https://go-review.googlesource.com/c/go/+/180499 Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>

agnivade added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 11, 2019

gopherbot closed this as completed in ea65d01 Feb 26, 2019

twmb mentioned this issue Apr 23, 2019

net/http: StripPrefix regression from fix for #30165 #31622

Closed

dmitshur reopened this Jun 4, 2019

dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jun 4, 2019

dmitshur self-assigned this Jun 4, 2019

dmitshur added this to the Go1.13 milestone Jun 4, 2019

dmitshur changed the title ~~net/http: StripPrefix makes the stripped URL having an empty root path rather than "/"~~ net/http: ServeFile panics when StripPrefix over-strips and results in empty path Jun 5, 2019

andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019

rsc modified the milestones: Go1.14, Backlog Oct 9, 2019

rsc unassigned dmitshur Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net/http: ServeFile panics when StripPrefix over-strips and results in empty path #30165

net/http: ServeFile panics when StripPrefix over-strips and results in empty path #30165

ggicci commented Feb 11, 2019 •

edited

gopherbot commented Feb 11, 2019

gopherbot commented Jun 4, 2019

dmitshur commented Jun 4, 2019

gopherbot commented Jun 4, 2019

dmitshur commented Jun 4, 2019

odeke-em commented Jul 31, 2019

andybons commented Jul 31, 2019

odeke-em commented Aug 2, 2019

net/http: ServeFile panics when StripPrefix over-strips and results in empty path #30165

net/http: ServeFile panics when StripPrefix over-strips and results in empty path #30165

Comments

ggicci commented Feb 11, 2019 • edited

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

gopherbot commented Feb 11, 2019

gopherbot commented Jun 4, 2019

dmitshur commented Jun 4, 2019

gopherbot commented Jun 4, 2019

dmitshur commented Jun 4, 2019

odeke-em commented Jul 31, 2019

andybons commented Jul 31, 2019

odeke-em commented Aug 2, 2019

ggicci commented Feb 11, 2019 •

edited

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?