New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
net/http: ServeFile may return 404 on temporary FS errors (such as »too many open files«) #46394
Comments
There are several questions here: Should What errors should be transient by default? Most interestingly, how should we communicate to the One possibility is to add a sentinel error to |
Since |
Drive-by security comment: The fact that When working on Go SafeWeb we had to write quite a bit of code to try and remove the current behavior that leaks information on existing files that are not accessible by the user who runs the server process.
If anything, I'd argue any error encountered by those functions should cause a 404 to be emitted. If users want to expose their server's internals they should have to write custom code themselves. I opened #46413 for reference. |
In my concrete case, As far as I'm aware, |
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (
go env
)?go env
Output(for completeness's sake, I first observed it on a linux server running a go1.14 binary)
What did you do?
Compile the following program into an executable (say
http-ulimit-404
by doinggo build -o http-ulimit-404 http-ulimit-404.go
).Set the number of open files very low using e. g.
ulimit -n 8
(I found that on Darwin, 8 works well to reproduce the issue. Note that on Darwin this only works as root user.)Request a file that exists in the working directory, if necessary, using multiple parallel instances of
curl
, e. g.curl http://localhost:8080/http-ulimit-404.go
. For me, using one singlecurl
instance withulimit -n 8
works already.Observe the resulting error to be 404.
What did you expect to see?
Package net/http returning a 5xx error indicating to clients that the request cannot be served due to a temporary condition (in this case number of open files), and may be retryable.
What did you see instead?
Package net/http returning a 404 error, indicating to clients that the request cannot be served due to a permanent condition, possibly causing cache posioning or other downstream failures.
(In the real world scenario where I found this, this caused a 404 on an extant file due to a high number of concurrent requests, leading to this file not being processed – and not retried – in a workload where it should've been.)
I believe this could easily be mitigated by checking if
originalErr
implementsinterface { Temporary() bool }
in mapDirOpenError (https://golang.org/src/net/http/fs.go#L48), but I'm not 100% sure yet if I'm overlooking something.The text was updated successfully, but these errors were encountered: