Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: handling range requests for empty files #47021

Open
MSevey opened this issue Jul 2, 2021 · 3 comments
Open

net/http: handling range requests for empty files #47021

MSevey opened this issue Jul 2, 2021 · 3 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@MSevey
Copy link

MSevey commented Jul 2, 2021

What version of Go are you using (go version)?

go version go1.16.3 darwin/amd64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

go env Output
$ go env

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/matt/Library/Caches/go-build"
GOENV="/Users/matt/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/matt/Code/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/matt/Code/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16.3"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/matt/Code/go/skyd/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/f9/2yp855dn19s0cqcjwbrv_0900000gn/T/go-build2190681930=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

In our API package we add a range request to the HEADER like so

	req, err := c.NewRequest("GET", resource, nil)
	if err != nil {
		return nil, errors.AddContext(err, "failed to construct GET request")
	}
	req.Header.Add("Range", fmt.Sprintf("bytes=%d-%d", from, to))
	httpClient := http.Client{CheckRedirect: c.CheckRedirect}
        res, err := httpClient.Do(req)

When we extended our testing to include zero-byte (ie empty) files we found that the response from the httpClient.Do method is 416 Requested Range Not Satisfiable. This was with a from and to values of 0.

Now I know that technically that is correct since that range request is asking for the first byte and that doesn't exist in an empty file. However, I could not find in the documentation a way to submit range requests for empty files.

Would it be possible for the standard library to be updated to provided direction on how to submit a range request for an empty file? Or do you expect the caller to make that determination and just not submit a range if it is an empty file.

What did you expect to see?

There should be a standard way to handle a range request for a zero-byte (empty) file.

What did you see instead?

416 Requested Range Not Satisfiable

@mdlayher mdlayher changed the title Range Requests for Empty Files net/http: handling range requests for empty files Jul 2, 2021
@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 2, 2021
@dmitshur dmitshur added this to the Backlog milestone Jul 2, 2021
@neild
Copy link
Contributor

neild commented Jul 6, 2021

What specific behavior do you want here?

From RFC 7433, Section 4.4:

The 416 (Range Not Satisfiable) status code indicates that none of the ranges in the request's Range header field (Section 3.1) overlap the current extent of the selected resource...

A request for "bytes=0-0" is a request for the first byte of a resource. If the resource is 0-length, the range [0,0] does not overlap the resource content and the server responds with a 416. That seems like a correct response.

RFC 7433, Section 4.4 does also state:

For byte ranges, failing to overlap the current extent means that the first-byte-pos of all of the byte-range-spec values were greater than the current length of the selected representation.

That could be read as stating that a request for [0,0] overlaps a 0-length representation, because the first-byte-pos (0) is not greater than the length (0). However, this interpretation would require responding with a 0-length 206 Partial Content response, which is not possible since the Content-Range header does not provide a way to encode a 0-length range. So I believe the interpretation of this sentence consistent with the rest of RFC 7433 is that a range fails to overlap if every first-byte-pos references a value past the end of the content.

@MSevey
Copy link
Author

MSevey commented Jul 8, 2021

What specific behavior do you want here?

@neild thanks for the response

I agree with what you said. Using range requests and handling 0 byte files seems like a pretty common case to handle, so my main question was if we could add documentation to the go library for how to handle that case or have a special case for this.

When I was looking into it I did find this project issue that talked about using a 1-0 range request to indicate a 0 byte file. SitePen/dstore#109 (comment)

If you don' t feel it is appropriate for the go library to handle the 0 byte file case then I think it would be good to add something to the documentation so that the next time someone is searching for how to handle 0 byte file range requests in golang they come across the docs that say they don't :-) and that each application to handle those requests.

@neild
Copy link
Contributor

neild commented Jul 8, 2021

This isn't really a Go issue. There's no way to issue an RFC 7433-conformant request for a range of a 0-length file. A range request is a request for a non-zero-length range of bytes. A 0-length file contains no bytes. I believe that a 416 Range Not Satisfiable response is correct in this situation. There are other correct responses (we could ignore the Range header entirely for 0-length files, for example, and return a 200 response), but I don't see a reason those other possibilities are better. I don't think there's anything surprising about responding to a request for data past the end of a file with, "Sorry, the file isn't that big".

A request containing bytes=1-0 is unambiguously invalid under RFC 7433:

A byte-range-spec is invalid if the last-byte-pos value is present and less than the first-byte-pos.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

3 participants