Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/url: url.RawQuery = url.Query().Encode() can change the URL #16460

Closed
tombergan opened this issue Jul 21, 2016 · 8 comments
Closed

net/url: url.RawQuery = url.Query().Encode() can change the URL #16460

tombergan opened this issue Jul 21, 2016 · 8 comments

Comments

@tombergan
Copy link
Contributor

In the following code:

u, _ := url.Parse("http://www.example.com/path?foo")
fmt.Println(u)
u.RawQuery = u.Query().Encode()
fmt.Println(u)

The third line actually changes the URL. The output is:

http://www.example.com/path?foo
http://www.example.com/path?foo=

A strict reading of RFC 3986 Section 6 suggests that these are two different URLs even after applying the (optional) syntax-based normalizations described in Section 6.2.2. I know of some servers that consider these URLs different. For example, compare the following links:

http://www.vcfed.org/forum/forumdisplay.php?23-DEC
http://www.vcfed.org/forum/forumdisplay.php?23-DEC=

I consider this a bug, but I doubt we could fix this bug in a backwards-compatible way. At least, I think we should update the documentation to make it clear that url.Query().Encode() can change query param "foo" to "foo=", which may be considered a different URL by some servers.

@bradfitz bradfitz added this to the Go1.8 milestone Jul 21, 2016
@bradfitz
Copy link
Contributor

Well, they do parse as different URLs:

https://play.golang.org/p/7xC_QxjGyI

But maybe the (*URL).Query() method could return different values?

@tombergan
Copy link
Contributor Author

tombergan commented Jul 21, 2016

Additionally, url.Query().Encode() will percent-encode query parameters that were not previously encoded, which some servers don't like. One example:

http://www.dayonepatch.com/index.php?/topic/135060-data-saver-and-d1p/
http://www.dayonepatch.com/index.php?%2Ftopic%2F135060-data-saver-and-d1p%2F=

This may be fixable by adding url.Value.RawEncode()? Or again, this may just be something to document more clearly.

@bradfitz
Copy link
Contributor

Maybe we should back up. What is the problem?

This bug is bordering on "Doctor, it hurts if I do this..."

That is, don't modify the RawQuery if the RawQuery was correct to begin with.

What is your use case?

@tombergan
Copy link
Contributor Author

The use case was adding/removing query params from a URL. We needed to do this without changing any of the existing query params. We're fixing this a different way, so I don't need a code change in the library, but I think the methods should document the behavior more clearly, particularly in the first example, which I think is surprising.

@bradfitz
Copy link
Contributor

Gotcha, thanks.

@quentinmit
Copy link
Contributor

I don't think there's anything to do here. url.Query() unfortunately only works for key=value query strings; isindex-style query strings have to be implemented using RawQuery.

@tombergan
Copy link
Contributor Author

I agree that a code change isn't needed, but I think the godoc could be clarified. URL.Query says that it "parses RawQuery" but doesn't say anything about formatting expectations. Something like:

// Query parses RawQuery and returns the corresponding values. It assumes
// RawQuery is a list of key=value pairs separated by '&' or ';'. If RawQuery
// has a different form, the return value is undefined.
func (u *URL) Query() Values

@gopherbot
Copy link

CL https://golang.org/cl/31331 mentions this issue.

@golang golang locked and limited conversation to collaborators Oct 18, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants