Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/url: ParseQuery splits key/value pairs using old standard #23447

Closed
andygarfield opened this issue Jan 15, 2018 · 11 comments
Closed

net/url: ParseQuery splits key/value pairs using old standard #23447

andygarfield opened this issue Jan 15, 2018 · 11 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@andygarfield
Copy link

andygarfield commented Jan 15, 2018

What version of Go are you using (go version)?

1.9.2

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

darwin/amd64

What did you do?

https://play.golang.org/p/DjUWkAay5Ds

What did you expect to see?

[value separated by; a semicolon]

What did you see instead?

[value separated by]

The HTML4 appendix recommends using semicolons (;) in addition to ampersands (&) to separate URL key/value pairs. However, the current standard describes only ampersands as a separator.

Pull request incoming.

@gopherbot
Copy link

Change https://golang.org/cl/87755 mentions this issue: Fixes #23447

@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 15, 2018
@ianlancetaylor ianlancetaylor added this to the Go1.11 milestone Jan 15, 2018
@bradfitz
Copy link
Contributor

I don't think we can change behavior at this point. It's been like this for ages. We even document the current behavior, and have a bunch of tests for it.

What problem is the current behavior causing?

@andygarfield
Copy link
Author

I've just run into it while submitting forms. Go will parse all of my input until it hits a semicolon, and then the value gets cut off. That's unexpected, and certainly caused me some confusion until I dug deeper. It also seems that most people only use the ampersand.

I did also rewrite the tests so that they would serve the same function, yet without expecting a semicolon to separate the key/value pairs.

@bradfitz
Copy link
Contributor

bradfitz commented Jan 16, 2018

Which browser? Got example HTML for the <form>?

@andygarfield
Copy link
Author

andygarfield commented Jan 16, 2018

I'm using Chrome. But I'm not using <form> actually. I believe I understand what you are getting at there. Submitting a form would encode the semicolon as %3B and would work fine. I'm doing a simple AJAX process instead. For example:

<!DOCTYPE html>
<html>

<head>
  <script>
    function submit() {
      var textbody = document.getElementById("textbody").value;

      var xhttp = new XMLHttpRequest();
      xhttp.open("POST", "/submit");
      xhttp.onload = function () {
        console.log(xhttp.response);
      }
      xhttp.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");
      xhttp.send("textbody=" + textbody);
    }
  </script>
</head>

<body>
  <textarea id="textbody"></textarea>
  <button onclick="submit()">Submit</button>
</body>

</html>
package main

import (
	"fmt"
	"net/http"
)

func main() {
	http.Handle("/", http.FileServer(http.Dir(".")))
	http.HandleFunc("/submit", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, string(r.FormValue("textbody")))
	})
	http.ListenAndServe(":8080", nil)
}

I would need to change the javascript to something like this to ensure the value wouldn't be separated with a semicolon:

var textbody = document.getElementById("textbody").value
  .split(";")
  .join("%3B")

@vdobler
Copy link
Contributor

vdobler commented Jan 16, 2018

I would need to change the javascript

Yes, because that would be the valid encoding for a query parameter.

@andygarfield
Copy link
Author

Yes, if we're being eminently correct, the first example should be: https://play.golang.org/p/eGcmGksZk0q

My point is more to talk about how the semicolon separator is a basically deprecated standard. Though I understand needing to maintain backward compatibility if the semicolon was commonly used.

@andygarfield
Copy link
Author

I'm finding that encodeURI() is the best way to go.

var textbody = encodeURI(document.getElementById("textbody").value)

@magical
Copy link
Contributor

magical commented Jan 16, 2018

@andygarfield You probably want encodeURIComponent, which will encode characters like & and = as well.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent#Description

@bradfitz
Copy link
Contributor

Sounds like there's nothing to do here then. You figured out how to escape things in JavaScript, and we can't change Go and break other people depending on our documented behavior.

@andygarfield
Copy link
Author

I appreciate you taking a look @bradfitz.

@golang golang locked and limited conversation to collaborators Jan 17, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

6 participants