Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/url: export escaper with mode encodePath #36102

Closed
movsb opened this issue Dec 12, 2019 · 5 comments
Closed

net/url: export escaper with mode encodePath #36102

movsb opened this issue Dec 12, 2019 · 5 comments

Comments

@movsb
Copy link

movsb commented Dec 12, 2019

What version of Go are you using (go version)?

$ go version
go version go1.13.4 darwin/amd64

Does this issue reproduce with the latest release?

It does.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/tao.yang/Library/Caches/go-build"
GOENV="/Users/tao.yang/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/tao.yang/codes/gocodes"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.4/libexec"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/tao.yang/codes/gocodes/src/github.com/elastic/beats/filebeat/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/2l/zkndzzg90t746hcwzwb9pbz00000gn/T/go-build399229267=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Use url.PathEscape to escape file system path.

package main

import (
	"fmt"
	"net/url"
)

func main() {
	fmt.Println(url.PathEscape((`/a?/b`)))
}

Notice that ? is a valid path character on Unix-like systems.
Don't we treat it as a query that follows.

What did you expect to see?

/a%3F/b

What did you see instead?

%2Fa%3F%2Fb

Why?

PathEscape is actually the path segment escaper.

// PathEscape escapes the string so it can be safely placed inside a URL path segment,
// replacing special characters (including /) with %XX sequences as needed.
func PathEscape(s string) string {
	return escape(s, encodePathSegment)
}

What I need is a path escaper, with mode == encodePath, not encodePathSegment. Something like:

func RealPathEscape(s string) string {
	return escape(s, encodePath)
}

Workarounds

escape path segments individually

package main

import (
	"fmt"
	"net/url"
	"strings"
)

func main() {
	segments := strings.Split(`/a?/b`, "/")
	for i, segment := range segments {
		segments[i] = url.PathEscape(segment)
	}
	fmt.Println(strings.Join(segments, "/"))
}

use EscapedPath

package main

import (
	"fmt"
	"net/url"
)

func main() {
	u := url.URL{Path: "/a?/b"}
	fmt.Println(u.EscapedPath())
}
@toothrot
Copy link
Contributor

I'm not sure I understand. If ? is a valid character in a filesystem path name, /a%3F/b would be incorrect. What are you trying to escape, or how are you using this escaped value?

@movsb
Copy link
Author

movsb commented Dec 14, 2019

Because the RealPathEscape does what http.FileServer does:

go/src/net/http/fs.go

Lines 119 to 123 in 7d30af8

// name may contain '?' or '#', which must be escaped to remain
// part of the URL path, and not indicate the start of a query
// string or fragment.
url := url.URL{Path: name}
fmt.Fprintf(w, "<a href=\"%s\">%s</a>\n", url.String(), htmlReplacer.Replace(name))

Here, url.String() == url.EscapedPath().

For example, if we have a static file server:

package main

import "net/http"

func main() {
    http.Handle("/", http.FileServer(http.Dir(".")))
    http.ListenAndServe(":1234", nil)
}

Let's cURL it:

$ curl localhost:1234
<pre>
<a href="a%3F/">a?/</a>
<a href="server.go">server.go</a>
</pre>

Note that a?/ is escaped to a%3F/.


how are you using this escaped value?

While I'm manually rendering an HTML page.

@toothrot toothrot added this to the Backlog milestone Dec 16, 2019
@toothrot
Copy link
Contributor

/cc @bradfitz

@bradfitz
Copy link
Contributor

bradfitz commented Jan 7, 2020

The URL.EscapedPath method already does what you want:

https://play.golang.org/p/cOFqKqyM88y

fmt.Println((&url.URL{Path: "/This is/a path/?foo=bar"}).EscapedPath())
/This%20is/a%20path/%3Ffoo=bar

(The implementation of EscapedPath is basically just return escape(u.Path, encodePath))

Given what a fringe use case this is, I don't think we want to add a top-level func for this.

@bradfitz bradfitz closed this as completed Jan 7, 2020
@movsb
Copy link
Author

movsb commented Jan 8, 2020

@bradfitz So, you are saying that PathEscape()EscapedPath(). That's really confusing.

@golang golang locked and limited conversation to collaborators Jan 7, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants