Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: the go command converts uppercase characters to !+ lowercase characters. #26456

Closed
saibing opened this issue Jul 19, 2018 · 4 comments
Closed

cmd/go: the go command converts uppercase characters to !+ lowercase characters. #26456

saibing opened this issue Jul 19, 2018 · 4 comments
Labels
FrozenDueToAge modules

Comments

@saibing
Copy link

saibing commented Jul 19, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version devel +c54bc34 Thu Jul 19 00:23:07 2018 +0000 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

baas@SZX1000428520:~/git/go/src/go-langserver [master]$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/baas/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/baas/git/go"
GOPROXY="http://10.75.203.36:9090"
GORACE=""
GOROOT="/home/baas/my/go"
GOTMPDIR=""
GOTOOLDIR="/home/baas/my/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/baas/git/go/src/go-langserver/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build071732067=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I am using the go 1.11 moudle function, I developed a go module proxy and used it by setting the environment variable GOPROXY

What did you expect to see?

the go module proxy can work well.

What did you see instead?

  • client
$ go get github.com/SimpleMessageNotification/smn-sdk-go@latest
go get github.com/SimpleMessageNotification/smn-sdk-go@latest: cannot find module providing package github.com/SimpleMessageNotification/smn-sdk-go
  • server
$ ./vgoproxy
0719 10:48:40.33 go config:
{
   "gopath": "/data1/baas/.gomod",
   "http": [
      "code.huawei.com",
      "rnd-isource.huawei.com",
      "rnd-github.huawei.com"
   ],
   "replace": {},
   "sortKeys": null
}
0719 10:48:40.33 start go proxy server at :9090
0719 10:49:23.565 GET /github.com/!simple!message!notification/smn-sdk-go/@v/list from 100.101.229.133:47174
0719 10:49:23.565 new url /github.com/!simple!message!notification/smn-sdk-go/@v/list
0719 10:49:23.566 go: list version failed: invalid github.com/ import path "github.com/!simple!message!notification/smn-sdk-go"
0719 10:49:23.58 GET /github.com/!simple!message!notification/@v/list from 100.101.229.133:47174
0719 10:49:23.58 new url /github.com/!simple!message!notification/@v/list
0719 10:49:23.58 go: list version failed: invalid github.com/ import path "github.com/!simple!message!notification"
0719 10:49:23.591 GET /github.com/@v/list from 100.101.229.133:47174
0719 10:49:23.591 new url /github.com/@v/list
0719 10:49:24.718 go: list version failed: unrecognized import path "github.com" (parse https://github.com?go-get=1: no go-import meta tags ())
@magical
Copy link
Contributor

magical commented Jul 19, 2018 •
edited

This appears to be working as intended:

go/src/cmd/go/internal/module/module.go

Lines 359 to 417 in c54bc34

// Safe encodings
//
// Module paths appear as substrings of file system paths
// (in the download cache) and of web server URLs in the proxy protocol.
// In general we cannot rely on file systems to be case-sensitive,
// nor can we rely on web servers, since they read from file systems.
// That is, we cannot rely on the file system to keep rsc.io/QUOTE
// and rsc.io/quote separate. Windows and macOS don't.
// Instead, we must never require two different casings of a file path.
// Because we want the download cache to match the proxy protocol,
// and because we want the proxy protocol to be possible to serve
// from a tree of static files (which might be stored on a case-insensitive
// file system), the proxy protocol must never require two different casings
// of a URL path either.
//
// One possibility would be to make the safe encoding be the lowercase
// hexadecimal encoding of the actual path bytes. This would avoid ever
// needing different casings of a file path, but it would be fairly illegible
// to most programmers when those paths appeared in the file system
// (including in file paths in compiler errors and stack traces)
// in web server logs, and so on. Instead, we want a safe encoding that
// leaves most paths unaltered.
//
// The safe encoding is this:
// replace every uppercase letter with an exclamation mark
// followed by the letter's lowercase equivalent.
//
// For example,
// github.com/Azure/azure-sdk-for-go -> github.com/!azure/azure-sdk-for-go.
// github.com/GoogleCloudPlatform/cloudsql-proxy -> github.com/!google!cloud!platform/cloudsql-proxy
// github.com/Sirupsen/logrus -> github.com/!sirupsen/logrus.
//
// Import paths that avoid upper-case letters are left unchanged.
// Note that because import paths are ASCII-only and avoid various
// problematic punctuation (like : < and >), the safe encoding is also ASCII-only
// and avoids the same problematic punctuation.
//
// Import paths have never allowed exclamation marks, so there is no
// need to define how to encode a literal !.
//
// Although paths are disallowed from using Unicode (see pathOK above),
// the eventual plan is to allow Unicode letters as well, to assume that
// file systems and URLs are Unicode-safe (storing UTF-8), and apply
// the !-for-uppercase convention. Note however that not all runes that
// are different but case-fold equivalent are an upper/lower pair.
// For example, U+004B ('K'), U+006B ('k'), and U+212A ('K' for Kelvin)
// are considered to case-fold to each other. When we do add Unicode
// letters, we must not assume that upper/lower are the only case-equivalent pairs.
// Perhaps the Kelvin symbol would be disallowed entirely, for example.
// Or perhaps it would encode as "!!k", or perhaps as "(212A)".
//
// Also, it would be nice to allow Unicode marks as well as letters,
// but marks include combining marks, and then we must deal not
// only with case folding but also normalization: both U+00E9 ('é')
// and U+0065 U+0301 ('e' followed by combining acute accent)
// look the same on the page and are treated by some file systems
// as the same path. If we do allow Unicode marks in paths, there
// must be some kind of normalization to allow only one canonical
// encoding of any character used in an import path.

@oiooj
Copy link
Member

oiooj commented Jul 19, 2018

As @magical said, this appears to be working as intended. You need to decode the path before you use it.

go/src/cmd/go/internal/module/module.go

Line 455 in c54bc34

func DecodePath(encoding string) (path string, err error) {

@oiooj oiooj closed this as completed Jul 19, 2018
@oiooj oiooj added the modules label Jul 19, 2018
@saibing
Copy link
Author

saibing commented Jul 19, 2018

@magical @magical

Thank you very much.

github.com/golang/vgo does not sync the latest code

@magical
Copy link
Contributor

magical commented Jul 19, 2018

github.com/golang/vgo does not sync the latest code

Correct. Per Russ's recent email to golang-dev,

Work on the module support will now be done exclusively in the main repo. We will automatically export the main repo to the x/vgo repo when we have a snapshot we want to make available to people using "vgo".

https://groups.google.com/forum/#!topic/golang-dev/a5PqQuBljF4

@seeruk seeruk mentioned this issue Jun 28, 2019
Closed
@golang golang locked and limited conversation to collaborators Jul 19, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@magical @oiooj @saibing @gopherbot