Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: GOPROXY using file:// protocol not working on Windows #27698

Closed
chowey opened this issue Sep 15, 2018 · 10 comments
Closed

cmd/go: GOPROXY using file:// protocol not working on Windows #27698

chowey opened this issue Sep 15, 2018 · 10 comments
Labels
FrozenDueToAge GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done. OS-Windows
Milestone

Comments

@chowey
Copy link

chowey commented Sep 15, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.11 windows/386

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

GOARCH=386
GOOS=windows

What did you do?

git clone https://github.com/pkg/errors c:/repos/github.com/pkg/errors
set GOPROXY=file:///c:/repos
echo package main;import _ "github.com/pkg/errors";func main(){}> main.go
go mod init test
go run .

What did you expect to see?

Successfully imported the package and ran the program. This is documented in cmd/go, where it says:

A Go module proxy is any web server that can respond to GET requests for URLs of a specified form. The requests have no query parameters, so even a site serving from a fixed file system (including a file:/// URL) can be a module proxy.

Also from the module wiki:

Run go help goproxy for more about the module proxy, including a pure file-based option via a file:/// URL.

What did you see instead?

build test: cannot find module for path github.com/pkg/errors

I may be doing something wrong, but the documentation seems to indicate file:/// is supported.

@dmitshur dmitshur added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go labels Sep 15, 2018
@dmitshur
Copy link
Contributor

/cc @rsc @bcmills

@magical
Copy link
Contributor

magical commented Sep 15, 2018

It looks like you're trying to point GOPROXY at a source tree. That's not how it works. GOPROXY is expected to point at a tree of .zip files and .mod files, not source files. I believe you should be able to point GOPROXY at your module cache, which is $GOPATH/pkg/mod/cache/download, though I haven't tried this myself.

This is all more or less explained in the documentation that you linked: https://golang.org/cmd/go/#hdr-Module_proxy_protocol.

@chowey
Copy link
Author

chowey commented Sep 15, 2018

Yes I missed that part:

Even when downloading directly from version control systems, the go command synthesizes explicit info, mod, and zip files and stores them in its local cache, $GOPATH/pkg/mod/cache/download, the same as if it had downloaded them directly from a proxy. The cache layout is the same as the proxy URL space, so serving $GOPATH/pkg/mod/cache/download at (or copying it to) https://example.com/proxy would let other users access those cached module versions with GOPROXY=https://example.com/proxy.

I am specifically trying to use go modules in an environment where there is no direct internet connection. I was hoping that it would be simple, but no such luck.

@chowey
Copy link
Author

chowey commented Sep 15, 2018

So I've done more testing. I set GOPATH to e.g. C:\gopath, cached the modules, and tried:

set GOPROXY=file:///C:/gopath/pkg/mod/cache/download
go run .

and that didn't work!

But, after accidentally making a typo, I discovered that THIS works:

set GOPROXY=file://C:/gopath/pkg/mod/cache/download
go run .

Note the two leading slashes instead of three.

Now per my recollection, and Wikipedia supports me here, I'd expect the former to work and the later not to work. So there still seems to be a bug in here somewhere.

@odeke-em odeke-em changed the title GOPROXY using file:// protocol not working on Windows x/vgo: GOPROXY using file:// protocol not working on Windows Sep 17, 2018
@gopherbot gopherbot added this to the vgo milestone Sep 17, 2018
@myitcv myitcv changed the title x/vgo: GOPROXY using file:// protocol not working on Windows cmd/go: GOPROXY using file:// protocol not working on Windows Sep 17, 2018
@myitcv myitcv modified the milestones: vgo, Go1.12 Sep 17, 2018
@bcmills
Copy link
Contributor

bcmills commented Sep 17, 2018

I am specifically trying to use go modules in an environment where there is no direct internet connection.

FWIW, go mod download is the intended command for that use-case: you should be able to run go mod download on a machine with a network connection, then copy GOPATH/pkg/mod/cache/download to your disconnected builder.

@bcmills
Copy link
Contributor

bcmills commented Sep 17, 2018

https://blogs.msdn.microsoft.com/ie/2006/12/06/file-uris-in-windows/ does seem to support the triple-slash as canonical.

Probably we need to do something more robust to translate the URI path to a file path here:

if strings.HasPrefix(url, "file:") {
body, err := ioutil.ReadFile(req.URL.Path)

(CC @alexbrainman, in case he has any advice on Go idioms to convert Windows URL paths to file paths.)

@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Sep 17, 2018
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 17, 2018
@bcmills
Copy link
Contributor

bcmills commented Sep 17, 2018

https://play.golang.org/p/5uOiwDwU0n4 for reference on the behavior of http.NewRequest.

@alexbrainman
Copy link
Member

https://blogs.msdn.microsoft.com/ie/2006/12/06/file-uris-in-windows/ does seem to support the triple-slash as canonical.

LGTM.

You could also look in net/url package - we already have some tests for urls that starts with file:///C:/.... Just remember that these paths are case insensitive on Windows (just like normal file paths on Windows).

Alex

@rsc
Copy link
Contributor

rsc commented Oct 24, 2018

OK, so the URL parser is putting the leading slash in, and on Windows we should take it out if the result passes filepath.IsAbs.

@gopherbot
Copy link

Change https://golang.org/cl/181037 mentions this issue: cmd/go/internal/web: support file:// URLs

@golang golang locked and limited conversation to collaborators Jun 9, 2020
@rsc rsc unassigned bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge GoCommand cmd/go NeedsFix The path to resolution is known, but the work has not been done. OS-Windows
Projects
None yet
Development

No branches or pull requests

8 participants