Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: add a new flag to disable unzipping after go mod download #32605

Open
aofei opened this issue Jun 13, 2019 · 2 comments
Open

cmd/go: add a new flag to disable unzipping after go mod download #32605

aofei opened this issue Jun 13, 2019 · 2 comments
Labels
FeatureRequest GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@aofei
Copy link
Contributor

aofei commented Jun 13, 2019

The go mod download caches the module files into the $GOPATH/pkg/mod/cache/download by default and extracts them into the $GOPATH/pkg/mod after the downloads are complete. This's fine. But today I encountered a problem that reminds me that we may need to disable the unzipping operation at some point.


So, I ran a program similar to the following using a non-root user:

package main

import (
	"io/ioutil"
	"log"
	"os"
	"os/exec"
)

func main() {
	tempDir, err := ioutil.TempDir("", "foobar")
	if err != nil {
		log.Fatalf("failed to create temp dir: %v", err)
	}

	cmd := exec.Command("go", "mod", "download", "-json", "golang.org/x/text@latest")
	cmd.Env = append(os.Environ(), "GO111MODULE=on", "GOPATH="+tempDir)
	cmd.Dir = tempDir
	if err := cmd.Run(); err != nil {
		log.Fatalf("failed to download module: %v", err)
	}

	if err := os.RemoveAll(tempDir); err != nil {
		log.Fatalf("failed to remove temp dir: %v", err)
	}
}

Then I got the following failed output:

2019/06/14 01:03:21 failed to remove temp dir: unlinkat /var/folders/j6/2330_fdx4tn9t9tx4y117w700000gn/T/foobar128162500/pkg/mod/golang.org/x/text@v0.3.2/codereview.cfg: permission denied
exit status 1

Actually, I just need to download the module files and save them somewhere else (not in the $GOPATH/pkg/mod/cache/download), then clear the download history after the download is complete. But I got stuck in that unlinkat operation.

In my scenario, I don't actually need the go command to extract the zip files I downloaded into the $GOPATH/pkg/mod, but the go command doesn't ask me if it can do that. This not only caused unnecessary computing, but also causes problems such as the inability to remove the unzipped files if you don't have enough permissions.

So I think we should probably add a new flag like -disable-unzip to go mod download to disable the unzipping operation.

@FiloSottile FiloSottile added GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jun 14, 2019
@FiloSottile FiloSottile added this to the Unplanned milestone Jun 14, 2019
@FiloSottile
Copy link
Contributor

/cc @bcmills

aofei added a commit to goproxy/goproxy that referenced this issue Jul 29, 2019
@dmitshur
Copy link
Contributor

dmitshur commented Nov 5, 2019

There are a few alternative solutions to this problem that can be considered, in addition to a new flag.

  1. Before doing os.RemoveAll(tempDir), you can insert a call to exec.Command("go", "clean", "-modcache"). That'll make it succeed without permission errors.

  2. In Go 1.13, by default, the go command downloads from the module mirror at https://proxy.golang.org. If that behavior is satisfactory for you, you can bypass the go command and download the module zip file from the module mirror directly.

  3. In the x/mod repository, there is a new zip package. /cc @jayconrod It brings us closer to being able to make a stand-alone tool that could create a module zip in a safe way. Such a tool can be better suited for your custom needs than the go command.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FeatureRequest GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

4 participants