Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: go clean --modcache downloads modules before cleaning #28680

Closed
hyangah opened this issue Nov 8, 2018 · 7 comments
Closed

cmd/go: go clean --modcache downloads modules before cleaning #28680

hyangah opened this issue Nov 8, 2018 · 7 comments
Labels
early-in-cycle A change that should be done early in the 3 month dev cycle. FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.13

Comments

@hyangah
Copy link
Contributor

hyangah commented Nov 8, 2018 

$ cat go.mod
module hello

require rsc.io/quote v1.5.2
$ cat main.go
package main
import "rsc.io/quote"
func main() { println(quote.Hello()) }
$ go clean --modcache
go: finding rsc.io/quote v1.5.2
go: finding rsc.io/sampler v1.3.0
go: finding golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
go: downloading rsc.io/quote v1.5.2
go: downloading rsc.io/sampler v1.3.0
go: downloading golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
$
$ go version
go version devel +644ddaa842 Wed Nov 7 16:12:02 2018 +0000 linux/amd64

Maybe working as intended. But this command output doesn't mention anything about the cleaning operation but is chatty about module downloading. That is confusing, especially when the go.mod has errors so the messages end with error reports.

For example, the following is the message I get when I attempted to clean up the module cache with problematic go.mod (referencing non-existing versions). The module cache got empty, but the message is confusing.

$ go clean --modcache
go: finding github.com/hyangah/thing1 v0.0.2
go: github.com/hyangah/thing1@v0.0.2: unknown revision v0.0.2
go: error loading module requirements
@heschi heschi added the modules label Nov 8, 2018
@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Nov 9, 2018
@bcmills bcmills added this to the Go1.12 milestone Nov 9, 2018
@bcmills
Copy link
Contributor

bcmills commented Nov 9, 2018

We should definitely not download modules during go clean --modcache.

@hyangah
Copy link
Contributor Author

hyangah commented Nov 9, 2018

Why is this trying to analyze the go.mod and dependency? Can't it simply delete the cache directory? I wondered if it was trying to delete modules selectively (for unknown reasons), but it turned out not. The whole directory under my default GOPATH was gone after the command.

@bcmills
Copy link
Contributor

bcmills commented Nov 9, 2018

Why is this trying to analyze the go.mod and dependency?

I'm not sure. If we figure that out, I think that will point to the fix. 🙂

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 9, 2018
@gopherbot gopherbot removed the NeedsFix The path to resolution is known, but the work has not been done. label Nov 9, 2018
@bcmills bcmills mentioned this issue Nov 13, 2018
Closed
@sam-sla
Copy link

sam-sla commented Nov 19, 2018

I was having a similar problem with go clean -modcache trying to find a local package can't load package: package github.com/foobar: unknown import path "github.com/foobar": cannot find module providing package github.com/foobar. Setting GO111MODULE=off before executing go clean helped me, since go didn't try to find anything and just did the cleanup.

@jwilner jwilner mentioned this issue Nov 20, 2018
Merged
@bcmills bcmills self-assigned this Dec 5, 2018
@bcmills
Copy link
Contributor

bcmills commented Dec 12, 2018

This turned out to be really complicated.

go clean (without arguments) removes extraneous binaries from the current directory. Specifically, it removes the corresponding binary for each .go file in the directory that is not part of the package in that directory, in case it was built using go build ${file}.go.

In order to figure out which files are in the package, the clean command calls out to the loader, which does a bunch of extra work to resolve transitive imports and then comes back with the names of the files that are in the package. (clean -r applies transitively to the packages imported by the named packages, at the moment also ignoring module boundaries, so it has to traverse the whole import graph to figure out the set of affected packages. I guess we also do that without -r to keep the code simpler.)

The loader, in turn, computes the MVS build list, which means it needs to resolve all of the go.mod files for the transitive requirement graph — even if we're not actually cleaning any packages at all.

I think for a proper fix we'll need to rework the interaction between the clean package and the load and modload packages, or define clean to be much more restricted in module mode.

@bcmills bcmills modified the milestones: Go1.12, Go1.13 Dec 12, 2018
@gopherbot
Copy link

Change https://golang.org/cl/153818 mentions this issue: cmd/go: reproduce #28680

gopherbot pushed a commit that referenced this issue Dec 12, 2018
cmd/go: reproduce #28680
c4a8a68 
This change encodes the current behavior in mod_clean_cache.txt.
A fix for that behavior will probably have to wait for 1.13.

Updates #28680

Change-Id: I216b5a783971309cc75187502bddccc58c3a9c35
Reviewed-on: https://go-review.googlesource.com/c/153818
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@bcmills bcmills added the early-in-cycle A change that should be done early in the 3 month dev cycle. label Jan 18, 2019
@bcmills bcmills mentioned this issue Jan 24, 2019
Closed
@bcmills bcmills mentioned this issue Feb 28, 2019
Closed
@gopherbot
Copy link

Change https://golang.org/cl/167717 mentions this issue: cmd/go: skip package loading if explicitly cleaning a cache

@bcmills bcmills mentioned this issue May 29, 2019
Closed
@hyangah hyangah mentioned this issue Jul 8, 2019
Open
@adamwalach adamwalach mentioned this issue Aug 19, 2019
Merged
@golang golang locked and limited conversation to collaborators Mar 14, 2020
@rsc rsc unassigned bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
early-in-cycle A change that should be done early in the 3 month dev cycle. FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.13
Development

No branches or pull requests
5 participants
@hyangah @bcmills @gopherbot @sam-sla @heschi