Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go install cache on vendor packages not used #14323

Closed
Everlag opened this issue Feb 14, 2016 · 4 comments
Closed

cmd/go: go install cache on vendor packages not used #14323

Everlag opened this issue Feb 14, 2016 · 4 comments

Comments

@Everlag
Copy link

Everlag commented Feb 14, 2016

When vendoring a large package such as github.com/mattn/go-sqlite3, go install will generate the appropriate cache file in $GOPATH/pkg but this will not be used.

This is present under go1.6rc2 windows/amd64 as well as go1.5.3. The cache is used when GO15VENDOREXPERIMENT=0. I've checked this with go build -v. The environment is windows/amd64 under cygwin. I used govendor to vendor the go-sqlite3 package.

Compile time went from ~2s to ~36s for me.

@bradfitz bradfitz added this to the Go1.6Maybe milestone Feb 14, 2016
@rsc
Copy link
Contributor

rsc commented Feb 16, 2016

I tried on darwin/amd64 and windows/amd64 and was unable to reproduce this problem. Specifically, I created $GOPATH/src/buggy/p.go containing:

package main
import _ "github.com/mattn/go-sqlite3"
func main() {}

and then I ran:

cd $GOPATH/src
go get github.com/mattn/go-sqlite3
mkdir buggy/vendor
mv github.com buggy/vendor

and then I ran:

go install -v buggy
echo >>buggy/p.go
go install -v buggy

and the rebuild of buggy does not rebuild go-sqlite3. If I change p.go to say package buggy, then go install -v buggy still doesn't rebuild go-sqlite3.

I'd like to fix this but I can't without reproducing it. Can you link to the exact code you are using and show the exact commands you are running? Thanks.

@Everlag
Copy link
Author

Everlag commented Feb 16, 2016

With those steps, the issue doesn't appear. I modified them to reproduce the issue by changing

mv github.com buggy/vendor to cp -R github.com buggy/vendor
go install -v buggy to go install -v github.com/mattn/go-sqlite3

As the unvendored source is still in src/, it looks like the go install is installing that when explicitly specifying the package name. Removing the unvendored source causes a package not found error.

Some tools, at least govendor, don't remove the unvendored source when vendoring a package. Perhaps installing a package by name when both unvendored and vendored source are available should result in an error due to ambiguity?

@rsc
Copy link
Contributor

rsc commented Feb 16, 2016

Great, thanks. As I understand it now, the report concerns this command sequence:

go install -v github.com/mattn/go-sqlite3
go install -v buggy

Specifically, the report is that the first command installs the non-vendored go-sqlite3 and the second one builds and installs the vendored copy.

If so, that's working as intended. On the go command line, paths are always interpreted relative to GOROOT/GOPATH roots, not vendor directories. The canonical path for a vendored package includes the .../vendor/ prefix even though source code-level import references do not. That is, on the command line, 'github.com/mattn/go-sqlite3' always means the unvendored copy of go-sqlite3, while buggy/vendor/github.com/mattn/go-sqlite3 is a vendored copy. You'll see these expanded paths in the output of 'go list -json' or 'go list all' as well.

What would be unintended is if the second of any of these command pairs printed anything at all (the first should do all the work):

go install -v buggy
go install -v buggy
go install -v buggy/vendor/github.com/mattn/go-sqlite3
go install -v buggy/vendor/github.com/mattn/go-sqlite3
go install -v github.com/mattn/go-sqlite3
go install -v github.com/mattn/go-sqlite3

Can you confirm that on your system those are all correctly doing nothing the second time? Thanks.

@Everlag
Copy link
Author

Everlag commented Feb 16, 2016

Ah, I see what you mean. I expected package names to check the vendor directory first rather than prefixing them with buggy/vendor/. That makes sense.

The pairs you posted are working as they should.

Maybe when vendored packages have a naming collision with an unvendored package present in the same GOPATH, it would worth putting a warning when building to avoid this kind of confusion?

I'm closing this issue as it's solved. Thanks!

@Everlag Everlag closed this as completed Feb 16, 2016
@golang golang locked and limited conversation to collaborators Feb 28, 2017
@rsc rsc removed their assignment Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants