Overview

In certain (reproducible and predictable) situations, running go get -u will partially fail the first time, but succeed the second time. During the first run, it will succeed to update the repository, but fail to build while printing an erroneous error message about a missing dependency. During the second run, everything will succeed.

This happens when Go code is refactored in a way so that one Go package that was previously imported ceases to exist, but it is also no longer imported hence the Go code is valid. For example, if a package that was previously imported is merged into another, or if a package is renamed and now has a new import path.

This is a problem because it hinders ones ability to refactor Go code. We now have ever-more-powerful refactoring tools like gorename that are able to help with refactoring. In its todo list is the ability to rename import paths. That ability is already present within another tool govers today. Being able to seamlessly improve and refactor Go code is important to allow it to get better. No one can come up with the perfect version the first time around, so it's possible they'll want to refactor after using the package for some time and having a better vision. An implementation detail limitation of go get -u should not hinder that.

What does go version print?

go version go1.4rc2 darwin/amd64

What steps reproduce the problem?

In order to prepare a (minimal) environment where the problem can be reproduced (i.e., it happens when go get -u runs and there are updates available), you need the following preparation steps first:

1. Run go get -d -u github.com/shurcooL/go-get-test-cmd. It has only one external dependency repo github.com/shurcooL/go-get-test-lib which will also be fetched.
2. Cd into $GOPATH/src/github.com/shurcooL/go-get-test-lib folder.
3. Being careful to do this in the right folder (!), run git reset --hard master^ to hard reset go-get-test-lib repo to previous commit. This will make it so that when we run go get -u there will be one update available.
   • The go-get-test-lib repo has 2 commits. We need to pretend that at this time it only has 1 commit, and a new commit is made available later on.

That simulates a scenario where a user has an existing version of your Go package and its dependencies, and you decide to refactor one of the dependencies in this way:

https://github.com/shurcooL/go-get-test-lib/compare/master^...master

Now, you are ready to reproduce the problem, follow these steps:

1. Run go get -u github.com/shurcooL/go-get-test-cmd.
2. Run ls $GOPATH/bin/go-get-test-cmd and see that it failed to build.
3. Run go get -u github.com/shurcooL/go-get-test-cmd a second time.
4. Run ls $GOPATH/bin/go-get-test-cmd.

What happened?

$ go get -u github.com/shurcooL/go-get-test-cmd
package github.com/shurcooL/go-get-test-cmd
    imports github.com/shurcooL/go-get-test-lib/liba
    imports github.com/shurcooL/go-get-test-lib/libb
    imports github.com/shurcooL/go-get-test-lib/libb
    imports github.com/shurcooL/go-get-test-lib/libb: cannot find package "github.com/shurcooL/go-get-test-lib/libb" in any of:
    /usr/local/go/src/github.com/shurcooL/go-get-test-lib/libb (from $GOROOT)
    /Users/Dmitri/GoPath/src/github.com/shurcooL/go-get-test-lib/libb (from $GOPATH)
$ ls $GOPATH/bin/go-get-test-cmd
ls: /Users/Dmitri/GoPath/bin/go-get-test-cmd: No such file or directory
$ go get -u github.com/shurcooL/go-get-test-cmd
$ ls $GOPATH/bin/go-get-test-cmd
/Users/Dmitri/GoPath/bin/go-get-test-cmd
$ 

What should have happened instead?

Everything should've succeeded the on the first run of go get -u.

$ go get -u github.com/shurcooL/go-get-test-cmd
$ ls $GOPATH/bin/go-get-test-cmd
/Users/Dmitri/GoPath/bin/go-get-test-cmd
$ go get -u github.com/shurcooL/go-get-test-cmd
$ ls $GOPATH/bin/go-get-test-cmd
/Users/Dmitri/GoPath/bin/go-get-test-cmd
$ 

Please provide any additional information below.

This happens when Go code is refactored in a way so that one Go package that was previously imported ceases to exist, but it is also no longer imported hence the Go code is valid. However, when a user runs go get -u to update, it first parses the previous code to get a list of Go packages, then updates repos and does not re-parse the updated code (to find that a certain dependency is actually no longer imported) before returning an error that the previously imported package doesn't exist.

In the reproduce steps above, I used a refactor example where previously liba and libb were used, but libb is merged into liba. I would imagine the same issue would happen if libb were renamed to libc (and imported under new name) for example.

This likely affects /internal/ Go packages too. That means one can not use the argument that "you should not be removing old import path since someone might be importing it" for /internal/ Go packages as they're guaranteed not to be imported by anyone else, and you should be free to refactor them.

Workarounds:

• Tell users to always run go get -u twice and ignore (possible) problems on first run.
• Keep the old package around indefinitely.
• Keep old package around for some amount of time. However, even if you keep it for a month, there's no guarantee some user will not run go get -u after 2 months and encounter this problem, it just reduces that chance.
  • Alternatively, provide an empty package stub at the old location, for example see shurcooL/go@c53996f. This allows the issue to be avoided.