Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: module fetch errors are unclear #26594

Closed
rogpeppe opened this issue Jul 25, 2018 · 17 comments
Closed

cmd/go: module fetch errors are unclear #26594

rogpeppe opened this issue Jul 25, 2018 · 17 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@rogpeppe
Copy link
Contributor

What version of Go are you using (go version)?

go version devel +5f5402b Tue Jul 24 23:54:08 2018 +0000 linux/amd64

What did you do?

go mod init -module example.com/mod
go get github.com/google/go-cloud

This produced the following output:

go: finding github.com/google/go-cloud v0.1.0
go: finding github.com/tidwall/gjson v1.1.2
go: finding github.com/aws/aws-xray-sdk-go v1.0.0-rc.5
go: finding github.com/fsnotify/fsnotify v1.4.7
go: finding github.com/gorilla/context v1.1.1
go: finding github.com/dnaeon/go-vcr v0.0.0-20180504081357-f8a7e8b9c630
go: finding github.com/aws/aws-sdk-go v1.13.20
go: finding go.opencensus.io v0.12.0
go: finding google.golang.org/grpc v1.13.0
go: finding contrib.go.opencensus.io/exporter/stackdriver v0.0.0-20180421005815-665cf5131b71
go: finding cloud.google.com/go v0.25.0
go: finding github.com/tidwall/sjson v1.0.0
go: finding github.com/smartystreets/assertions v0.0.0-20180301161246-7678a5452ebe
go: finding github.com/googleapis/gax-go v1.0.0
go: finding golang.org/x/sys v0.0.0-20180329131831-378d26f46672
go: finding github.com/gopherjs/gopherjs v0.0.0-20180424202546-8dffc02ea1cb
go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: finding golang.org/x/text v0.3.0
go: finding golang.org/x/net v0.0.0-20180702212446-ed29d75add3d
go: finding gopkg.in/ini.v1 v1.37.0
go: finding github.com/go-ini/ini v1.37.0
go: finding github.com/golang/protobuf v1.1.0
go: finding github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af
go: finding github.com/census-ecosystem/opencensus-go-exporter-aws v0.0.0-20180411051634-41633bc1ff6b
go: finding golang.org/x/tools v0.0.0-20180314180217-d853e8088c62
go: finding github.com/go-sql-driver/mysql v0.0.0-20180308100310-1a676ac6e4dc
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: github.com/jmespath/go-jmespath@v0.0.0-20180206201540-c2b33e8439af: git fetch -f --depth=1 origin c2b33e8439af944379acbdd9c3a5fe0bc44bd8a5:refs/dummy in /home/rog/src/go/src/mod/cache/vcs/7b1106ecb177564b0bc9784f963c6c785e31d09dcd9f08114684d32af620443f: exit status 1
go: finding golang.org/x/oauth2 v0.0.0-20180603041954-1e0a3fa8ba9a
go: github.com/census-ecosystem/opencensus-go-exporter-aws@v0.0.0-20180411051634-41633bc1ff6b: git fetch -f --depth=1 origin 41633bc1ff6bd23f2beb03aafb533465f3e774d8:refs/dummy in /home/rog/src/go/src/mod/cache/vcs/28c9bbc7175856813846e87fcbc6da1a390ca3679ea2b8f3e8aa0698f2334be7: exit status 1
go: finding github.com/jtolds/gls v0.0.0-20170503224851-77f18212c9c7
go: finding gopkg.in/yaml.v2 v2.2.1
go: github.com/golang/glog@v0.0.0-20160126235308-23def4e6c14b: git fetch -f --depth=1 origin 23def4e6c14b4da8ac2ed8007337bc5eb5007998:refs/dummy in /home/rog/src/go/src/mod/cache/vcs/0dac7c3b21e56acaa4cfab3e2c4d83e362d33743d1e198c0d546115da37bae02: exit status 1
go: finding github.com/smartystreets/gunit v0.0.0-20180314194857-6f0d6275bdcd
go: finding google.golang.org/genproto v0.0.0-20180627194029-ff3583edef7d
go: finding github.com/stretchr/testify v1.2.1
go: finding google.golang.org/api v0.0.0-20180606215403-8e9de5a6de6d
go: github.com/smartystreets/gunit@v0.0.0-20180314194857-6f0d6275bdcd: git fetch -f --depth=1 origin 6f0d6275bdcdd4411ad1c9b3b69eda55eea5b2db:refs/dummy in /home/rog/src/go/src/mod/cache/vcs/8b17522217382c37f84409229f7711408129a6c273254893129c5df1940e85cc: exit status 1
go: finding github.com/gorilla/mux v1.6.1
go: finding github.com/tidwall/match v0.0.0-20171002075945-1731857f09b1
go: github.com/tidwall/match@v0.0.0-20171002075945-1731857f09b1: git fetch -f --depth=1 https://github.com/tidwall/match 1731857f09b1f38450e2c12409748407822dc6be:refs/dummy in /home/rog/src/go/src/mod/cache/vcs/94ba56ba5b973670f7952b0a5b76feea7af42eb6e1f973fdb61999862b84c7a7: exit status 1
go: finding google.golang.org/appengine v1.1.0
go: finding github.com/smartystreets/goconvey v0.0.0-20180222194500-ef6db91d284a
go: finding github.com/GoogleCloudPlatform/cloudsql-proxy v0.0.0-20180321230639-1e456b1c68cb
go: github.com/smartystreets/goconvey@v0.0.0-20180222194500-ef6db91d284a: git fetch -f --depth=1 origin ef6db91d284a0e7badaa1f0c404c30aa7dee3aed:refs/dummy in /home/rog/src/go/src/mod/cache/vcs/1693d9c59c0c1838de53cecb8601eec3084f084265b72c82e78de941f1c8f58c: exit status 1

go: error loading module requirements

It's clear that some of the git fetch commands failed, but because all the actual error output is discarded, it's not possible to see why. "exit status 1" doesn't convey any significantly useful info, so it's hard to know what went wrong and hence how the problem might be resolved.

@mvdan
Copy link
Member

mvdan commented Jul 25, 2018

If you're on an older versison of Git, this might be #26501. But I agree that the errors should be clearer.

@thepudds
Copy link
Contributor

#26501 is now closed, but adding some suggestions from that issue that could help future troubleshooting:

Consider extending what gets emitted by -v (#26501 (comment) by @rudis)

I stumbled over the same issue and I found the fact that go doesn't report anything about a failing command troublesome. I needed to use strace to figure out what's going on because even with -v nothing was printed about the failing command.

If possible please also extend -v's output to include for example failing commands to help debugging such issues.

And/or address TODO from modload (#26501 (comment) from @rogpeppe):

I encountered this issue today (I'm using a stock version of Ubuntu 16.04). It's made harder to diagnose by the fact that this TODO from src/cmd/go/internal/modload.go remains unaddressed:

// TODO(rsc): It would be nice to return a specific error encountered
// during the loop above if possible, but it's not clear how to pick
// out the right one.
This means that we don't get to see the actual error encountered (unknown revision refs/tags/v1.0.1) in my case. Even that error isn't ideal when the actual error printed by git is fatal: unrecognised argument: --no-show-signature. Perhaps that output could be shown when using the -v flag?

My git version is 2.7.4.

@rogpeppe
Copy link
Contributor Author

I just tried one of those git commands and as it happens it does return exit code 1 without printing any error, so perhaps the problem is with git itself. I am using git 2.7.4.

@rogpeppe
Copy link
Contributor Author

I could move off git 2.7.4 but as it's the default version you get when using Ubuntu 16.04 I think it's important that Go works correctly with that version.

@rogpeppe
Copy link
Contributor Author

BTW I just tried with a newer version of git (2.18.0) and everything works OK.

g-k pushed a commit to mozilla-services/autograph-edge that referenced this issue Jul 25, 2018
g-k pushed a commit to mozilla-services/autograph-edge that referenced this issue Jul 25, 2018
@rsc
Copy link
Contributor

rsc commented Jul 25, 2018

I'm not sure what more to do here. I would like to turn off the finding prints by default, although I worry that people will think the go command is just stuck if it's not printing those. But this is very precise:

go: github.com/jmespath/go-jmespath@v0.0.0-20180206201540-c2b33e8439af: git fetch -f --depth=1 origin c2b33e8439af944379acbdd9c3a5fe0bc44bd8a5:refs/dummy in /home/rog/src/go/src/mod/cache/vcs/7b1106ecb177564b0bc9784f963c6c785e31d09dcd9f08114684d32af620443f: exit status 1

It contains enough information to repeat the command, and as you found, something is wrong with that git command. I suppose we could make it say "cd dir && git fetch" instead of "git fetch ... in dir". What's unclear is git here, I think.

Does anyone have any idea what is wrong with git 2.7.4? Does it not support --depth=1?

@thepudds
Copy link
Contributor

This is not conclusive, but --depth=<depth> at least shows up in git's Documentation/fetch-options.txt for 2.7.4:

https://github.com/git/git/blob/v2.7.4/Documentation/fetch-options.txt#L10

--depth=<depth>::
	Limit fetching to the specified number of commits from the tip of
	each remote branch history. If fetching to a 'shallow' repository
	created by `git clone` with `--depth=<depth>` option (see
	linkgit:git-clone[1]), deepen or shorten the history to the specified
	number of commits. Tags for the deepened commits are not fetched.

@trashhalo
Copy link

trashhalo commented Jul 26, 2018

for what its worth git 2.7.4 and go head works fine for me.

➜  httpstat git:(master) ✗ go mod -init -module github.com/davecheney/httpstat
go: creating new go.mod: module github.com/davecheney/httpstat
go: copying requirements from Gopkg.lock
➜  httpstat git:(master) ✗ go build
go: finding github.com/fatih/color v1.5.0
go: finding github.com/mattn/go-isatty v0.0.3
go: finding github.com/mattn/go-colorable v0.0.9
go: finding golang.org/x/text v0.0.0-20170915090833-1cbadb444a80
go: finding golang.org/x/sys v0.0.0-20170922123423-429f518978ab
go: finding golang.org/x/net v0.0.0-20170922011244-0744d001aa84
go: downloading github.com/fatih/color v1.5.0
go: downloading golang.org/x/net v0.0.0-20170922011244-0744d001aa84
go: downloading github.com/mattn/go-colorable v0.0.9
go: downloading github.com/mattn/go-isatty v0.0.3
go: downloading golang.org/x/text v0.0.0-20170915090833-1cbadb444a80
➜  httpstat git:(master) ✗ git --version
git version 2.7.4
➜  httpstat git:(master) ✗ go version
go version devel +e5b1340 Wed Jul 25 23:53:54 2018 +0000 linux/amd64

@trashhalo
Copy link

@rogpeppe can you try running one of the git commands with some of these verbose env vars set

https://stackoverflow.com/questions/6178401/how-can-i-debug-git-git-shell-related-problems

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 26, 2018
@bcmills bcmills added this to the Go1.11 milestone Jul 26, 2018
@thepudds
Copy link
Contributor

@gopherbot, please add label help wanted

@myitcv
Copy link
Member

myitcv commented Aug 7, 2018

#25982 (comment) is related here. Whilst as @rsc notes in #26594 (comment) we shouldn't/don't have to log the finding statements, we can log the output in case of error.

@thepudds
Copy link
Contributor

thepudds commented Aug 7, 2018

This issue #26594 is slightly complicated by the fact that apparently here outputting errors from git would not have helped here (because here git was silent), but it seems outputting errors from git would have helped other issues, e.g., see #26594 (comment)

@trashhalo
Copy link

@thepudds is there any thoughts on when running go in verbose mode running git fetch with one of those verbose flags mentioned in my link? Just thinking about how we can give people the tools they need to debug these issues without filing an issue every time.

https://stackoverflow.com/questions/6178401/how-can-i-debug-git-git-shell-related-problems

@myitcv
Copy link
Member

myitcv commented Aug 8, 2018

Ok, unless I've missed this above, there appears to be something orthogonal to the git version that is affecting things here, above and beyond any differences in behaviour introduced by the git version itself.

Consider the following as context, which is roughly the state one of my CI builds get into via various Go commands:

cd $(mktemp -d)
git init
git remote add origin https://github.com/neelance/astrewrite

The build was failing because, ultimately, the following git (2.11.0) command failed:

git fetch -f --depth=1 origin 99348263ae862cc230986ce88deaddbf7edcc034:refs/dummy

It fails with exit code 1 and no output to stdout or stderr. A non-zero exit code is fine, but the empty stdout and stderr is not expected (

if !strings.Contains(err.Error(), "unadvertised object") && !strings.Contains(err.Error(), "no such remote ref") && !strings.Contains(err.Error(), "does not support shallow") {
) and hence the Go command ultimately fails.

If I try and reproduce this locally (using the same docker container), I instead get exit code 128 and some error output, specifically the expected output:

docker run -t -i --rm circleci/golang:1.10 bash
cd $(mktemp -d)
git init
git remote add origin https://github.com/neelance/astrewrite
git fetch -f --depth=1 origin 99348263ae862cc230986ce88deaddbf7edcc034:refs/dummy

gives

error: no such remote ref 99348263ae862cc230986ce88deaddbf7edcc034

As far as I can tell, this is exactly the same setup as my CI build; same docker image, git version (2.11.0) etc.

But something is clearly different (I'm probably missing something obvious); I just unable to recreate the environment/scenario in which that git command gives an exit code of 1.

I also can't reproduce the exit code 1 using git 2.7.4:

docker run --rm -i -t ubuntu:16.04 bash
apt-get -y -qq update < /dev/null > /dev/null
apt-get -y -qq install git < /dev/null > /dev/null
cd $(mktemp -d)
git init
git remote add origin https://github.com/neelance/astrewrite
git fetch -f --depth=1 origin 99348263ae862cc230986ce88deaddbf7edcc034:refs/dummy

However, @rogpeppe, with a standard Ubuntu 16.04 setup (git 2.7.4) can reproduce the exit code 1.

At this stage, we're unclear what is different between our environments/setups that causes this difference in behaviour.

Any help/thoughts much appreciated (and apologies if any of this repeats what's gone before, to be honest I'm totally lost at what's going on here).

@myitcv
Copy link
Member

myitcv commented Aug 8, 2018

@mvdan has the answer 🎖

This particular case appears to be affected by the use of an ssh-based git remote.

Using git 2.11.0 (because this particular command is not a problem with more recent git versions) and assuming an appropriate ssh setup:

cd $(mktemp -d)
git init
git remote add origin git@github.com:neelance/astrewrite
git fetch -f --depth=1 origin 99348263ae862cc230986ce88deaddbf7edcc034:refs/dummy

gives exit code 1 and no output.

Using an https-based remote:

cd $(mktemp -d)
git init
git remote add origin https://github.com/neelance/astrewrite
git fetch -f --depth=1 origin 99348263ae862cc230986ce88deaddbf7edcc034:refs/dummy

gives exit code 128 and the following output (which is what we want/need):

error: no such remote ref 99348263ae862cc230986ce88deaddbf7edcc034

I haven't looked into possible solutions, so this is just to note findings so far.

@myitcv
Copy link
Member

myitcv commented Aug 8, 2018

Sorry, another message in this thread

I'm seeing the aforementioned behaviour on CircleCI v2 (which has git 2.11.0), so not an uncommon setup people will be using. CircleCI globally sets url.ssh://git@github.com.insteadof=https://github.com, hence how this comes about.

@rsc
Copy link
Contributor

rsc commented Aug 17, 2018

We fixed this particular error, which was really git's fault.

@rsc rsc closed this as completed Aug 17, 2018
@rsc rsc added help and removed help wanted labels Aug 17, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

8 participants