Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/vgo: generated go.mod does not use the latest version of a subdependency #26030

Closed
sneko opened this issue Jun 24, 2018 · 9 comments
Closed
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone

Comments

@sneko
Copy link

sneko commented Jun 24, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.10 windows/amd64 vgo:2018-02-20.1

Does this issue reproduce with the latest release?

Yes (this is the latest)

What did you do?

Running either "vgo mod -sync" or "vgo mod -vendor"

What did you expect to see?

Everything working and a filled go.mod file

What did you see instead?

During the resolving/finding/adding process I get an error for a library I have imported in my project (github.com/go-kit/kit/metrics/prometheus).

        import "github.com/go-kit/kit/metrics/prometheus" ->
        import "github.com/prometheus/client_golang/prometheus" ->
        import "github.com/prometheus/common/expfmt" ->
        import "github.com/matttproud/golang_protobuf_extensions/pbutil" ->
        import "github.com/golang/protobuf/proto/testdata" [XXXX\src\mod\github.com\golang\protobuf@v1.1.0\proto\testdata]: open XXXX\src\mod\github.com\golang\protobuf@v1.1.0\proto\testdata: Le fichier spécifié est introuvable.

(in english it means it doesn't find the testdata file)

While looking for a solution I found a similar issue:
matttproud/golang_protobuf_extensions#13

Indeed the released v1.0.1 of this plugin has fixed the wrong import but why vgo is downloading the v1.0.0 instead? I don't understand 😢

I checked the subdependencies from "github.com/go-kit/kit/metrics/prometheus" to "github.com/matttproud/golang_protobuf_extensions/pbutil" and no one has a "go.mod" file trying to force the v1.0.0 so what's happening hmmm?

It would be great if you could clarify this issue please.

Also, about UX: I found annoying that even with this error the process continue to analyze other dependencies. It results in loosing the error among a lot of:

...
vgo: resolving import "google.golang.org/genproto/googleapis/rpc/status"
vgo: finding google.golang.org/genproto latest
vgo: finding google.golang.org/genproto (latest)
vgo: adding google.golang.org/genproto v0.0.0-20180621235812-80063a038e33
vgo: resolving import "github.com/Shopify/sarama"
vgo: finding github.com/Shopify/sarama (latest)
...

I think the process should stop immediately after triggering the error, otherwise it should display a message at the end saying an error has occured.

Thank you 😃

EDIT: "Stupid note 😀 ": v1.0.0 and v1.0.1 tags target the same day "24 Apr 2016" (I don't know exactly how vgo sorts releases)

@gopherbot gopherbot added this to the vgo milestone Jun 24, 2018
@sneko
Copy link
Author

sneko commented Jun 26, 2018

Maybe I'm doing something wrong?

cc @bcmills @rsc

@bcmills
Copy link
Contributor

bcmills commented Jun 28, 2018

What did you do?

Running either "vgo mod -sync" or "vgo mod -vendor"

With what contents in your go.mod and working directory?

@bcmills bcmills changed the title x/vgo: It does not download the latest version of a subdependency x/vgo: generated go.mod does not use the latest version of a subdependency Jun 28, 2018
@bcmills bcmills added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jun 28, 2018
@sneko
Copy link
Author

sneko commented Jun 28, 2018

@bcmills the content of my go.mod file at the root of my Go project is:

module github.com/YYYY/service-skeleton

And that's all. I created this one by running the "vgo mod -init" without having any error from it.

@bcmills
Copy link
Contributor

bcmills commented Jun 28, 2018

What packages do your Go source files import? Do they have any build constraints?

Did you run any commands that would modify go.mod (such as vgo build) before vgo mod -sync?

@sneko
Copy link
Author

sneko commented Jun 28, 2018

Quite a lot, it's to make microservice and they are all relative to go-kit librairies except the ones of my project packages. Note that it was working well with the old way inside a "$GOPATH/src/...". Do you want me to list them all?

About build constraints I have none in my project. And I hope all the go-kit librairies don't have them either... Should I check all source code of these imported librairies?

I don't exactly remember the order of the commands I executed 4 days ago so I just retried by:

  1. Removing the "mod" directory containing all the cached librairies from vgo
  2. Removing the go.mod file inside my project
  3. Running "vgo mod -init"
  4. Running "vgo mod -sync" --> Results in just having "module github.com/YYYYY/service-skeleton" in my go.mod
  5. Running "vgo build-sync" --> Results in just having the following in my go.mod
module github.com/YYYYY/service-skeleton

require (
	github.com/Shopify/sarama v1.17.0
	github.com/apache/thrift v0.0.0-20180627210808-129f332d72fa
	github.com/eapache/go-resiliency v1.1.0
	github.com/eapache/go-xerial-snappy v0.0.0-20160609142408-bb955e01b934
	github.com/eapache/queue v1.1.0
	github.com/go-kit/kit v0.7.0
	github.com/gogo/protobuf v1.0.0
	github.com/golang/protobuf v1.1.0
	github.com/oklog/oklog v0.3.2
	github.com/opentracing-contrib/go-observer v0.0.0-20170622124052-a52f23424492
	github.com/opentracing/opentracing-go v1.0.2
	github.com/openzipkin/zipkin-go-opentracing v0.3.4
	github.com/pierrec/lz4 v1.0.1
	github.com/pierrec/xxHash v0.1.1
	github.com/prometheus/client_golang v0.9.0-pre1
	github.com/prometheus/prometheus v1.5.2
	github.com/rcrowley/go-metrics v0.0.0-20180503174638-e2704e165165
	golang.org/x/net v0.0.0-20180627171509-e514e69ffb8b
	google.golang.org/genproto v0.0.0-20180627194029-ff3583edef7d
	google.golang.org/grpc v1.13.0
)
  1. I'm able to run the .exe that has been compiled
  2. Running "vgo mod -vendor" to put dependencies in the subdirectory instead of the $GOPATH/src/mod/... (useful when using CI/CD), but I'm getting the following error and don't have any vendor/ folder created 😢 :
vgo: import "github.com/YYYYYYY/service-skeleton" ->
        import "github.com/go-kit/kit/metrics/prometheus" ->
        import "github.com/prometheus/client_golang/prometheus" ->
        import "github.com/prometheus/common/expfmt" ->
        import "github.com/matttproud/golang_protobuf_extensions/pbutil" ->
        import "github.com/golang/protobuf/proto/testdata" [E:\meta-back\gocode\src\mod\github.com\golang\protobuf@v1.1.0\proto\testdata]: open E:\meta-back\gocode\src\mod\github.com\golang\protobuf@v1.1.0\proto\testdata: Le fichier spécifié est introuvable.
vgo: import "github.com/golang/protobuf/proto/testdata" [E:\meta-back\gocode\src\mod\github.com\golang\protobuf@v1.1.0\proto\testdata]: open E:\meta-back\gocode\src\mod\github.com\golang\protobuf@v1.1.0\proto\testdata: Le fichier spécifié est introuvable.

I checked my "mod" directory where all dependencies have been download I can saw this one:
src\mod\github.com\matttproud\golang_protobuf_extensions@v1.0.0

Which is still in v1.0.0 instead of the v1.0.1 released on the repository. So I'm still thinking it could be a issue in vgo since none of the libraries are forcing "golang_protobuf_extensions" to v1.0.0.

Tell me if you need more information, and thank you for helping me 😀

@bcmills
Copy link
Contributor

bcmills commented Jun 29, 2018

I tried to reproduce the issue starting with importing github.com/go-kit/kit/metrics/prometheus, but vgo mod -sync seems to have picked the right versions.

I wonder whether https://golang.org/cl/120999 fixed the underlying bug. Could you try building vgo from commit cd1f2ee17589ba3f7f17471441f18612f864e850 or later and see if it still occurs? (If it does, I'm afraid we'll need smaller / more-explicit steps to reproduce the problem.)

@sneko
Copy link
Author

sneko commented Jul 3, 2018

@bcmills yeah it works well with the new version! Thank you so much!

Note that when my CI/CD tool try to run vgo mod -sync it gets the following error:

vgo: downloading github.com/go-kit/kit v0.7.0
vgo: verifying github.com/go-kit/kit@v0.7.0: checksum mismatch
	downloaded: h1:ApufNmWF1H6/wUbAG81hZOHmqwd0zRf8mNfLjYj/064=
	go.sum:     h1:ilPnO0G0/npDhhtCEgwufoDl6lgRcBoTbSwVD7ewQYE=

It happens directly on the first dependency that vgo tries to download. Maybe you're still working on that?

@sneko sneko closed this as completed Jul 3, 2018
@sneko
Copy link
Author

sneko commented Jul 3, 2018

@bcmills I just noticed it seems that's a common issue while synchronizing accross different platform:
#25525 (comment)

@rsc
Copy link
Contributor

rsc commented Jul 6, 2018

@sneko, now tracked as #26229, fix pending.

@golang golang locked and limited conversation to collaborators Jul 6, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

4 participants