Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: cannot vendor protoc-gen-go if not imported in module code import #31239

Closed
codeliger opened this issue Apr 3, 2019 · 4 comments
Closed
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@codeliger
Copy link

codeliger commented Apr 3, 2019

What version of protobuf and what language are you using?
Version: v1.3.1

What did you do?

go build -mod=vendor sourcefile.go

What did you expect to see?
I expected my go.mod dependency of github.com/golang/protobuf to be included in the vendor folder, even though it doesnt exist as an import in any of my go files because i need the packages proto and protoc-gen-go for using protoc to generate .proto files.

I should be able to include a vendored dependency thats not imported by my go source code files to allow my dockerfile to build vendored .proto files without pregenerating them and commiting them to a repo. The reason for this is to ensure other users of my library do not ignore the .proto file and just use *.pb.go which could possibly be out of sync with its related .proto. If they have to use protoc everytime they pull my .proto file from my package then they are guarunteed to be in sync.

What did you see instead?

/home/$USER/dev/go/src/github.com/username/modulename/vendor/github.com/golang/protobuf/proto no such file or directory

go.mod

go 1.12

require (
	github.com/confluentinc/confluent-kafka-go v0.11.6
	github.com/go-redis/redis v6.15.2+incompatible
	github.com/golang/protobuf v1.3.1 // indirect
	github.com/onsi/ginkgo v1.8.0 // indirect
	github.com/onsi/gomega v1.5.0 // indirect
	github.com/pkg/errors v0.8.1 // indirect
	github.com/username/modulename v0.0.0-20190403143051-76fe0c9dbba9
	go.uber.org/zap v1.9.1
)

module github.com/username/modulename

Extra information:

Including this import github.com/golang/protobuf/protoc-gen-go in a empty go file called tools.go throws a go error saying that protoc-gen-go has a main method so its a compiled tool and not a shared library.

This prevents any type of import being allowed when trynig to vendor protoc code.

When using -mod=vendor the mod.go file is completely ignored so there is not even a way to put an indirect dependency in there.

The only option is to not use vendoring, and that requires me to put my gitlab https auth credentials directly in the docker file or environment variable so i can download gitlab dependencies which is not ideal.

@codeliger codeliger changed the title cannot vendor protoc-gen-go if not imported in module .go code cannot vendor protoc-gen-go if not imported in module code import Apr 3, 2019
@andybons andybons changed the title cannot vendor protoc-gen-go if not imported in module code import cmd/go: cannot vendor protoc-gen-go if not imported in module code import Apr 4, 2019
@andybons
Copy link
Member

andybons commented Apr 4, 2019

@bcmills @jayconrod

@andybons andybons added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Apr 4, 2019
@andybons andybons added this to the Unplanned milestone Apr 4, 2019
@bcmills
Copy link
Contributor

bcmills commented Apr 4, 2019

The model for go:generate is that you must check in the generated source code. Per https://blog.golang.org/generate:

if the containing package is intended for import by go get, once the file is generated (and tested!) it must be checked into the source code repository to be available to clients.

If you check in the generated .proto files, then go mod vendor will copy in github.com/golang/protobuf/proto.

To ensure that the files remain in sync, you can write a unit-test that invokes protoc and compares the resulting files, or have the CI system re-run go generate and check for diffs.

@bcmills
Copy link
Contributor

bcmills commented Apr 4, 2019

See also the // +build tools idiom described in #25922.

@bcmills
Copy link
Contributor

bcmills commented Apr 4, 2019

Closing as a duplicate of #25922.

@bcmills bcmills closed this as completed Apr 4, 2019
@golang golang locked and limited conversation to collaborators Apr 3, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

4 participants