More relevant details are available in my message in a related thread on golang-nuts. In particular, the tool we've developed in-house was actually open-sourced by us a few years ago at: https://github.com/zpas-lab/vendo/; notably, use-cases.md might be interesting. Unfortunately, we didn't have enough resources to even write a proper readme, or advertise it well enough on golang-nuts.

@rsc has responded to these concerns here, requesting feedback:

https://groups.google.com/forum/m/#!topic/golang-dev/FTMScX1fsYk

@kardianos Yep, and I've already responded, and linked to the thread and my response from my comment above.

@akavel, we've decided to keep support for vendor in the top-level directory of the overall module being built (not in dependencies). If your company monorepo stays a mono-module, then you'll need to mv _vendor vendor, but otherwise it should work. Probably you should save vendor for clean copies of available external dependencies, though, and move the code.google.com stuff to somewhere else in your tree (maybe a top-level external directory, with appropriate import path changes).

Since we are keeping vendor at the top level of the tree, I'll mark this as closed.

I hope and assume, that this should let us have fully reproducible builds, by setting GOPROXY to some nonexistent address, and thus disabling downloading of any new packages, leaving vendor as the only source for packages. That makes it sound usable for us, thanks!

At this point, a few things are still not clear to me, however:

1. @rsc Why the suggestion to move code.google.com stuff somewhere else? I don't grasp reasoning behind this, can you please try to give some more details, so I could better understand? For now, the main advantage and goal of the vendor directory in my eyes was to be able to keep reproducibility by safely storing a pinned copy of dependencies, protected from external circumstances which could try to influence them. It's not clear to me what changes are planned which could interfere with this understanding?
2. In a "mono-module", do I understand correctly, that I can still build multiple executables from its multiple subpaths, as long as they share a common root with a common go.mod?
3. One point I didn't express in the above issue, but I did in the linked thread on golang-nuts, is a hope for a tool which could automatically manage this vendor directory for us, based on the new module-based ecosystem, in a conservative and controlled way. I didn't find any communication in this regard yet; is there a chance for this to be part of official vgo? Or is it maybe expected to be delegated to some proxy tool? Or was it forgotten? Is it enough that I mentioned this on golang-nuts, or should I better open a new issue to make sure this concern won't get lost?

Hi @akavel,

Yes, there will be some way to say "no network allowed".

1. Unless you want to maintain the vendor directory entirely by hand (you really don't), the code.google.com packages are going to confuse every tool that tries to help you. In particular "vgo vendor" is going to assume it can just rm -rf vendor and start over, and when it does that it won't be able to re-fetch the code.google.com packages.

2. Yes.

3. Yes, I think "vgo vendor" will do this. But again that will depend on your vendor dir not diverging from what's actually on the network (and code.google.com isn't, anymore).

@rsc Thanks for the reply! And thanks for not letting this slip. I hope you'll see my follow-up below too.

Re 1. & 3.: Hmm, then this concerns me with regards to our use case. As I see it, this will put a worrying burden on anyone in our org who'd want to add a new dependency to the repo, making this a punishing operation, which I see as bad incentive. I'll try to explain:

First of all, just to repeat for background, the main value I see in vendor directory is that of reliable reproducibility. As you recently communicated, this stays, and that makes me super happy. In other words: if we vendor gopkg.in/foobar today, and gopkg.in closes doors next week, we can still build our app from our repo even 1 year away from now. Reproducibility achieved, we can sleep safe!

However, let's take a look now what this means for Joe the Programmer working at our company in one particular use case. Joe writes a new app in our module, and wants to use influxdb, so he adds a dependency on influxdb_client package. We want to vendor it, so Joe runs "vgo vendor". Assuming it can rm -rf vendor and start over, there are suddenly some unwelcome things that can happen:

• a) gopkg.in (alias code.google.com) closed doors last week, and suddenly Joe is blocked from adding completely unrelated package influxdb_client! As long as he doesn't add the new dependency and run "vgo vendor", everything works and builds OK. But if he does try, he's punished for this with having to take burden of resolving a completely unrelated external problem/downtime. In other words, the cost of coping with Internet being unstable is "amortized" on the first unlucky Joe who needs to do "vgo vendor".
• b) in the meantime, because of Reasons, the author of gopkg.in/foobar may have decided to move the repo's v1 tag to a different commit. Suddenly, although go.mod stays the same, the vendor/gopkg.in/foobar directory shows some diffs in our repo, and Joe has to investigate. Again, this is something completely unrelated to influxdb_client.

Arguably, (b) is somewhat similar to a situation when MVS would kick in and upgrade some package. But there's a subtle difference, in that for MVS this is justifiable as a consequence of adding influxdb_client, so the burden is acceptable as reasonable. In case (b), it's accidental, unexpected and "unjust punishment". But regardless of (b), the (a) is what worries me most.

For now, one workaround I imagine could help bear with (a) and (b) would be to write a proxy and try to implement the logic we want in there (probably something like "copy our vendor to tmp, then serve any packages first from there, or from net if not found"). This looks like it could mitigate the issue to some extent. However, I still see this as an extra cost required of Joe whenever he wants to add a new dependency, and thus needs to run "vgo vendor" — that he now needs to remember to start the proxy before running "vgo vendor". FWIW, I imagine we could at least script this for him, so that instead of "vgo vendor" he runs "vendorize.sh", which does "setup proxy; VGOPROXY=localhost:1234 vgo vendor; tear down proxy". (And most probably actually: "setup proxy; for each GOOS, GOARCH in [...]; do GOOS=... GOARCH=... VGOPROXY=... vgo vendor; done; tear down proxy"; I hope multi-GOOS operation gets reasonably supported.)