VGO Caching Strategy and Enterprise Development

Hello,
I wanted to bring up some concerns I have around the vgo cache and doing Enterprise / Lots of Micro services development. I work in an environment where I have a lot of services which are only related via their RPC Apis, Twirp in our case.

Can't use internet go repositories directly
Like many enterpries, pulling code live from github or the internet is not really allowed:

• I can't control when my access to github / etc is comprimised.
• I can't have my builds fail when github is down.
• I have to vet / sign off on all libraries in use and vet their licenses.
• I need to have security review and audits of all the point releases of libraries I use.

Generally somthing like this is done via a tool like Artifactory and only allowing vetted libraries into it.

This can be solved with firewall in some cases. However, tools like maven solve this by having you list repositories and only hitting those with a default of mvncentral (I think) if no repos are configured.

Versions of libraries aren't the same across all projects
I may do a bad job of explaining this but here goes. Often when developing with lots of services you will have many parallel unrelased versions of libraries. So v1.0.3 would mean completely different things depending on using them. Lets say for instance we have a company library loglib that is currently released (internally) to v1.0.2. Now lets say there are two projects who are consuling loglib, chatty and watchy. Chatty has an issue with loglib that they have to fix and are testing that branch. Obviously this is v1.0.3b. Watchy adds a new feature that they also want to bake, this is also I guess v1.0.3b but is totally different code. Whoever gets their code out first really is the one who releases 1.0.3, but internally they both need to reference unreleased versions of the package. Worse they may need to share these unreleased versions with other teams.

Now even worse, lets say I'm a dev for loglib, and working on both watchy and chatty fixes. I need to have a go development env for watchy setup, and a different one for chatty. If I use unified system cache then these 2 projects will be stepping on each other and really these unreleased versions don't resolve across gopaths.

This may sound wierd but artifactory lets you setup project specific views into packages.

In general it might make more sense to tie the cache to the "compilation directory".

Tho I do think that the ability to setup the cache directory with a envvar is a workaround. Just may be awkward and lead to many issues.

Go Version Dependent?
Also a mistake I think glide makes is that the cache is tied to the version of go and system wide which causes major issues when swapping back and forth between go versions.

Inability to provide your own cache system
At my current work and at some previous places we needed the ability to control caching ourselves for various reasons, such as the multi-version one above. Another was at a game studio where we had distributed build engines. Providing a standard cache layout structure or a standard interface for the cache would allow specialized cache systems and resolvers to be placed in where needed.