Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: consider some type of tooling to easily test multiple major versions in a single binary #27514

Closed
thepudds opened this issue Sep 5, 2018 · 2 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@thepudds
Copy link
Contributor

thepudds commented Sep 5, 2018

What version of Go are you using (go version)?

Go 1.11

Brief background

Semantic import versioning means multiple major versions of a package might be included in a single build (e.g., v1 and v2 might both be in a single build, or perhaps something like v1, v7, and v15 in a single build). This approach helps with multiple aspects of the modules system, including helping with diamond dependency problems, or allowing a major version to be implemented as a shim around a different major version, etc.

Some related snippets from the "Avoiding Singleton Problems" section of the https://research.swtch.com/vgo-import post, which outlines how this is an improvement over multiple copies due to vendoring and sketches some possible approaches to avoiding conflicts across major versions of a module:

One common objection to the semantic import versioning approach is that package authors today expect that there is only ever one copy of their package in a given build. Allowing multiple packages at different major versions may cause problems due to unintended duplications of singletons. An example would be registering an HTTP handler. If my/thing registers an HTTP handler for /debug/my/thing, then having two copies of the package will result in duplicate registrations, which causes a panic at registration time. Another problem would be if there were two HTTP stacks in the program. Clearly only one HTTP stack can listen on port 80; we wouldn't want half the program registering handlers that will not be used. Go developers are already running into problems like this due to vendoring inside vendored packages.

Moving to vgo and semantic import versioning clarifies and simplifies the current situation though. Instead of the uncontrolled duplication caused by vendoring inside vendoring, authors will have a guarantee that there is only one instance of each major version of their packages. By including the major version into the import path, it should be clearer to authors that my/thing and my/thing/v2 are different and need to be able to coexist. Perhaps that means exporting debug information for v2 on /debug/my/thing/v2. Or perhaps it means coordinating. Maybe v2 can take charge of registering the handler but also provide a hook for v1 to supply information to display on the page. This would mean my/thing importing my/thing/v2 or vice versa; with different import paths, that's easy to do and easy to understand. In contrast, if both v1 and v2 are my/thing it's hard to comprehend what it means for one to import its own import path and get the other.

Suggestion

This is not fully fleshed out idea, but recording here in case it triggers others in the community to comment with more precise thoughts.

Ideally the go tooling would make it easy to test multi-major version scenarios to help catch any related problems with global registries, global handlers, singletons, etc.

Consider some type of testing option that automatically creates a single binary containing all major versions of a module and runs their tests in parallel from within a single binary. This would not catch all issues, but might help catch the most common issues that arise from having more than one major version in a single build.

Perhaps this could be a go test feature, or part of go release (#26420), or ____. Perhaps this could be for all major versions, as well as perhaps for a subset of major versions (e.g., perhaps a set of manually selected major versions, or perhaps the tooling could automatically selecting different subsets of major versions).

Ideally it could done in a way such that it is very easy for a module author.

Alternatively, perhaps this is just a documented recommendation with the steps needed to do this manually.

@gopherbot, please add label modules.

@bcmills
Copy link
Contributor

bcmills commented Sep 6, 2018

Ideally the go tooling would make it easy to test multi-major version scenarios to help catch any related problems with global registries, global handlers, singletons, etc.

I would go a step further: ideally, all major versions except for the latest would be simple forwarding shims (to the next major version). Then the test of v1 necessarily depends on v2, because v1 itself is just a wrapper around v2.

I think I'm going to take a stab at a tool to automatically construct such shims, as sort of a dual to @marwan-at-work's mod tool.

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 6, 2018
@bcmills bcmills added this to the Go1.12 milestone Sep 6, 2018
@bcmills
Copy link
Contributor

bcmills commented Oct 25, 2018

I think I'm going to take a stab at a tool to automatically construct such shims

https://golang.org/cl/137076 contains an initial version of that tool.

In general, you can use blank-imports to pull in all of the major versions you care about to at least test that they can compile and init together.

Beyond that, I don't think there's an obvious path forward on this issue for now. If we discover specific pain points — or have ideas for specific tools — we can open separate issues for those.

@bcmills bcmills closed this as completed Oct 25, 2018
@golang golang locked and limited conversation to collaborators Oct 25, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

3 participants