Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: add golang.org/x/internal repository #31089

Closed
bcmills opened this issue Mar 27, 2019 · 9 comments
Closed

proposal: add golang.org/x/internal repository #31089

bcmills opened this issue Mar 27, 2019 · 9 comments
Labels
FrozenDueToAge modules NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. Proposal
Milestone

Comments

@bcmills
Copy link
Contributor

bcmills commented Mar 27, 2019

Some of the code in the golang.org/x repositories is shared between multiple repositories: for example, the program to generate the lookup tables in golang.org/x/net/idna needs access to a number of utility packages that are shared with (and currently internal to) golang.org/x/text, including Unicode tables containing large constant strings and arrays.

I suspect that there may be similar opportunities for factoring out internal packages from x/build and x/tools.

Ideally, we should be able to share that data through some common module dependency, rather than by copying code from one module to another. And we have a mechanism for that: the internal path component works in module mode, too, so we could add a golang.org/x/internal component and move any shared dependencies there.

I propose that we add a new repository, golang.org/x/internal, to contain these sorts of project-internal cross-repo dependencies.

Note that the packages within x/internal may need to import from a large number of the other golang.org/x modules, so we would need to be careful to minimize its dependencies. This may be a reasonable use-case for a multi-module repository, since packages within x/internal may be only very loosely coupled to each other.

CC @bradfitz @mpvl @dmitshur @jayconrod @matloob @hyangah @ianthehat @rsc

@bcmills bcmills added Proposal NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. modules labels Mar 27, 2019
@bcmills bcmills added this to the Unreleased milestone Mar 27, 2019
@matloob
Copy link
Contributor

matloob commented Mar 27, 2019

Speaking for myself, I think the behavior that golang.org/x/internal would create packages internal to the "golang.org/x/" "namespace" is surprising. I'm worried that our users and contributors will get confused by this too.

@bcmills
Copy link
Contributor Author

bcmills commented Mar 27, 2019

@matloob, why would that be surprising? It's the same way the internal/ path component works everywhere...

@matloob
Copy link
Contributor

matloob commented Mar 27, 2019

@bcmills My personal intuitive model is that internal/ is used within a repo or module, so it's surprising to me according to that model. I wonder if there are others who think with that model.

@bradfitz
Copy link
Contributor

I'm fine with this as long as it's multi-module.

If this is going to be a single module 'util' dumping ground then I'm firmly opposed.

@mpvl
Copy link
Contributor

mpvl commented Mar 28, 2019

At least in the x/text case it seems to solve only half the problem. Most of the cross-repo generation is from x/text to core, which this would not solve. This could be mitigated with "vendoring", but not for everything (e.g. core unicode). Also, most of these packages, like unicode/norm, would ideally live as a stripped down version in core that x/text can depend on (this can reduce table size significantly.) The same perhaps for net/idna. This would diminish the benefit of x/internal greatly from x/text's perspective.

@andybons
Copy link
Member

Per discussion with @golang/proposal-review

What are the initial set of packages in this repo and how would they help? Would they each have their own module?

@rsc
Copy link
Contributor

rsc commented May 28, 2019

I think a "golang.org/x/internal" repo is a type error. I understand why others would think not and have done exactly this, but if you have to put version numbers on it and manage references to it from other repos, it's not particularly internal anymore. I also worry about the tendency to think "oh we'll just put this in internal for now instead of thinking about an API we are willing to commit to", which would lead to the internal repo being a bit of a dumping ground. It would also end up with bits of lots of different subject domains and cut useful git history out of the more specific repos. All of this cautions against an internal repo. I'd rather address case-by-case specific examples as they come up.

@bcmills
Copy link
Contributor Author

bcmills commented May 29, 2019

The specific example I had in mind was x/text, but Marcel notes that it would not actually help for that. On that basis, I'll withdraw this proposal until/unless we find a more compelling use-case.

@bcmills
Copy link
Contributor Author

bcmills commented May 29, 2019

I also worry about the tendency to think "oh we'll just put this in internal for now instead of thinking about an API we are willing to commit to", which would lead to the internal repo being a bit of a dumping ground.

Even worse, an internal repo doesn't actually let us avoid committing to an API: the internal repo would have to have a stable API to maintain forward-compatibility, and at that point I guess we may as well make the packages public rather than internal.

@bcmills bcmills closed this as completed May 29, 2019
@golang golang locked and limited conversation to collaborators May 28, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. Proposal
Projects
None yet
Development

No branches or pull requests

7 participants