Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

proposal: cmd/go: Declare module content in go.files config #52807

Closed
pav-kv opened this issue May 10, 2022 · 8 comments
Closed

proposal: cmd/go: Declare module content in go.files config #52807

pav-kv opened this issue May 10, 2022 · 8 comments
Labels
FrozenDueToAge GoCommand cmd/go Proposal
Projects
Proposals (old)
Milestone
Proposal

Comments

@pav-kv
Copy link

pav-kv commented May 10, 2022 •
edited

Issue

A typical non-trivial repository with a Go module contains files/directories that don't need to be packaged into the module and distributed with the code, such as:

  • User documentation, which may include large files like images and PDFs.
  • Auto-generated at build time files.
  • Configuration for other tooling, such as CI/CD pipeline for this repository.

It would be ideal to have a fine-grained control on what gets included into the module content. Currently, there are workarounds:

  1. Packages starting with . or _, or named testdata are excluded.
  2. An empty go.mod in a directory cuts this entire directory from the Go module it was in.

Approach (1) is not universal, and requires remembering the exceptions, which is an unnecessary cognitive load.

Approach (2) is more complete, but is less convenient at scale. It also does not allow an easy way to have a consolidated view of the included/excluded paths.

More discussion here: #30058.

Proposal

Introduce a top-level go.files config (next to go.mod and co) which represents the consolidated view of the Go module content. The file consists of include/exclude rules/globs/regexps, in order of their priority. This would be similar in semantics to .gitignore. For example, it can look like this:

# Exclude a bulk of non-Go files.
exclude: ^/docs/
exclude: \.pdf$
exclude: ^/scripts/
exclude: ^/cloudbuild\.yaml$

# Exclude experimental code, except the nearly good one.
include: ^/experimental/nearlygoodcode/
exclude: ^/experimental/

# Example from the issue above.
exclude: ^/node_modules/

# The catch-all default.
include: .*

The current (and the default, if go.files is not present) behaviour would be described by a file like this:

# The exceptions that represent the workaround (1).
exclude: /testdata/
exclude: /[._][^/]*$

# Include any files, except those matching the filters above.
include: .*

This format allows various approaches to controlling the content of the module:

  • default: includes everything but the default exceptions,
  • defensive: author lists all the things they want to include,
  • optimistic: author lists only the things to exclude,
  • hybrid: something in between defensive and optimistic.

Another example:

# Filter the module content to just purely Go files. The repository can have
# any structure.
include: \.go$
include: /go\.mod$
include: /go\.sum$
include: LICENSE
@gopherbot gopherbot added this to the Proposal milestone May 10, 2022
@pav-kv pav-kv changed the title proposal: cmd/go: Declare module contents in a top-level go.files config proposal: cmd/go: Declare module contents in go.files config May 10, 2022
@seankhliao
Copy link
Member

see also #42965

@pav-kv pav-kv changed the title proposal: cmd/go: Declare module contents in go.files config proposal: cmd/go: Declare module content in go.files config May 10, 2022
@seankhliao seankhliao added the GoCommand cmd/go label May 10, 2022
@ianlancetaylor
Copy link
Contributor

CC @bcmills @matloob

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) May 11, 2022
@rsc
Copy link
Contributor

rsc commented May 11, 2022

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc rsc moved this from Incoming to Active in Proposals (old) May 11, 2022
@hherman1
Copy link

This makes go modules substantially less simple for seemingly very minor convenience gains. I guess I have a hard time sympathizing with the problem as you describe it, and would rather protect my simple understanding of go modules. Perhaps there’s something I’m not understanding about the importance of having a consolidated list of excludes?

@pav-kv
Copy link
Author

pav-kv commented May 11, 2022

@hherman1 Could you clarify what your simple understanding of Go modules is?

This mechanism would be optional (default = old behaviour), so the module author could still do the simple thing. In which way would it make Go modules more difficult for you?

One of the gains would be performance: excluding bulky/non-Go clutter from the module saves time and space on the dev machine, module proxy, and all the downstream dependencies (dependent modules, other developers machines, CI workflows).

Potential target auditory for this flexible configuration are the owners of complex repositories who can't necessarily do the simple thing, e.g. put all the Go code in a single directory. This includes projects started before Go modules / semver introduction, who did not account for this in the repository structure.

@hherman1
Copy link

The addition of optional features still affects me even if I don’t personally use them because

  1. Other people use them, so I have to consider the possible existence of these files when trying to interact with other repos
  2. Documentation will have to describe them, making it harder to relearn modules after a sabbatical or teach them to new devs

I prefer uniformity over fragmentation where possible, and I can already foresee the complaints about “go modules are so confusing you have to read globs to figure out where it’s safe and not safe to write code”.

Also, it’s still not entirely clear to me how big the win here is over using the empty go.mod files you referenced.

@rsc
Copy link
Contributor

rsc commented May 18, 2022

This seems like a duplicate of #42965. Neither seems particularly likely given past discussions.

@pav-kv pav-kv closed this as completed May 18, 2022
@rsc rsc moved this from Active to Declined in Proposals (old) May 18, 2022
@rsc
Copy link
Contributor

rsc commented May 18, 2022

This proposal is a duplicate of a previously discussed proposal, as noted above,
and there is no significant new information to justify reopening the discussion.
The issue has therefore been declined as a duplicate.
— rsc for the proposal review group

@hyangah hyangah mentioned this issue May 11, 2023
Open
@golang golang locked and limited conversation to collaborators May 18, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge GoCommand cmd/go Proposal
Projects
No open projects
Proposals (old)
Declined
Milestone
Proposal
Development

No branches or pull requests
6 participants
@rsc @hherman1 @ianlancetaylor @pav-kv @gopherbot @seankhliao