Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/go/packages: can't distinguish dynamically-generated sources from checked-in ones #27909

Open
bcmills opened this issue Sep 27, 2018 · 3 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Refactoring Issues related to refactoring tools Tools This label describes issues relating to any tools in the x/tools repository.
Milestone

Comments

@bcmills
Copy link
Contributor

bcmills commented Sep 27, 2018

When using go list -test, we can distinguish user-provided sources from generated ones based on whether the paths are absolute, as documented here (emphasis mine):

By default, the lists GoFiles, CgoFiles, and so on hold names of files in Dir (that is, paths relative to Dir, not absolute paths). The generated files added when using the -compiled and -test flags are absolute paths referring to cached copies of generated Go source files. Although they are Go source files, the paths may not end in ".go".

Unfortunately, golang.org/x/tools/go/packages removes that distinction by making all of the GoFiles it returns absolute, without apparently preserving enough information to undo that transformation. That makes it very difficult to use go/packages to write tools that transform or otherwise refactor user-provided source code, such as the one in CL 137076.

CC @matloob @alandonovan @ianthehat @rsc

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 27, 2018
@gopherbot gopherbot added this to the Unreleased milestone Sep 27, 2018
@alandonovan
Copy link
Contributor

The assumption "absolute <=> generated" doesn't hold for other build systems such as Bazel, in which a package may contain a mixture of sources from different directories. Indeed, it doesn't even hold for go list in the presence of modules, since source files may reside in the module cache.

The go/packages API could in principle record which files are generated (and at one point it did), but we removed on the grounds that refactoring tools should decide which files to edit based on the "DO NOT EDIT" comment convention proposed in github.com//issues/13560.

@bcmills
Copy link
Contributor Author

bcmills commented Sep 27, 2018

The DO NOT EDIT convention relies on three assumptions:

  1. Generated code is not checked in to the repository.
  2. Users never want refactoring tools to apply even trivial edits to files within the repository that say DO NOT EDIT.
  3. All generated files actually follow the convention.

I don't know about (3), but I do not believe that (1) or (2) holds outside of Blaze/Bazel.

@bcmills
Copy link
Contributor Author

bcmills commented Sep 27, 2018

In particular, you could imagine a workflow that goes:

  1. Refactor the code using some tool, including arbitrary package moves.
  2. Re-run the generators for all generated files that were touched.

Step 1 should not require pruning or excluding checked-in generated files, since they may be needed to type-check packages at any intermediate state.

@bcmills bcmills changed the title x/tools/go/packages: can't distinguish generated sources from user-provided ones x/tools/go/packages: can't distinguish dynamically-generated sources from checked-in ones Sep 28, 2018
@gopherbot gopherbot added the Tools This label describes issues relating to any tools in the x/tools repository. label Sep 12, 2019
@adonovan adonovan added the Refactoring Issues related to refactoring tools label Apr 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Refactoring Issues related to refactoring tools Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Development

No branches or pull requests

4 participants