Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/exp/apidiff: support diffing entire modules #60213

Closed
noahdietz opened this issue May 16, 2023 · 7 comments
Closed

x/exp/apidiff: support diffing entire modules #60213

noahdietz opened this issue May 16, 2023 · 7 comments

Comments

@noahdietz
Copy link

Proposal

The documentation in the README indicates the following:

The current version of apidiff compares only packages, not modules.

It would be great if it did support module diffing.

I've done some hacking locally and I have a basic, green-path approach working. Before I go any further, I want to make sure this is a desirable change and that my approach is reasonable.

Basic idea

The basic approach is to add a flag -module-mode that converts the interpretation of the positional command line arguments into a module path instead of a package path, then take on the /... wildcard when loading all of the packages underneath it e.g. ./... for current dir or cloud.google.com/go/batch/... for a specific submodule. From there, compare the set of packages in each module. For the export data file workflow, we can utilize the gcexportdata WriteBundle and ReadBundle APIs.

My motivation

In google-cloud-go, we have 100+ nested modules with multiple packages in each. Most of the code is autogenerated. We get massive PRs with all of the generated code. We use apidiff to detect breaking changes in the generated code our current implementation around apidiff checks each package that changed using an individual "job", even if multiple of the packages belong to the same module. As a result, we have occasionally hit the maximum "job" limit of 256 (see googleapis/google-cloud-go#7939). Ideally, we would be able to apidiff entire modules instead of each individual package, saving us jobs to spawn, and reducing the likelihood of missing a backwards incompatible change.

cc @jba

@gopherbot gopherbot added this to the Unreleased milestone May 16, 2023
@jba
Copy link
Contributor

jba commented May 16, 2023

Thanks for wanting to improve apidiff! I'm happy that it continues to be useful to your team.

I'd be glad to review your CLs, but I no have bandwidth now to work on this myself. However, it might be simpler to, say, do 5 packages per "job". (I'm not familiar with GitHub Actions, so I don't know how feasible this is.)

I should point out that comparing modules is more than just comparing each package in the modules:

  • Adding a package is a backwards-compatible change.
  • Removing a package is not, except that if the package is "removed" because it is now part of a nested module, so that its import path doesn't change, then that is OK—client code will not break, though its go.mod file will need updating. (I think it is OK to ignore this case initially, especially since if the modules come from downloaded zips rather than a filesystem, there is no way to know about the nested module's existence.)
  • Packages under internal directories should be ignored. vendor too.
  • It should be possible to compare a v2 with a v1 module. (Thus packages should be matched by their import path relative to the module root, not their full import path.)

I feel like there are more subtleties but I can't think of them now.

Final nit: -module-mode is a long flag name. How about -m?

@noahdietz
Copy link
Author

I'd be glad to review your CLs, but I no have bandwidth now to work on this myself.

No problem, I was hoping to contribute this with your guidance :)

However, it might be simpler to, say, do 5 packages per "job". (I'm not familiar with GitHub Actions, so I don't know how feasible this is.)

For sure, I thought about "fixing" this in our GitHub CI itself, but also felt like this tool and others would benefit from the feature. And I wanted to give back to this tool in some way.

Adding a package is a backwards-compatible change.

Yep, and it should be reported when not running in -incompatible mode.

Packages under internal directories should be ignored. vendor too.

Yeah I was hoping to implement this in a way that would leverage the existing logic for ignoring internal.

Removing a package is not [compatible], except that if the package is "removed" because it is now part of a nested module...I think it is OK to ignore this case initially,

Great point, and I agree it would be best to add this later on for the reasons you mention.

It should be possible to compare a v2 with a v1 module

Good call.

-module-mode is a long flag name. How about -m?

😄 Yep, you got it!

My current PoC heavily relies on extending logic in x/exp/cmd/apidiff/main.go to essentially use the same apidiff.Changes API to compare individual packages.

Question: Do you think it would be better to implement that "module comparison" logic in the x/exp/apidiff "library" instead as a ModChanges API, for example? I'm not sure what the right separation of concerns is between the two, because there will be "module mode" logic in main.go no matter what, and the apidiff.Changes API works great comparing two packages...

@noahdietz
Copy link
Author

  • Adding a package is a backwards-compatible change.
  • Removing a package is not [compatible]
  • Packages under internal directories should be ignored. vendor too.
  • It should be possible to compare a v2 with a v1 module. (Thus packages should be matched by their import path relative to the module root, not their full import path.)

I've got these figured out (sans nested module creation edge case) with a solution based entirely on fancy/hacky (depends on your perspective) logic in x/exp/cmd/apidiff 😄. So I think I'm well positioned to move things into x/exp/apidiff as a ModuleChanges API if that is the preferred approach.

@jba
Copy link
Contributor

jba commented May 16, 2023

All sounds good. I'd prefer if ModuleChanges was in x/exp/apidiff.

@noahdietz
Copy link
Author

CL is here: https://go-review.git.corp.google.com/c/exp/+/495635

First time contributing so feel free to over communicate about best practices here :)

@jba
Copy link
Contributor

jba commented May 17, 2023

Looks great! Commented on the CL.

@noahdietz
Copy link
Author

Maybe automation hasn't gotten around to closing this but the CL was submitted so we can close this. Thanks for all of the help @jba

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Status: Declined
Development

No branches or pull requests

3 participants