Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: 'go mod why -m' and 'go mod graph' should report modules in the compatibility graph. #46366

Closed
bcmills opened this issue May 25, 2021 · 14 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@bcmills
Copy link
Contributor

bcmills commented May 25, 2021

To ease the migration path for pruned module graphs in Go 1.17 (#36460), we added a -compat flag to go mod tidy (#46141) to preserve additional checksums for Go 1.16.

Ideally, go mod why -m (and perhaps go mod graph?) should also report why those checksums are needed, since they're a bit mysterious otherwise.

CC @jayconrod @matloob

@bcmills bcmills self-assigned this May 25, 2021
@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label May 25, 2021
@bcmills bcmills added this to the Go1.17 milestone May 25, 2021
@bcmills bcmills added okay-after-beta1 Used by release team to mark a release-blocker issue as okay to resolve either before or after beta1 release-blocker labels May 25, 2021
@heschi heschi removed the okay-after-beta1 Used by release team to mark a release-blocker issue as okay to resolve either before or after beta1 label Jun 10, 2021
@gopherbot
Copy link

Change https://golang.org/cl/329529 mentions this issue: cmd/go: add a -go flag to 'go mod graph'

@bcmills
Copy link
Contributor Author

bcmills commented Jun 18, 2021

CL 329529 updates go mod graph, which I think should suffice to unblock the release.

I would also like go mod why to report this information, but since go mod why is already underpowered for transitive module dependencies (#27900) it's perhaps not critical to improve for 1.17.

gopherbot pushed a commit that referenced this issue Jun 21, 2021
For #46366

Change-Id: I8417e6e4dbb8cb56ff7afc16893a01b7bb938217
Reviewed-on: https://go-review.googlesource.com/c/go/+/329529
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@bcmills
Copy link
Contributor Author

bcmills commented Jun 22, 2021

go mod graph -go=1.16 should now show the compatibility graph, so this no longer blocks Go 1.17.

@gopherbot
Copy link

Change https://golang.org/cl/334375 mentions this issue: cmd/go: remove a duplicated word from 'go help mod graph'

gopherbot pushed a commit that referenced this issue Jul 13, 2021
For #46366

Change-Id: Ie9735027a3c4c0f4a604df30ca4d64dcdc62b45a
Reviewed-on: https://go-review.googlesource.com/c/go/+/334375
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
@bcmills
Copy link
Contributor Author

bcmills commented Jul 15, 2021

I think the remaining work is under the umbrella of #27900, so I'm going to go ahead and close this out — the portion unique to the Go 1.17 release is done.

@bcmills bcmills closed this as completed Jul 15, 2021
@liggitt
Copy link
Contributor

liggitt commented Aug 11, 2021

does this mean if we want to avoid seeing compatibility deps in go mod graph, we have to pass -go 1.16?

edit: no, that doesn't work

@liggitt
Copy link
Contributor

liggitt commented Aug 11, 2021

IIUC, go 1.17 go mod tidy adds all indirect dependencies to go.mod and makes them explicit-yet-indirect requires.

we use go mod graph to get visibility to direct and indirect dependency counts, and with go1.17 it is seeing all of the compatibility deps added by go mod tidy as direct deps... this makes go mod graph significantly less useful for understanding where dependencies are coming from

@jayconrod
Copy link
Contributor

@liggitt go mod graph -go 1.16 will show dependencies that go mod graph -go 1.17 would have pruned out.

I don't think there's any precise way to show direct / indirect dependencies with go mod graph. Even with everything pre-1.17, it won't distinguish // indirect requirements. Some go list fanciness may be the best option for this.

@liggitt
Copy link
Contributor

liggitt commented Aug 11, 2021

pre-1.17, go mod tidy would omit indirect dependencies from the go.mod file (unless the go.mod file was requiring a newer version), so go mod graph would only show edges from the main module to deps it actually used or influenced version selection for... by default with 1.17, go mod tidy seems to make go mod graph unusable for understanding which deps the main module is depending directly on or influencing version selection of

@bcmills
Copy link
Contributor Author

bcmills commented Aug 11, 2021

The go.mod file continues to mark indirect dependencies with // indirect comments, just as it did in prior Go versions. go mod graph continues to report the transitive dependency graph, just as it did in prior Go versions.

It is true that go mod graph in a go 1.17 module reports more dependency edges from the main module (to the explicitly-listed // indirect modules), and it is true that that makes go mod graph somewhat less useful for analyzing the modules that provide directly-imported packages.

I think we should address that use-case more directly (ha!), and the right way to do that is probably #40364. In the interim, you can use two passes of go list, I think:

$ PKGS=$(go list -f '{{range .Imports}}{{.}}{{"\n"}}{{end}}' -test ./... | sed 's/ \[.*\.test\]//' | sort -u)
$ go list -f '{{with .Module}}{{.Path}} {{.Version}}{{end}}' $PKGS | sort -u

@liggitt
Copy link
Contributor

liggitt commented Aug 11, 2021

was the impact on tools built on top of go mod graph considered? the 1.17 changes to go mod tidy seem to make go mod graph unsuitable for evaluating the module graph

@bcmills
Copy link
Contributor Author

bcmills commented Aug 11, 2021

go mod graph reports exactly the same things that it always reported. If your go.mod file included explicit // indirect dependencies in a pre-1.17 module, they would be included in exactly the same way that they are in a 1.17 module.

(That is: go mod graph has never been an appropriate tool for analyzing direct vs. indirect dependencies, because that requires some very specific assumptions about indirect dependencies not being upgraded.)

@liggitt
Copy link
Contributor

liggitt commented Aug 11, 2021

this closed issue is probably the wrong place to have this discussion, I'll open a new issue

@liggitt
Copy link
Contributor

liggitt commented Aug 11, 2021

opened #47648

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants