x/pkgsite: devtools/cmd/seeddb -keep_going defers termination until all modules have had a fetch attempt #48587
Labels
FrozenDueToAge
NeedsInvestigation
Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
pkgsite
Milestone
Current behaviour
When
devtools/cmd/seeddb
encounters any errors at all, it immediately aborts:This is because the run function uses an errgroup.Group, for early termination:
https://cs.opensource.google/go/x/pkgsite/+/fc5196d5:devtools/cmd/seeddb/main.go;l=91-114
Since historical versions of github.com/quantcast/g2 are broken, you cannot seed with “github.com/quantcast/g2@all”. In addition, if there is any module that is broken in the seed listing, it must be excluded and the seeddb command must be run again. In our codebase, this seems quite common due to early confusion around the “module” directive in go.mod files.
Desired behaviour
When seeding with a list of modules, an operator probably wants to fetch as many valid modules as possible. Understandably, this is the opposite behavior from what a developer wants.
I suggest a -keep_going flag, inspired by GNU make’s --keep-going, which would collect errors until the program has tried to fetch every version:
Workaround
Users may manually specify valid versions to seed, but that requires them to know which versions are valid. For our example repository, our seed.txt contains:
The text was updated successfully, but these errors were encountered: