cmd/go: [modules + integration] go mod buildrequires, list the build requirements of a set of unpacked modules #31300
Labels
modules
NeedsInvestigation
Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
This report is part of a series, filled at the request of @mdempsky, focused at making Go modules integrator-friendly.
Please do not close or mark it as duplicate before making sure you’ve read and understood the general context. A lot of work went into identifying problems points precisely.
Needed feature
Go needs an official
go mod buildrequires
command that processes a set of unpacked Go modules and returns the list of other Go modules that need to be present for this set to be processed by the rest of the go toolchain.This kind of analysis is required to populate build environments in CI/CD systems accurately, either to complete the set with other modules, or make sure it is self-hosting before a CI/CD run.
go mod buildrequires
could be implemented as ago mod graph
mode, via specific option flags, or as a separate subcommand.Constrains
require
,exclude
andreplace
of all thego.mod
files in the set,go.mod
individual requirementsgo.mod
filesystem paths (as produced bygo mod discover
in issue cmd/go: [modules + integration] go mod discover, discover a set of unpacked modules within a directory #31299), one or several directory paths (similar to the directory paths defined in issue cmd/go: [modules + integration] go mod discover, discover a set of unpacked modules within a directory #31299), or a mix of both¹ The experimental code is probably horrible. That's not the point. Most of its functions are generic and should have been provided by generic Go tooling in the first place.
Motivation
the Go modules design allows the splitting of projects in multiple modules and nested submodules. Therefore, the CI/CD integration unit for Go, is now a source state that may contain a variable number of coordinated Go modules. The go toolchain should not assume that the inputs/outputs of a CI/CD run are a single module declared in a single
go.mod
file.robust CI/CD systems will cut internet access during a run to ensure the reproducibility and security of run results. A CI/CD build environment needs to be populated with all the code the run needs, before the run starts. Later
go get
calls won’t work.populating a CI/CD build environment with more code, than the strict minimum the run will need, gets prohibitively expensive in run time, for busy build farms with a huge list of jobs to run. Therefore the build requirement list needs to be as cut down as possible, allowing to remove:
missing modules, identified by the
go mod buildrequires
call, will typically be populated from the organisation baseline. Because parts of this baseline can be shared between organisation projects, it won't be mirrored in each project VCS in a vendor directory.missing module population will make use of recent CI/CD improvements, driven by the needs of Rust, Go and Python ³
adding new modules to the organisation baseline is a lot of work². It requires
therefore, there was violent disbelief and rejection among consulted integrators⁴, of any Go module setup, that forced them to process new modules just because the corresponding imports exists in module parts they have no use for:
² Initial import represents the bulk of the work, keeping the module updated once imported is a lot more reasonable.
³
go mod buildrequires
to its maximum effectgo mod buildrequires
⁴ Sometimes, interrupting before the end of the presentation of Go module changes
The text was updated successfully, but these errors were encountered: