x/build: ensure there are accessible ways to test Go pre-release versions in common CI systems #38773
Comments
dmitshur
added
Builders
|
I think this is a good idea. I get why few companies test pre-releaess on their staging or production environments: it's extra work to set up, and extra work to be careful that it can't put the business at risk. However, there's really no excuse to not run unit and integration tests against Go pre-releases. Most CI systems have an "allow failures" mode that would be a good fit here, so that pre-release failures can be flagged, but they won't be treated as critical. Since a good portion of CI systems are Docker-based (like Gitlab and Drone), a nice quick win would be to have pre-releases published on Docker Hub, such as betas and RCs. I'm actually not sure why that isn't the case already. I think another important point to keep in mind is that a general solution shouldn't require a Go version to already be installed. Some CI systems like GitHub Actions have a wide variety of software pre-installed, but that's not the norm. So I don't think we should rely on The only last thought I have here is that it should be easy for the end user to set this up, without having to update their CI config whenever a new beta or RC version is out. This is very related to #36898. That way, the Go project can do a shout-out for people to update their CI configs when the first beta is out, and we don't have to remind everyone to update it every other week as more pre-releases come out. They should all automatically be using the latest pre-release. The only requirement is that the logs print what version is used. |
The Go project and its users benefit from additional testing of pre-release versions of Go, because it leads to a stable release with fewer issues. When beta and release candidates are released, we encourage everyone to test them as much as possible and report issues that are uncovered.
There are many CI systems that are in use today (Travis CI, Circle CI, GitHub Actions, and many more). Most of them have some level of support for testing Go code with various release versions of Go, but it's not clear whether pre-release versions are supported as well.
We should ensure it's possible and easy for users to start testing against pre-release versions of Go, so that if someone is willing to do it, we can point them to an existing solution rather than hoping they'll invent it themselves.
It is generally the responsibility of the CI solution itself to offer flexibility in the testing capabilities for its users, but perhaps there are things we can do on our side to help facilitate that.
So, the work here is to check whether this is already possible and convenient for various CI systems. If not, we can see if it can and should be made better through changes on our side. The specifics will depend on the CI system and the tools it offers. It might involve coming up with scripts or configurations, taking advantage of the existing
golang.org/dl/...commands, modifying our existing API endpoints (see #34864, #36898), or something else.This is the tracking issue for that work.
(This idea was suggested to me by @hyangah, and I was reminded about it when seeing @mvdan's #36898 (comment).)
/cc @golang/osp-team @rsc @mvdan @Lyoness
The text was updated successfully, but these errors were encountered: