Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/build: automated testing of popular, open source Go projects #38803

Open
mdempsky opened this issue May 1, 2020 · 4 comments
Open

x/build: automated testing of popular, open source Go projects #38803

mdempsky opened this issue May 1, 2020 · 4 comments
Labels
Builders x/build issues (builders, bots, dashboards) FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@mdempsky
Copy link
Member

mdempsky commented May 1, 2020

Basic idea is that we'd automatically build and test a large, representative sampling of open source Go projects, to help catch regressions during the development cycle that aren't caught by tests in the Go repo itself.

Some thoughts:

If running tests is a concern for resource/security concerns, even just verifying that projects still build would be useful data sometimes. E.g., cmd/cgo changes have a high frequency of breaking builds.

I think ~nightly testing would be adequate, but even weekly would probably be useful.

/cc @dmitshur
@mdempsky mdempsky added this to the Backlog milestone May 1, 2020
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label May 1, 2020
@mdempsky
Copy link
Member Author

mdempsky commented May 2, 2020

The thing that got me thinking of this was the Coq proof assistant project offers a CI service where projects that commit to certain maintenance responsibilities will received automated CI testing and Coq developer support to deal with upstream changes: https://github.com/coq/coq/blob/master/dev/ci/README-users.md

Of course, circumstances for Go are somewhat different. In Go we have to deal with flaky tests, and we also have stronger backwards compatibility guarantees (I think). But I think the basic principles of being more proactive in watching for downstream breakage is still applicable to Go.

@dmitshur dmitshur added FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels May 2, 2020
@dmitshur
Copy link
Contributor

dmitshur commented May 2, 2020

Thanks for filing this @mdempsky.

/cc @cagedmantis @toothrot @andybons

@cherrymui
Copy link
Member

I don't remember whether @dr2chase 's daily bent run includes tests.

cc @dr2chase

@dr2chase
Copy link
Contributor

dr2chase commented May 4, 2020

My daily run does not include tests, but it is capable of tests, and I ran them for a recent CL.

bent doesn't do a good job (yet) of quantifying test results -- yet another possible intern project, I suppose, though it is a small one (and bent is internally somewhat cruddy).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Builders x/build issues (builders, bots, dashboards) FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
5 participants
@mdempsky @dmitshur @dr2chase @gopherbot @cherrymui