Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/build: use rr-project.org for running Go builds/tests on Linux? #45219

Open
mdempsky opened this issue Mar 24, 2021 · 2 comments
Open

x/build: use rr-project.org for running Go builds/tests on Linux? #45219

mdempsky opened this issue Mar 24, 2021 · 2 comments
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Unreleased

Comments

@mdempsky
Copy link
Member

We have a lot of intermittent failures on the builders, which are hard to reproduce. Evidently the Firefox project had similar struggles, so they built https://rr-project.org/, which allows recording and replaying program executions on linux/x86. Perhaps we should look into using it for our builders too?

Quickly skimming build.golang.org, it's not clear to me if there are enough flaky failures on linux/x86 to warrant the ~20% slowdown (according to their website). It seems like most flakiness is on non-linux or non-x86, unfortunately, and rr doesn't currently support any other platforms.

Maybe the regabi builder could benefit from it at least. It seems to have had some flakiness a couple weeks ago.

/cc @aclements @bcmills
@mdempsky mdempsky added Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Mar 24, 2021
@gopherbot gopherbot added this to the Unreleased milestone Mar 24, 2021
@dmitshur
Copy link
Contributor

CC @golang/release.

@tobiaskohlbau
Copy link

I'm not sure wether this is from any help. But I've stumbled upon this issue and the description from rr-project reminded me a lot about a technique I've heard before. @mdempsky stated that most flakey tests are on non Linux. Maybe it's from interest that MS has a similar technique called Time travel debugging. I've no experience with this but maybe it's worth evaluating if TTD can help with some none Linux flakiness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
4 participants
@mdempsky @tobiaskohlbau @dmitshur @gopherbot