Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/build/cmd/watchflakes: failures posted with misleading timestamps #62116

Open
bcmills opened this issue Aug 17, 2023 · 3 comments
Open

x/build/cmd/watchflakes: failures posted with misleading timestamps #62116

bcmills opened this issue Aug 17, 2023 · 3 comments
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Unreleased

Comments

@bcmills
Copy link
Contributor

bcmills commented Aug 17, 2023

In #62095 (comment), watchflakes posted a number of failures in x/ repos with timestamps dating back to June, like:

2023-06-21 11:18 linux-amd64-racecompile mod@baa5c2d0 go@2fcfdb96 go/internal/gccgoimporter [build] (log)

That timestamp is misleading: go@2fcfdb96 was committed 2023-08-17, not 2023-06-21.
(The timestamp is for mod@baa5c2d0, ignoring the go commit time.)

Those kinds of misleading timestamps can easily cause an issue to be mistriaged: a sudden new failure starting two months ago is probably a misconfigured or recently-added builder, whereas a sudden widespread recent failure is almost certainly a bad commit that should be rolled back.

(attn @cherrymui)
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label Aug 17, 2023
@gopherbot gopherbot added this to the Unreleased milestone Aug 17, 2023
@bcmills bcmills mentioned this issue Aug 17, 2023
Closed
@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 17, 2023
@bcmills
Copy link
Contributor Author

bcmills commented Oct 25, 2023

#63730 (comment) is another one with misleading timestamps, which would prevent it from being correctly matched to #63739.

@cherrymui
Copy link
Member

Given the current data model with the coordinator, this is rather difficult to do. LUCI will provide us better data. As we're going to switch to LUCI, we will solve it with LUCI.

@bcmills
Copy link
Contributor Author

bcmills commented Oct 25, 2023

I agree that the coordinator makes this difficult. FWIW, fetchlogs uses maintner for this information:
https://cs.opensource.google/go/x/build/+/master:cmd/fetchlogs/fetchlogs.go;l=143-158;drc=a3f1e410a493bb04cbc55bb7c99b81457f6cae66

It's not the most efficient (maintner is kind of a memory hog and takes a short while to load from scratch), but it at least sidesteps the need to make changes to the coordinator.

If watchflakes has enough network access to use maintner, I expect this could have a workaround in place with less than a day's work.

@bcmills bcmills mentioned this issue Dec 11, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
4 participants
@dmitshur @bcmills @gopherbot @cherrymui