It's common for Go tests that require certain capability or dependency to skip themselves if the environment doesn't meet that requirement. For example, see the various MustHave* helpers in internal/testenv.

Perhaps it'd be possible for watchflakes to detect instances of patterns where an existing test goes from passing to skipping, and report them in a useful way (i.e., a high enough signal to noise ratio). It might only be viable if it's possible to filter out cases where such changes are intended, e.g., a CL that directly adds a t.Skip(…) line to the test.

Filing this as a tracking issue.